MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
Jee Jee Li
e56b1603af
[Kernel]Add norm fused kernel for Step-5 Preview and Muse Spark Open ( #62132 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-17 01:53:00 -07:00
Wentao Ye
8e25bbca19
[Refactor] Remove dead code multiple places ( #61843 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-16 15:01:07 +00:00
Jack Hu and Douglas Lehr
ef38e8d006
[Model][Hardware][AMD][Kernel] Part 2/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_ATTN ( #42755 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Signed-off-by: Jack Hu <jingxujaster@hotmail.com >
Signed-off-by: Jack Hu <jack.hu@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-15 17:10:14 -07:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
Tanish Malekar and Claude Sonnet 4.6
e26264f3ef
[Kernel] Implement CUDA kernel for ReLUSquaredActivation (relu^2) ( #39058 )
...
Signed-off-by: Tanish Malekar <tanishmalekar32@gmail.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-07-12 19:18:03 -07:00
Fadi Arafeh
f1073c050c
[CPU][BugFix] Multiple fixes to w4a8_int8 CPU MoE path ( #46739 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
2026-07-06 05:39:20 +00:00
TJian
de2a8fc042
[ROCm] [PyTorch] Move to stable abi since ROCm upgraded to torch 2.11 ( #47128 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-07-02 18:34:07 +08:00
almayne and Li, Jiang <jiang1.li@intel.com>
89e99202f2
[CPU][Perf]Added tanh AOR for faster gelu activations. ( #44639 )
...
Signed-off-by: Anna Mayne <anna.mayne@arm.com >
Signed-off-by: almayne <anna.mayne@arm.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-06-30 23:24:40 -07:00
Wentao Ye
cc7981599e
[Refactor] Remove dead kernel code ( #46405 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-06-25 18:09:56 -07:00
Chris Leonard
b9a7cd464c
[12/n] final _C library kernel migration ( #45415 )
2026-06-19 06:57:26 -07:00
arghyadeep sarkar
93bbe94d3a
[Kernel] Add weightless RMSNorm CUDA kernels for has_weight=False ( #41430 ) ( #44109 )
...
Signed-off-by: hello-args <args.sarkar@gmail.com >
2026-06-16 23:45:55 -07:00
0a1c5034f5
[Model] Add MiniMax M3 support ( #45381 )
...
Signed-off-by: youkaichao <youkaichao@gmail.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-06-16 01:01:25 +08:00
TJian
aa6fb8a329
[Bugfix] [ROCm] [Critical] fallback to regular abi for ROCm ( #44648 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-06-05 15:51:17 +00:00
Chris Leonard
56aff0dd15
[10/n] Migrate cuda_view and silu_and_mul_per_block_quant kernels to torch stale ABI. ( #44334 )
2026-06-04 20:14:43 -07:00
Chris Leonard and Shengqi Chen
59d0236193
[10b/n] Migrate custom all-reduce, DeepSeek V4 fused MLA, MiniMax reduce-RMS, and MXFP8 MoE to libtorch stable ABI ( #44365 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Signed-off-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-06-04 00:29:46 +08:00
22a58640b4
[9/n] Migrate attention and cache kernels to torch stable ABI (continued) ( #43717 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-05-29 04:44:45 +00:00
284e6f543d
[8/n] Migrate merge_attn_states, mamba, sampler to torch stable ABI (continued) ( #43361 )
...
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-05-27 09:35:24 -07:00
Yongye Zhu
6ab6ffb428
[Feat][DSV4] Fuse q pad into deepseek v4 fused kernel ( #43162 )
2026-05-26 05:12:54 -10:00
a7be0f342d
[7/n] Migrate pos_encoding and norm kernels to libtorch stable ABI (continued) ( #43209 )
...
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-05-23 13:20:00 +08:00
07aeaf9d4d
[6/n] Migrate activation kernels, gptq, gguf, non cutlass w8a8 to libtorch stable ABI (continued) ( #42663 )
...
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-05-20 00:18:12 -07:00
Wentao Ye
00e20e76f7
[Refactor] Remove dead cuda kernels ( #42767 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-05-18 11:14:21 -07:00
Chris Leonard and Mikayla Gawarecki
85b2fecab7
[5/n] Migrate CUTLASS MLA, hadamard, awq, allspark and DSV3 fused a gemm to torch stable ABI (continued) ( #42339 )
...
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
2026-05-13 07:24:39 +00:00
Yongye Zhu
706a04d34b
[DSV4] Add silu clamp limit to shared expert ( #40950 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-04-27 00:37:43 -07:00
+7
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
Fadi Arafeh and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
445b7093fd
[perf][cpu] Accelerate BF16 GELU with LUT impl on Arm CPUs ( #37469 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-15 22:26:17 -07:00
Kaicheng Yang and Claude Sonnet 4.6
4beeb0689c
fused qknorm+rope kernel optimization for SM9.0 ( #37376 )
...
Signed-off-by: EricccYang <yangyang4991@gmail.com >
Signed-off-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-12 19:58:37 -07:00
Jee Jee Li
ecd1ea1363
[Kernel] Porting the TRTLLM minimax_allreduce_rms kernels ( #37045 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-11 00:20:20 +08:00
b55d830ec7
[Perf][Kernel] Persistent TopK scheduler: unified CUDAGraph-safe kernel with dynamic per-row dispatch - DeepSeek-V3.2 DSA decode ( #37421 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-04-08 13:35:57 -04:00
Gregory Shtrasberg
56c976c1b5
[ROCm] Enable fused_silu_mul_block_quant on ROCm ( #38817 )
...
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com >
2026-04-08 11:23:32 -05:00
Carl Y
3bc2734dd0
[Kernel] Fuse FP8 output quantization into merge_attn_states ( #36518 )
...
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com >
2026-04-03 01:47:04 +00:00
3aab680e3e
[ROCm][Bugfix] Fix ROCm runtime failure due to missing symbol ( #38750 )
...
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com >
Signed-off-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: tjtanaavllm <tunjian.tan@amd.com >
2026-04-01 21:30:11 -07:00
Monishver
c09ad767cd
Feature/silu block quant fusion v1 ( #32996 )
...
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com >
2026-04-01 18:50:43 +00:00
Olya Kozlova
598190aac3
[fix] Remove trtllm ragged mla prefills ( #36540 )
...
Signed-off-by: Olya Kozlova <okozlova@nvidia.com >
2026-03-31 12:30:27 -07:00
mikaylagawarecki
7c080dd3c5
[4/n] Migrate FP4/W4A8 CUTLASS kernels to torch stable ABI ( #37503 )
...
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
2026-03-31 10:21:13 -07:00
bcc6f67447
[Bugfix] Use null block (0) for padded block table entries ( #35431 )
...
Signed-off-by: SandishKumarHN <sandish@fb.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-03-30 14:02:51 -07:00
mikaylagawarecki
ab1a6a43fa
[3/n] Migrate cutlass/scaled_mm_entry.cu torch stable ABI ( #37221 )
...
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
2026-03-30 11:20:13 -07:00
mikaylagawarecki
bf4cc9ed2d
[2/n] Migrate per_token_group_quant to torch stable ABI ( #36058 )
...
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
2026-03-25 10:15:13 -07:00
Kyle Sayers
38364a7e32
[Sparse24] [Deprecation] Remove Sparse24 CT integration and kernels ( #36799 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-03-23 16:03:29 -04:00
mikaylagawarecki
8b10e4fb31
[1/n] Migrate permute_cols to libtorch stable ABI ( #31509 )
...
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
2026-03-19 11:27:26 -04:00
Michael Goin
09e4576f65
[Kernel] Add non-gated support for NVFP4 CUTLASS MoE ( #37320 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-03-17 18:12:04 -04:00
Terry Gao
3e6a1e1686
[Custom Ops] Add functional + out variant for scaled_fp4_quant ( #34389 )
...
Signed-off-by: tianrengao <terrygao87@gmail.com >
2026-03-16 18:51:46 -04:00
Asaf Gardin
bbf81f9a92
[Mamba1] - Kernel Level Chunk Alignment for Prefix Caching ( #34798 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
2026-03-01 20:40:23 +08:00
Tyler Michael Smith and Claude Opus 4.6
eb19955c37
[WideEP] Remove pplx all2all backend ( #33724 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-26 14:30:10 -08:00
Robert Shaw and Robert Shaw
6874638bc4
[Model Bash] DeepSeek R1 BF16 Min Latency QKV A GEMM (0.5% E2E Speedup) ( #34758 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2026-02-18 07:42:36 -08:00
ElizaWszola and Luka Govedič
a88b3be7c4
[Bugfix] Fix quant RMS norm fusion for quantization with TMA-aligned scales ( #33255 )
...
Signed-off-by: ElizaWszola <ewszola@redhat.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2026-02-17 23:35:04 -08:00
Roberto L. Castro and Claude Sonnet 4.5
afdce12c89
[Perf][Kernel] Add faster topKperRow decode kernel for DeepSeek-V3.2 sparse attention ( #33680 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com >
2026-02-10 10:29:52 -05:00
Roberto L. Castro
fcb9df99bd
[Perf][Kernel] Optimize FP4 quantization kernels (SM100F) ( #32520 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
2026-01-24 18:45:27 -07:00
Wentao Ye
eebc58df0c
[Refactor] Remove unused cutlass moe problem size function ( #32047 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-01-18 12:46:59 -08:00