MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
MOVZX and Hinata
5f3c7bd8ab
[Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
...
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
the live server process maps none of them
- Delete tests that only build architectures this fork no longer
registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
import at module level
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 19:31:07 +07:00
MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
MOVZX
c5ab42bc0f
Merge branch 'main' into patches
2026-10-18 16:04:36 +07:00
Jee Jee Li
e56b1603af
[Kernel]Add norm fused kernel for Step-5 Preview and Muse Spark Open ( #62132 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-17 01:53:00 -07:00
MOVZX
e0fe5df731
Merge branch 'main' into patches
2026-10-17 08:06:56 +00:00
08a65e287e
[Bugfix][Kernel] hadacore: don't load past the input on a partial last warp ( #61944 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 00:52:17 -07:00
Jee Jee Li
39983a5e7a
[Kernel] Add FP32 router GEMM support for Step-5 Preview and Muse Spark Open ( #62126 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-16 22:17:55 -07:00
596e530675
[Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm ( #62042 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Lain <fusiyuan2000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 15:39:46 -07:00
Wentao Ye
8e25bbca19
[Refactor] Remove dead code multiple places ( #61843 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-16 15:01:07 +00:00
MOVZX
be95edb7f6
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-16 17:35:42 +07:00
Monishver and Kevin H. Luu
fb3fec970e
[Bugfix][Kernel] silu_and_mul_quant: run the vectorized loop ( #61716 )
...
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 18:16:53 -07:00
Jack Hu and Douglas Lehr
ef38e8d006
[Model][Hardware][AMD][Kernel] Part 2/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_ATTN ( #42755 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Signed-off-by: Jack Hu <jingxujaster@hotmail.com >
Signed-off-by: Jack Hu <jack.hu@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-15 17:10:14 -07:00
f8460ef236
[Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache ( #61371 )
...
Signed-off-by: juntianl <juntianl@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-15 13:58:25 -07:00
MOVZX
76df77c04e
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-15 21:16:39 +07:00
Wentao Ye
c19c263ac8
[GLM 5.3 Perf] Handle two elements at the same time for per token group quant, 2.8% E2E ITL improvement ( #60927 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-13 13:06:41 -07:00
e83222396f
[ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches ( #57199 )
...
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-13 17:21:30 +00:00
Elvir Crnčević
b9b401b2ef
[Kernel] Fuse DeepEP-v2 decode globalize+align into one launch ( #53911 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
2026-10-13 11:10:38 -04:00
Thien Tran
0fae39a014
[Perf] per_token_group_quant_8bit: multiply by reciprocal instead of dividing per element ( #61343 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 21:37:10 +08:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
Wentao Ye
838526c7e5
[Compile] Fix sampler compile warning ( #60680 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-10 08:09:45 -07:00
Matthew Bonanni and Claude Opus 5.5
d5f356dae0
[HiSparse] Own GPU residency instead of editing core block-table rows ( #60647 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:25:44 -07:00
a6f43e57d6
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm ( #59523 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: indianspeedster <cspandey016@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: indianspeedster <cspandey016@gmail.com >
2026-10-09 08:29:55 +00:00
Rehan Khan and Bob
fd0216d3b5
[Bugfix][CPU] Fix fused Gumbel-max wrapping-window bias ( #60221 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
Co-authored-by: Bob <bob@ibm.com >
2026-10-08 15:15:53 +08:00
Matt and Claude Opus 5.5
5dd4a628dd
[ROCm][Bugfix] Map sleep-mode cuMem memory for the host ( #60367 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-08 04:26:12 +00:00
Matthias Gehre
b267fe4881
[ROCm][Perf] W4A16: pad gfx11 weight and activation strides ( #56301 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-10-06 08:22:33 -05:00
Mia Ojeda
3313825983
[Bugfix][DiffusionGemma] Read the causal mask as int32 and compile the sample step ( #59992 )
...
Signed-off-by: ojeda-e <eojeda@league.com >
2026-10-06 19:25:51 +08:00
4ac0d0eac2
[ROCm][RDNA3] Fix W4A16 split-K accuracy and determinism ( #54706 )
...
Signed-off-by: AIwork4me <AIwork4me@users.noreply.github.com >
Co-authored-by: AIwork4me <AIwork4me@users.noreply.github.com >
Co-authored-by: JartX <sagformas@epdcenter.es >
2026-10-03 23:44:17 +08:00
6660926d92
[Bugfix][CPU] Fix macOS build on Apple Clang < 17: structured binding… ( #58932 )
...
Signed-off-by: Miajan Chowdhury <chowdhurymiajan@gmail.com >
Co-authored-by: Miajan Chowdhury <chowdhurymiajan@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 10:01:07 +08:00
Yongye Zhu and Claude Opus 5.5
7566d83bd3
[Attention][MiniMax-M3] NVFP4 KV cache on the MSA sparse attention path ( #59300 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 14:39:55 -07:00
almayne and Li, Jiang <jiang1.li@intel.com>
93dd80f636
[CPU] Conv1d optimised kernel for aarch64 ( #54093 )
...
Signed-off-by: Anna Mayne <anna.mayne@arm.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-30 20:23:42 +08:00
Rukhaiya2004 and Li, Jiang <jiang1.li@intel.com>
72e7874fa6
[Hardware][Power] Enable W4A16 (AWQ & GPTQ) quantization on POWER10 using VSX ( #59149 )
...
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-30 19:44:48 +08:00
Rehan Khan and Li, Jiang <jiang1.li@intel.com>
583637f126
[CPU][Perf] Add vectorized Sampler Kernel ( #53913 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-30 15:16:28 +08:00
Matthew Bonanni and Claude Opus 5.5
2798f66860
[Bugfix][HiSparse] Resolve MTP verification rows with a union residency kernel ( #59235 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 03:01:45 +00:00
Yongye Zhu and Claude Opus 5.5
77871126f9
[Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary ( #58586 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 03:08:36 +08:00
38cc054fc2
[Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 quantization ( #58194 )
...
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-25 09:19:48 -06:00
chuan932 and Misha Goin
f550ad8bf7
[Kernel][Perf] Register-resident path for per-token-group 8-bit quant ( #55330 )
...
Signed-off-by: chao.huan <chao.huan@nio.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-25 08:11:42 -07:00
Yongye Zhu and Summer Yang
442bd00d9f
[Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 ( #55072 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
2026-09-25 03:27:21 +00:00
f34a0e0797
[CPU] Gate the AVX10.2 paths on compiler support ( #58133 )
...
Signed-off-by: R <Ganesh.R@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-24 18:07:53 +08:00
Ankit-Jaiswal-AMD and Douglas Lehr
88afb9dcac
[CPU] Adds support for fp32 attention sinks ( #56252 )
...
Signed-off-by: Ankit Jaiswal <ankit.jaiswal@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 08:55:09 -05:00
Gabriel Wu and Kimi Code
c843f0cab7
[Bugfix][SM120][MLA] Support NoPE sparse MLA (GLM-5.3-Flash) on the FlashInfer SM120 backend ( #55277 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-23 02:03:20 -07:00
Misha Goin
25965ee583
[Kernel] Remove AllSpark INT8 W8A16 GEMM backend ( #58001 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-22 14:07:34 -07:00
Gilles Turpin and Wentao Ye
74370a0e30
[Bugfix] hadacore_transform: respect inplace parameter to fix garbage outputs with QuIP transforms ( #43462 )
...
Signed-off-by: Gilles Turpin <turpingilles15@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-22 12:48:20 -04:00
Woosuk Kwon and Codex
9a70c233cd
[Perf][DSV4.1] Fuse TP all-reduce with mHC input preparation ( #57643 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-20 20:28:06 -07:00
Wentao Ye
4868312128
[Refactor] Remove dead kernel code ( #57621 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-20 10:39:23 -07:00
7d99c2c4fd
[Feature][Humming] Humming feature integration ( #56685 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-20 08:11:01 -07:00
92b40f5a14
[CPU][Perf] refactor paged attention for Arm CPUs ( #56045 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-20 12:25:13 +08:00
a7fda4c88b
[Bugfix] Fix SM100 fp8_ds_mla cache scales ( #49435 )
...
Signed-off-by: Ariel Bereslavsky <arielb@getimpala.ai >
Signed-off-by: Ariel Bereslavsky <ariel@getimpala.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ariel Bereslavsky <arielb@getimpala.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-19 17:15:07 -07:00
Yuwen Zhou and Li, Jiang <jiang1.li@intel.com>
b346479065
[CPU] Add CPU FP8 W8A8 linear/MoE support ( #49942 )
...
Signed-off-by: yuwenzho <yuwen.zhou@intel.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-18 17:59:42 +08:00
MOVZXandHinata
Jee Jee Li