MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
bfdbfa1632
[Bugfix][PCP] Align MoE routing padding with gathered tokens ( #60946 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: wangyicong <wangyicong@bytedance.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-14 21:04:00 -04:00
ed7ae705a1
[Bugfix][MRV2] Use int64 index mappings to fix row-offset overflow ( #60736 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-09 14:38:11 -07:00
Qiu Chunshuo and Lucas Wilkinson
8486f1ebfe
[Perf][PCP] Shard decode requests across PCP ranks ( #52162 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-01 23:50:13 -04:00
7314360c9e
[Perf][PP] Skip sampled-token broadcasts whose requests leave the engine ( #58542 )
...
Signed-off-by: LostFox11 <wangziyue17@huawei.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: LostFox11 <wangziyue17@huawei.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-30 19:18:55 -07:00
Qiu Chunshuo and Lucas Wilkinson
98b45c1a94
[Feat] Support PP with PCP in GPU Model Runner V2 ( #59139 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-30 10:21:56 -04:00
e7c903609f
[Perf][MRV2] Allow FULL decode graphs for one-token prompt tails ( #58400 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-09-28 12:48:52 -07:00
927c87b347
[PCP][DCP] Support DCP target model with non-DCP Dspark ( #56723 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-26 18:12:35 -04:00
e6dc16cebd
[PCP] Support prefill context parallelism with data parallelism ( #57075 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
Nick Hill
afec265bca
[MRV2] Miscellaneous code cleanup ( #57980 )
2026-09-23 01:51:52 -07:00
Nick Hill
f2aad6aa70
[MRV2] Buffer util simplifications ( #56888 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-15 20:36:51 +00:00
676650397f
[Feature][PCP] Support decode-only FULL CUDA graphs ( #53867 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: QiuChunshuo <chunshuoq@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-15 16:27:46 +08:00
ebe1dec2da
[PCP][DCP] Enable PCP+DCP on sparse-MLA models ( #56157 )
...
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai >
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai >
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai >
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Patryk Saffer <PatrykSaffer@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-12 23:24:44 +00:00
980c16c8e4
[PCP][Spec Decode] Adds PCP support for single-module MTP and replicated DSpark. ( #56107 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-10 22:07:53 -07:00
83990f5bcc
[Bugfix][MRV2] Initialize DCP metadata after batch partitioning ( #55212 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Ziyang Zhang <hafuhafu@qq.com >
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai >
Co-authored-by: Ziyang Zhang <hafuhafu@qq.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 06:50:09 +00:00
Ausar and Claude
6bafc049aa
[Bugfix][PP] Never drop a decoding request from the sampled-token broadcast ( #54436 )
...
Signed-off-by: ArcheyChen <45146080+ArcheyChen@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-31 23:49:54 +00:00
b1fbbc2ade
BugFix(PCP): use persistent input buffers for PIECEWISE CUDA graphs ( #53515 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-26 05:58:13 +00:00
0a94d85a66
[Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch ( #51865 )
...
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com >
Co-authored-by: Janelle Cai <janelle.cai@modal.com >
2026-08-11 17:15:06 -07:00
Nick Hill
c5d470ac4c
[ModelRunner V2] Minor indexing optimizations ( #51210 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-06 14:44:13 -07:00
Qiu and Lucas Wilkinson
811622c410
[Refactor][PCP] Make PCPManager construction extensible ( #50066 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-08-05 23:05:14 +00:00
b6ff8a2f50
[Core] Add MRV2 virtual-batch PCP for MLA ( #46570 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-07-19 02:53:15 +00:00