MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
Nick Hill
f8471164bf
[MRV2] Detatch weight-reload logic from MRV1 ( #60937 )
2026-10-10 21:15:05 -07:00
f8d92cae41
[Quantization] Layer re-quantization for linear layers through online quantization API (MXFP8 -> FP8 PTPC showcase) ( #55684 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com >
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com >
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 11:38:18 -05:00
Ashraf Bhuiyan
b9e2903b89
[Mypy] Fix mypy typing for Ultravox and Unlimited-OCR models ( #58239 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
2026-09-27 21:19:04 -04:00
78fdf4efb8
[Bugfix][MoE] Convert FlashInfer BF16 weights in place ( #54699 )
...
Signed-off-by: Yuchen Wang <93700456+yuchenwang3@users.noreply.github.com >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-17 22:00:04 +08:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Wyett
b26039b09f
Add routed expert loading for gpt-oss ( #52209 )
...
Signed-off-by: Wyett <wyettzeng@gmail.com >
Signed-off-by: wyett <wyettzeng@gmail.com >
2026-08-23 17:59:08 +08:00
aoshen02 and Codex
f2906b492e
[Bugfix] Avoid repeated layerwise reload warning scans ( #49505 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-08-12 13:53:50 +08:00
aoshen02 and Roger Wang
99e62b802c
[Bugfix][Model Loader] Defer post-load attention weight processing ( #49519 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-10 19:16:57 -07:00
Harry Mellor and Claude Opus 5
9321aff536
[Bugfix] Wait for the linear bias before layerwise online processing ( #49805 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-07-25 17:56:32 +00:00
li-jinpeng and xymli
163ecba377
[Bugfix] Skip linear bias in layerwise reload to avoid corruption ( #49586 )
...
Signed-off-by: li-jinpeng <3332126450@qq.com >
Signed-off-by: xymli <xymli@tencent.com >
Co-authored-by: xymli <xymli@tencent.com >
2026-07-24 16:53:48 +08:00
d96aee0951
[Bugfix] Re-sync parameter tp_rank after process_weights_after_loading (fix replicated / disable_tp weight reload) ( #48025 )
...
Signed-off-by: Alex Xu <alexxu@roblox.com >
Co-authored-by: YQ-Wang <yiqingwang@roblox.com >
Co-authored-by: alexhxu <alex.xu1015@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-07-18 08:40:06 +00:00
Joan and Dakai An
0bd6b85a1f
[Bugfix] Preserve unloaded non-persistent buffers during layerwise reload ( #44371 )
...
Signed-off-by: Joan Velja <joan.velja22@gmail.com >
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com >
2026-07-14 17:46:29 -07:00
Harry Mellor
5051698e41
Remove unnecessary load_weights methods ( #44589 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-29 01:52:23 -07:00
c9e5bf8135
[Bugfix] Fix layerwise reload dropping params after a composed weight loader ( #44814 )
...
Signed-off-by: hallerite <git@hallerite.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Kyle Sayers <kylesayrs@gmail.com >
2026-06-10 06:42:05 -07:00
Harry Mellor
a80af24356
Speed up docs build ( #44635 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-05 14:51:44 +00:00
1ccdf87507
[Bugfix] Fix layerwise reload alias-buffer corruption ( #42481 )
...
Signed-off-by: rasdani <73563550+rasdani@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-05-15 15:20:53 -07:00
Alex Brooks
db9a84e0cd
[Bugfix] Fix FP8 Bias Loading ( #41424 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-05-03 20:30:04 +00:00
Kyle Sayers and Flora Feng
a085b5257d
[Docs] [QeRL] Layerwise Reloading Documentation ( #40317 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-04-28 21:06:38 -07:00
Asaf Gardin
19fa90ed0d
[Quantization] - Layerwise reloading of Attention/KV quantized models ( #38995 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
2026-04-15 18:03:32 -07:00
Aaron Hao
4729b90838
[Bug] Add e_score_correction_bias to SKIP_TENSORS ( #38746 )
...
Signed-off-by: ahao-anyscale <ahao@anyscale.com >
2026-04-02 21:15:05 -07:00
Asaf Gardin
3dc01ef352
[Quantization] Consolidate dummy format logic into DummyModelLoader ( #38637 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
2026-03-31 22:20:45 +00:00
Kyle Sayers
5869f69c5f
[Online Quant] [QeRL] Minor code cleanup ( #38574 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-03-31 14:56:43 +00:00
Asaf Gardin
1fc69f59bb
[Bug fix][Quantization] Fix dummy weight loading ( #38478 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
2026-03-30 16:38:02 -04:00
Kyle Sayers
d28d86e8a3
[QeRL] Fix online quantized reloading ( #38442 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-03-29 14:56:41 -06:00
Kyle Sayers
648edcf729
[QeRL] Compose online quantization with quantized reloading ( #38032 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-03-27 13:22:33 -07:00
Aaron Hao
5f82706a21
[BUG] Exclude SKIP_TENSORS from get_layer_size() + new weight sync example for dpep ( #37334 )
...
Signed-off-by: ahao-anyscale <ahao@anyscale.com >
2026-03-19 00:45:10 +00:00
Aaron Hao and SumanthRH
c1858b7ec8
[Feat][RL][1/2] Native Weight Syncing API: NCCL ( #31943 )
...
Signed-off-by: ahao-anyscale <ahao@anyscale.com >
Signed-off-by: Aaron Hao <ahao@anyscale.com >
Co-authored-by: SumanthRH <sumanthrh99@gmail.com >
2026-02-05 12:13:23 -05:00
Vasiliy Kuznetsov and Michael Goin
3f96fcf646
fix QERL attention import path ( #33432 )
...
Signed-off-by: vasiliy <vasiliy@fb.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-01-30 09:29:09 -08:00
Kyle Sayers
f857a03f6b
[QeRL] Layerwise Reloading ( #32133 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-01-30 08:50:05 -07:00