MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
MOVZX
c5ab42bc0f
Merge branch 'main' into patches
2026-10-18 16:04:36 +07:00
f98ae6a9fd
[Bugfix][Core] Keep resumable sessions that stop while preempted ( #58778 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yifan Zong <yzong@redhat.com >
2026-10-17 09:08:14 +00:00
c3de0b37e2
[Core][Spec Decode] Replace EAGLE group heuristics with explicit draft-layer tags ( #61455 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 08:22:33 +00:00
MOVZX
e0fe5df731
Merge branch 'main' into patches
2026-10-17 08:06:56 +00:00
b544587408
[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning ( #53558 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-16 23:16:47 +00:00
MOVZX
be95edb7f6
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-16 17:35:42 +07:00
Juntian Liu and Claude Opus 5.5
19b0efce30
[Perf][Core] Skip the null head of sliding-window block tables ( #61724 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 16:14:51 -07:00
d486219fc2
[Bugfix][Core] Uncount stop-trimmed spec tokens from num_computed_tokens ( #58668 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-15 12:50:06 -07:00
ed3050add4
[Core] Apply prefill_schedule_interval outside data parallelism ( #54627 )
...
Signed-off-by: lifulu <fululi12@amd.com >
Signed-off-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
Co-authored-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-15 11:03:13 -05:00
MOVZX
76df77c04e
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-15 21:16:39 +07:00
Taksh Kothari
12b005eefa
[Bugfix] Don't split a multimodal item that a prefill chunk starts exactly on ( #52412 )
...
Signed-off-by: Taksh <takshkothari09@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-13 15:32:38 +00:00
2cc2e68d1c
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" ( #60043 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:58:59 -07:00
df6c7ad031
[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py ( #61183 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 03:05:39 +00:00
Matthew Bonanni and Claude Opus 5.5
fe341a53dd
[HiSparse] Report host pool in KV cache capacity logs ( #60285 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 20:49:49 +00:00
ztysth and OpenAI Codex
1bdd3fa2d5
[Bugfix][Core] Check multimodal chunk boundaries before encoder cache reuse ( #59721 )
...
Signed-off-by: ztysth <rnk.liulw@gmail.com >
Signed-off-by: ztysth <166926442+ztysth@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-12 19:25:35 +00:00
Woosuk Kwon and Claude Opus 5.5
e1a5086256
[Attention] Remove cascade attention ( #61126 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:41:07 -07:00
2339bd39d6
[Bugfix][Mooncake] Use the EAGLE replay boundary when retaining sliding-window tails ( #61012 )
...
Signed-off-by: Stu Cao <stu@primeintellect.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 09:15:18 +00:00
bzsuni
01abe975b5
[Bugfix][KV Connector] Fix speculative block reservations across KV cache groups ( #59880 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
2026-10-10 20:55:41 -07:00
Matthew Bonanni and Claude Opus 5.5
d5f356dae0
[HiSparse] Own GPU residency instead of editing core block-table rows ( #60647 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:25:44 -07:00
Matthew Bonanni and Claude Opus 5.5
4c43254d52
[HiSparse] Report host-pool block residency as HiSparse metrics ( #59485 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 16:45:34 -06:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
MOVZX
ac305c44cb
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-09 17:41:13 +00:00
d8fa463e2b
[Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries ( #60533 )
...
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 18:33:48 +00:00
Nicolò Lucchesi
e161a0798c
[Perf][Core] Reduce free KV cache queue overhead ( #60269 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 10:17:11 +00:00
7816afdb25
[Bugfix][Core] Drain async KV loads in pause(mode="wait"), and pause the drain test only once a request is in flight ( #60572 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 09:33:41 +00:00
ef63c23d35
[Bugfix][DSv4.1] Skip SWA bounded replay for KV loads that carry the window ( #59197 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 15:28:42 -07:00
e6fc81bc78
[Bugfix][Scheduler] Let pooling chunked prefill use full context ( #48039 )
...
Signed-off-by: Tommy <tran.tommy@hotmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-07 07:32:00 +00:00
e37e51dd24
[Metrics] Expose cached prompt tokens by cache tier ( #56318 )
...
Signed-off-by: Cam Quilici <cjquilici@gmail.com >
Signed-off-by: Cam Quilici <cameron@semianalysis.com >
Co-authored-by: Cam Quilici <cameron@semianalysis.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-06 14:52:05 -04:00
Sherif Waly
25385102fe
[Bugfix][KV Offload] Honor speculative cacheability in SimpleCPU ( #60071 )
...
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai >
2026-10-06 14:25:08 +02:00
zhangchengzhucufe-dev
68088ed392
Fix grammar in BlockPool.reset_prefix_cache docstring: "to invalid" → "to invalidate" ( #59886 )
...
Signed-off-by: zhangchengzhucufe-dev <zhangchengzhucufe@gmail.com >
2026-10-06 10:19:54 +00:00
6fa67afed9
[Bugfix][Core] Retire stale Mamba spec blocks in prefill checkpoint steps ( #59759 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-06 11:01:19 +08:00
2b32d9ac6a
[Bugfix][Core] Drop stale block hashes when a session truncates tokens ( #59103 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Brian Woods <me@brian.biz >
Co-authored-by: Change72 <changg@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-06 02:51:03 +00:00
bzsuni and cjackal
528772a4bf
[Bugfix][Core] Retain encoder cache references for repeated multimodal inputs ( #59942 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-10-05 15:52:46 +00:00
MOVZX
85c31d47ad
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 22:06:10 +07:00
MOVZX
2877121b2d
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 21:59:03 +07:00
ca0df1f122
[Bugfix][Core] Exempt exactly the blocks an async KV load writes from zeroing ( #59504 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-03 06:26:12 +00:00
f7999d2e44
[Bugfix][HiSparse] Size the KV cache from the groups HiSparse allocates ( #59450 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-02 04:11:55 +00:00
d58da79de7
[Metrics][P/D] Add KV-fetch stage gauges for async KV loads ( #58874 )
...
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-02 01:24:33 +00:00
ca65eb67d9
[Perf][HiSparse] Avoid repeated prefix scans and residency updates ( #57930 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-01 21:48:18 +00:00
Matthew Bonanni and Claude Opus 5.5
3a69636645
[Bugfix][HiSparse] Fix a chunked-prefill preemption livelock ( #59494 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 13:02:31 -04:00
34c458d27f
[Bugfix] Don't let a structured-output request sample from an unmasked row ( #54442 )
...
Signed-off-by: ArcheyChen <45146080+ArcheyChen@users.noreply.github.com >
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Co-authored-by: GPT-5.6 <noreply@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-01 02:11:44 +00:00
Nick Hill and Claude Opus 5.5
c055c1e075
[Core] Include the LoRA path in prefix-cache block hashes ( #59335 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 08:37:09 +08:00
kliuae and Nick Hill
7e583e615c
[Bugfix][Core] Fix mamba prefill checkpoint block reservation and prompt-end eviction in align mode ( #59175 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-30 21:26:54 +00:00
73c7cae4d7
[Bugfix][HiSparse] Stop the host pool feeding device KV cache residency metrics ( #58725 )
...
Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 20:50:01 +00:00
Giulio De Pasquale and Isotr0py
2e52a558f0
[Bugfix][Core] Schedule encoder-only prompts larger than one step ( #59029 )
...
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io >
Co-authored-by: Isotr0py <2037008807@qq.com >
2026-09-30 22:04:28 +08:00
Varun Sundar Rabindranath
5463fe4962
[KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA ( #57652 )
2026-09-30 09:11:03 -04:00
90e13fc757
[Bugfix][HiSparse] Never allocate GPU pages without host backing ( #59036 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 04:13:29 +00:00