MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
Luca Motz and Claude Opus 5.5
83ed0c7c09
[Bugfix][NIXL] Fix receive post-process for different P/D block sizes ( #58860 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 19:30:11 +08:00
cb6458c5ce
[Core] Remove deprecated mamba_cache_mode "all" ( #58997 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
2026-10-01 02:59:37 -07:00
YukioZzz and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
30ae8248b6
[KVConnector][MoRIIO] Support K3 DSpark hybrid READ ( #57700 )
...
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 19:14:24 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
d43bb2f37f
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering ( #53781 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com >
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-12 00:36:47 -04:00
Nicolò Lucchesi
7f4793eaa3
[Nixl][PD] DCP support for MLA models ( #50611 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-08-29 16:03:53 +00:00
Lucas Wilkinson
8bdc70ec7b
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout ( #51718 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-21 17:47:49 -07:00
Nick Hill
9b0afeb4f6
[Perf] Avoid some more unnecessary GPU<->CPU syncs ( #51458 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-08 17:13:26 -07:00
Daniel Socek and Kunshang Ji
7154856f3d
[Bugfix] Fix handling 5D KV cache in kv_postprocess_layout_on_receive ( #47791 )
...
Signed-off-by: Daniel Socek <daniel.socek@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-26 22:22:42 +08:00
6700813f86
[3/N][KV-Cache Layout Refactor] Standardize Mamba cache; drop get_transfer_cache_regions ( #44456 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-21 09:16:15 +00:00
51878e5b6e
[2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends ( #44455 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-07-11 11:11:16 -04:00
Nicolò Lucchesi
750aab5b8e
[Bugfix] Fix CPU memory leak related to not cleaning up old remotes data ( #44424 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-06-11 07:54:52 -07:00
qizixi
96229fa99e
[KVConnector][1/N] PP-aware handshake aggregation and intermediate-PP output plumbing ( #43720 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
2026-06-04 22:04:19 -07:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e6018c644a
[Refactor] Remove dead code in tests and parallel_state ( #41471 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-03 19:32:39 -07:00
7e33081cee
[Attention] Make FlexAttention and FlashAttention use num-blocks first layouts ( #42095 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-05-26 19:55:56 -07:00
Nicolò Lucchesi and ZhanqiuHu
171d59ae8d
[Bugfix][PD] Fix DSv4 Disaggregated ( #41957 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
Co-authored-by: ZhanqiuHu <zhu@redhat.com >
2026-05-09 16:48:24 +00:00
zhanqiuhu and NickLucche
df8e63f4ed
nixl refactor: new transfer design ( #40731 )
...
Signed-off-by: ZhanqiuHu <zhu@redhat.com >
Signed-off-by: NickLucche <nlucches@redhat.com >
Co-authored-by: NickLucche <nlucches@redhat.com >
2026-05-06 06:16:25 -07:00
zhanqiuhu
cc3993b05d
nixl refactor [2/N]: unify TpKVTopology + HeteroTPTransferConfig into TransferTopology ( #39529 )
...
Signed-off-by: Zhanqiu Hu <zhu@redhat.com >
2026-04-20 12:39:08 +02:00
Or Ozeri
512c5eb455
[kv_offload+HMA][5/N]: Track group block hashes and block IDs ( #37109 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-04-08 19:50:28 +03:00
zhanqiuhu
bfdc0a3a99
[NIXL][Mamba][3/N] Heterogeneous TP: 3-read conv state transfer ( #37635 )
2026-04-06 19:07:02 +02:00
yjz and Nicolò Lucchesi
992368522f
[KVTransfer] Fix TpKVTopology.is_kv_replicated equality case ( #38179 )
...
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-04-01 12:41:49 +02:00
Nicolò Lucchesi
f5c081d432
[PD][Nixl] Add support for hybrid SSM-FA models ( #36687 )
2026-03-16 19:58:06 +01:00
Or Ozeri and Nicolò Lucchesi
a1a3523a56
[KVConnector] Support worker -> scheduler metadata ( #31964 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-03-11 17:36:37 +00:00
Nicolò Lucchesi
098d844731
[NIXL][1/N] Refactor kernel_block_size detection ( #35752 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-03-11 01:11:23 -07:00
Harry Mellor
f83b933b84
[CI] Bump mypy version to 1.19.1 ( #36104 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-10 09:18:28 -07:00
Matthew Bonanni
77a73458e3
Reapply [Attention] Refactor check_and_update_config ( #35122 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-03-09 07:17:14 -07:00
Nicolò Lucchesi
5b3ba94ab4
[Core][KVConnector] Support HMA+NixlConnector ( #35758 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-03-06 08:51:21 +01:00
Harry Mellor
17dc9c7fc9
[CI] Bump mypy version ( #34950 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-04 20:55:11 +00:00
Nicolò Lucchesi
7d8c6804e2
[Misc] Add debug logs ( #33931 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-02-05 09:42:40 -08:00
8322d4e47f
Enable Cross layers KV cache layout at NIXL Connector V2 ( #33339 )
...
Signed-off-by: Liran Schour <lirans@il.ibm.com >
Signed-off-by: liranschour <liranschour@users.noreply.github.com >
Co-authored-by: Or Ozeri <or@ozery.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-02-05 02:17:02 -08:00
Or Ozeri and Kevin H. Luu
2e8de86777
Revert "Enable Cross layers KV cache layout at NIXL Connector ( #30207 )" ( #33241 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-01-28 04:36:00 -08:00
Nicolò Lucchesi
492a7983dd
[Bugfix] Fix DeepseekV32 AssertionError: num_kv_heads == 1 ( #33090 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-01-27 15:03:20 +00:00
liranschour and Or Ozeri
64e3d67ac0
Enable Cross layers KV cache layout at NIXL Connector ( #30207 )
...
Signed-off-by: Liran Schour <lirans@il.ibm.com >
Signed-off-by: liranschour <liranschour@users.noreply.github.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-01-22 10:12:58 +00:00
Nicolò Lucchesi
5b68107411
[Misc][PD] Fix get_attn_backend usage in transfer connectors ( #31988 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-01-12 18:10:05 +01:00
Chendi.Xue
94578127a4
[NIXL] refine decoder side post process for heterogeneous BlockSize and kv_layout ( #30275 )
2026-01-09 21:22:19 +00:00
Matthew Bonanni
2612ba9285
[1/N][Attention] Restructure attention: move files ( #31916 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-01-09 13:10:24 -08:00
weiyu
e7596371a4
[Refactor][TPU] Remove torch_xla path and use tpu-inference ( #30808 )
...
Signed-off-by: Wei-Yu Lin <weiyulin@google.com >
Signed-off-by: weiyu <62784299+weiyu0824@users.noreply.github.com >
2026-01-07 16:07:16 +08:00
Chendi.Xue and Nicolò Lucchesi
6ca74bc11a
[NIXL][BUG FIX] Fix both failing issue and accuracy issue with nixl + host_buffer on CUDA ( #30419 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
Signed-off-by: Chendi.Xue <chendi.xue@intel.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2025-12-18 22:10:02 +00:00
Nicolò Lucchesi
bc3700e0cd
[NIXL] Support P tensor-parallel-size > D tensor-parallel-size ( #27274 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2025-12-18 11:53:30 +08:00
Martin Hickey
f4417f8449
[KVConnector] Add KV events to KV Connectors ( #28309 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2025-12-11 15:30:29 +01:00
dtc and Nicolò Lucchesi
842aba501d
[P/D] Introduce Mooncake Transfer Engine as kv_connector ( #24718 )
...
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com >
Signed-off-by: dtc <dtcccc@linux.alibaba.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com >
2025-12-04 09:51:36 +00:00
Kuntai Du
8aaa81b35f
[KVConnector] remove unused code (the model aware kv ops class) ( #29709 )
...
Signed-off-by: KuntaiDu <kuntai@uchicago.edu >
2025-12-04 06:00:52 +00:00
Nick Hill
289eb6c537
[Core] Simplify async KV output aggregation ( #28327 )
...
Signed-off-by: Nick Hill <nhill@redhat.com >
2025-11-09 09:44:13 -08:00
Nick Hill
67a2da890e
[PerfFix] Avoid separate thread for MP executor shm spin (take 2) ( #28319 )
...
Signed-off-by: Nick Hill <nhill@redhat.com >
2025-11-07 22:11:03 +00:00
Nicolò Lucchesi
68a72a5cc1
Revert "[PerfFix] Avoid separate thread for MP executor shm spin ( #28012 )" ( #28289 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2025-11-07 15:07:01 +00:00
Nick Hill
c9f66da8fd
[PerfFix] Avoid separate thread for MP executor shm spin ( #28012 )
...
Signed-off-by: Nick Hill <nhill@redhat.com >
2025-11-04 08:33:55 -08:00
Nick Hill
0cdbe7b744
[Core] Async scheduling + structured outputs compatibility ( #26866 )
...
Signed-off-by: Nick Hill <nhill@redhat.com >
2025-11-01 00:35:04 +00:00