MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
de7d70676d
[Perf] Low-latency CPU all-reduce for DP metadata syncs ( #61767 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: hjjq <50634613+hjjq@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-15 15:29:13 -07:00
Ning Xie and Simon Mo
a0f28d73a8
[sharded state loader] support pp in sharded state loader ( #52110 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
Co-authored-by: Simon Mo <simon.mo@hey.com >
2026-10-09 11:08:31 -07:00
caff2b44cb
[BugFix] Pick the DP world-group port at bind time via the coordination store ( #51018 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-08 02:30:35 +00:00
Samuel Nordmann and Misha Goin
4a99ca40d6
[Perf][MoE] Eliminate staging from NCCL symmetric reduce-scatter ( #49194 )
...
Signed-off-by: samnordmann <snordmann@nvidia.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 07:37:01 -07:00
927c87b347
[PCP][DCP] Support DCP target model with non-DCP Dspark ( #56723 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-26 18:12:35 -04:00
Matt
2d8a5c5741
[ROCm][Bugfix] Reduce CUDA graph divergences ( #57229 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-09-17 05:12:13 +00:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
d2d649e674
[DS V4.1][Engram] Support async prefetch for offloaded engram lookups and engram DP sharding ( #56512 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: ywang96 <>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-13 02:44:23 -07:00
Matt and Claude Opus 4.8
72d4d83157
[ROCm][Bugfix] Fix elastic EP scaling deadlock ( #56610 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-09-12 23:09:19 +00:00
Simon Danielsson
ae48466cf3
[Perf][ROCm] Add AITER custom AG/RS (DP only) ( #48247 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-11 13:19:57 +08:00
Matt and Claude Opus 5
9163190dda
[ROCm][Bugfix][Perf] Tune multi-stream shared experts use; wvSplitKrc fixes ( #56098 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-10 15:14:22 -07:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
3116c5d06b
[Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism ( #54371 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 07:32:34 -07:00
4ac452ad98
[Core] Release NCCL communicator memory in sleep mode ( #51485 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-01 14:31:16 +08:00
446c769482
[Distributed] Add opt-in FlashInfer PCIe IPC all-reduce backend ( #53576 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-01 03:34:56 +00:00
AI-Infra Rookie
c92b29a1d4
[Bugfix] Make metadata send non-blocking in GroupCoordinator.isend_tensor_dict ( #49274 )
...
Signed-off-by: z00897177 <zhouxinyi6@huawei.com >
2026-08-30 23:51:25 +00:00
Itay Alroy
a2257f95b7
[Elastic EP] Reduce eager-mode reconfiguration downtime ( #51885 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
2026-08-19 02:13:06 +00:00
11ba93f364
[BugFix] Use file:// rendezvous for single-node executors to eliminate startup port races ( #50999 )
...
Signed-off-by: Parsa Mazaheri <p_mazaheri@apple.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Parsa Mazaheri <p_mazaheri@apple.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-09 22:00:04 -04:00
liuzhenwei
d54b58cc48
[Hardware] Use torch.accelerator.empty_host_cache() for host cache cl… ( #51107 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-08-06 10:27:57 +08:00
bnellnm
373fe8b83e
[MoE Refactor] Remove MoE legacy code ( #51078 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-08-05 14:35:36 -07:00
Wentao Ye
0cf49a5d15
[Refactor] Remove multiple dead codes ( #50285 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-03 10:59:08 -04:00
Itay Alroy
35efdf6b34
[Elastic EP] Async preparation ( #47288 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
2026-07-28 05:19:13 -07:00
Schwinn Saereesitthipitak
b5b61c622c
[Core][Distributed] Add process-checkpoint lifecycle hooks for communicators (starting with Flashinfer) ( #46877 )
...
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com >
2026-07-26 14:47:50 -04:00
Michał Ganczarenko and Kunshang Ji
37bf988c2f
[XPU][Bugfix] Fix GroupCoordinator device_index ( #47295 )
...
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-20 15:25:56 +08:00
b6ff8a2f50
[Core] Add MRV2 virtual-batch PCP for MLA ( #46570 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-07-19 02:53:15 +00:00
05eed72aec
[ROCm] Re-enable cudagraph memory profiling, captured on the current stream ( #48526 )
...
Signed-off-by: pei.zhang <pei.zhang@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-15 10:03:48 -05:00
jialoop-git and Claude Opus 4.8
a4f019fa25
fix(distributed): propagate distributed_timeout_seconds to NCCL device groups ( #45159 )
...
Signed-off-by: jialoop-git <joane8913456@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-07 02:52:51 +00:00
ebfbcfe46a
Stop setting CUDA_VISIBLE_DEVICES internally in vLLM, add device_ids arg ( #45026 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: kourosh hakhamaneshi <kouroshHakha@users.noreply.github.com >
2026-06-20 13:38:10 -07:00
Wentao Ye
41a4829f22
[Logs Refactor] Optimize shutdown logs, easier to follow and consistent ( #43707 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-06-04 14:36:32 -04:00
Tushar Jain and Tushar Jain
38fd2405f3
use split_group for pytorch process group creation ( #41980 )
...
Signed-off-by: Tushar Jain <tushar00jain@users.noreply.github.com >
Co-authored-by: Tushar Jain <tushar00jain@users.noreply.github.com >
2026-06-04 14:36:07 -04:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e6018c644a
[Refactor] Remove dead code in tests and parallel_state ( #41471 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-03 19:32:39 -07:00
Nick Hill
e15f20258b
[ModelRunnerV2] Avoid pipeline parallel bubbles ( #42187 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-02 14:02:01 -07:00
e26e1f0928
[Feature] Add --cpu-distributed-timeout-seconds CLI Option for CPU Process Group Timeout ( #42968 )
...
Signed-off-by: fangyuchu <fangyuchu@qq.com >
Signed-off-by: zWaNg3 <389750525@qq.com >
Co-authored-by: zWaNg3 <389750525@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-21 15:42:07 -07:00
Xinyu Chen
d740e2c029
[XPU] update xpu graph usage ( #43043 )
...
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com >
2026-05-19 23:09:07 +08:00
529c671e80
[ROCm][FEAT] AITER Fused Allreduce + RMSNorm ( #37646 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com >
Signed-off-by: junkang1991 <junkangchow@gmail.com >
Co-authored-by: Rita Brugarolas <Rita.BrugarolasBrufau@amd.com >
Co-authored-by: junkang1991 <junkangchow@gmail.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-05-01 23:07:18 +08:00
Fadi Arafeh
2aab9acf48
[CPU][BugFix] Fix inter-node pipeline parallel ( #40150 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
2026-04-20 17:21:12 +08:00
Andrey Talman
2111997f96
[release 2.11] Update to torch 2.11 ( #34644 )
2026-04-07 18:55:48 -07:00
Ilya Markov and Markov Ilya
abdbb68386
[EPLB] Add alternative communication for EPLB weight exchange ( #33176 )
...
Signed-off-by: ilmarkov <markovilya197@gmail.com >
Signed-off-by: Markov Ilya <markovilya19@gmail.com >
Co-authored-by: Markov Ilya <markovilya19@gmail.com >
2026-03-31 08:17:12 -04:00
Andreas Karatzas
bdc1719eb9
[ROCm][CI] Fix AITER state leak in shared_fused_moe_routed_transform test ( #38137 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-03-26 09:26:46 -07:00
Itay Alroy
de1a86b7de
elastic_ep: Fix stateless group port races ( #36330 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
2026-03-18 14:36:18 +00:00
Harry Mellor
f83b933b84
[CI] Bump mypy version to 1.19.1 ( #36104 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-10 09:18:28 -07:00
nkm-meta
792cbd64ca
Add platform method to enable custom collective ops registration ( #34760 )
...
Signed-off-by: Naina Kuruballi Mahesh <nainakm@meta.com >
2026-03-05 00:50:32 +00:00
Kunshang Ji and Harry Mellor
16d2ad1d38
[Hardware] Replace torch.cuda.empty_cache with torch.accelerator.empty_cache ( #30681 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-04 09:49:47 +00:00
Li, Jiang
440f0e7dc6
[Bugfix] Avoid src/dst as None in irecv/isend_tensor_dict ( #35754 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-03-03 05:56:08 -08:00
wangxiyuan
510bc9e1df
[Misc] Cleanup useless current_platform import ( #35715 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-03-02 09:36:54 +00:00
dea268336f
[1/N] Elastic EP Milestone 2 ( #34861 )
...
Signed-off-by: Yongji Wu <wuyongji317@gmail.com >
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Signed-off-by: Ron Tourgeman <rtourgeman@nvidia.com >
Co-authored-by: Yongji Wu <wuyongji317@gmail.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com >
2026-02-28 04:46:42 +00:00
Wentao Ye and Nick Hill
3d2a026fd0
[Feature] Pipeline Parallel Async send/recv, 2.9% E2E throughput improvement ( #33368 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Nick Hill <nhill@redhat.com >
2026-02-13 16:38:16 +08:00
67132945bb
[Perf] Move eplb rebalance algo to async thread ( #30888 )
...
Signed-off-by: ilmarkov <markovilya197@gmail.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-02-10 22:19:10 +00:00
杨朱 · Kiki and Claude Opus 4.5
1a7894dbdf
[Misc] Replace Optional[X] with X | None syntax ( #33332 )
...
Signed-off-by: carlory <baofa.fan@daocloud.io >
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-30 01:56:59 -08:00
5a93b9162b
[MoE Refactor] Integrate Naive Prepare Finalize into MK ( #32567 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: amirkl94 <203507526+amirkl94@users.noreply.github.com >
2026-01-27 01:28:02 +00:00