MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
haotiaa
6e6e86078a
[Bugfix][Platform] Don't resolve current_platform during import vllm ( #60766 )
...
Signed-off-by: Haotian An <haotiaa@amazon.com >
2026-10-12 17:05:34 +00:00
aditi-amd
385d86a6cf
[Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) ( #57057 )
...
Signed-off-by: Aditi Rana <aditi.rana@amd.com >
2026-10-06 20:42:52 +00:00
aoshen02 and Claude Opus 5.5
d09ff77637
[Bugfix] Give each data-parallel engine its own global RNG streams ( #59788 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 17:38:18 +00:00
Chaojun Zhang
7230dfea50
[XPU] Preserve non-contiguous strides when pinning CPU tensors for UVA view ( #54874 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-09-29 09:43:07 +08:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Nick Hill
f2aad6aa70
[MRV2] Buffer util simplifications ( #56888 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-15 20:36:51 +00:00
zhaoguochun1995
bbbd0a02c9
[Bugfix] Carry over queued work when materializing the dedicated stream ( #56382 )
...
Signed-off-by: zhaoguochun1995 <zhaoguochun1995@163.com >
2026-09-14 23:52:20 +00:00
siyu and Isotr0py
73fb19151f
[Fast Start] Support fp4 ( #55465 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 13:22:17 +00:00
Wentao Ye
6b15bea080
[Refactor] Remove utils dead code ( #53941 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-08 11:22:09 -04:00
fxmarty-amd and Andreas Karatzas
d1c15e589d
[CI] Speed up quantization test group ( #53291 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-01 13:15:24 -05:00
sychen52
7c5dc571cb
[Attention][DSA] Enable W4A16 DSA ( #51724 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
2026-09-01 12:37:47 +08:00
Lucas Wilkinson
8bdc70ec7b
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout ( #51718 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-21 17:47:49 -07:00
Chaojun Zhang
f863387250
[XPU] Fix UVA weight offloading (non-pinned-tensor views and static Triton launcher) ( #51770 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-08-11 21:48:44 +08:00
Wei-Ming Chen
1c94e8dc7d
Add NVFP4 KV 4-over-6 scale search ( #45187 )
...
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com >
2026-08-07 13:22:27 -07:00
Nick Hill and Claude
7635a9002b
[Core] Explicitly manage torch CPU threads in workers ( #49919 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-04 10:06:38 -07:00
51878e5b6e
[2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends ( #44455 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-07-11 11:11:16 -04:00
JartX and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
70749fdcca
[Feature] Triton INT4 per-token-head KV cache quantization ( #40835 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 10:21:25 +00:00
Nick Hill
7df3d7dada
[Core] Ensure memory is pinned prior to async h2d copy ( #45424 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-20 20:02:24 -07:00
Nick Hill
989c176c0a
[Perf][3/n] Eliminate GPU<->CPU syncs in attention impls ( #41434 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-05-07 19:44:24 -07:00
66dfee7121
[Bugfix] Fix degenerate KV cache stride causing TMA cudaErrorIllegalInstruction ( #40737 )
...
Signed-off-by: David Oy <david@baseten.co >
Signed-off-by: David Oy <58150256+the-david-oy@users.noreply.github.com >
Signed-off-by: David Oy <david.oy@baseten.co >
Co-authored-by: David Oy <david@baseten.co >
Co-authored-by: Claude <claude@anthropic.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-05-03 23:52:18 +00:00
sychen52
6b2b7bd0eb
Add nvfp4 support to reshape_and_cache_flash ( #37332 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
2026-04-17 07:28:00 -07:00
f4b42df048
[Attention Backend] TurboQuant: 2-bit KV cache compression with 4x capacity ( #38479 )
...
Signed-off-by: vibhavagarwal5 <vibhavagarwal5@gmail.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-04-14 19:57:13 -07:00
Richard Zou
f44afef6d6
[compile] Allow strings in custom ops without regressing compilation times ( #38123 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
2026-04-10 07:26:37 +00:00
Yan Ma
ec68d53b2b
Add platform manual_seed_all API ( #38468 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-04-10 13:43:50 +08:00
Richard Zou
ba4a78eb5d
[torch.compile] Allow usage of Opaque Objects in PyTorch 2.11 ( #39286 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
2026-04-08 23:21:10 +00:00
Andrey Talman
2111997f96
[release 2.11] Update to torch 2.11 ( #34644 )
2026-04-07 18:55:48 -07:00
2ce3d0ce36
[Feature] KV cache per-token-head INT8/FP8 quantization ( #38378 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: yangyang4991 <yangyang4991@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
2026-04-02 08:13:26 -04:00
Matthew Bonanni
116f4be405
[1/N][Cleanup] Standardize on use of is_quantized_kv_cache ( #38659 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-04-01 04:08:01 +00:00
4dfad17ed1
replace cuda_device_count_stateless() to current_platform.device_count() ( #37841 )
...
Signed-off-by: Liao, Wei <wei.liao@intel.com >
Signed-off-by: wliao2 <wei.liao@intel.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-03-31 22:32:54 +08:00
Andreas Karatzas
36f6aede23
[Misc] Optimized check to encapsulate both CUDA and ROCm platforms ( #34549 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-03-26 09:43:07 +08:00
Kunshang Ji
53ec16a705
[Hardware] Replace torch.cuda.device_count/current_device/set_device API ( #36145 )
...
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-03-12 07:57:47 -07:00
Richard Zou
e82fbeec7b
[torch.compile] Undo the fast_moe_cold_start hack in torch>=2.11 ( #35475 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
2026-03-01 21:44:22 +00:00
35d44b4557
[XPU]Support CUDAGraph on XPU Platform ( #34482 )
...
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com >
Co-authored-by: chzhang <chaojun.zhang@intel.com >
Co-authored-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-02-24 22:22:52 -08:00
Manrique Vargas
ad5aa6bd9f
fix(docs): fix typos in comments and docstrings ( #34836 )
...
Signed-off-by: machov <mv1742@nyu.edu >
2026-02-18 23:17:41 -08:00
Andreas Karatzas
a0638d052d
[Bugfix] Fix ROCm UVA CPU weight offloading broken by #32993 ( #34543 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-02-13 20:01:42 -08:00
Wei Zhao and Michael Goin
59d53066d8
[Feature] Support CPU Offloading without Pytorch Pinned Memory that leads to doubled allocation ( #32993 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-02-13 08:11:26 -08:00
Xinyu Chen
e969a169ef
support view_from_cpu_tensor on XPU ( #33868 )
...
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com >
2026-02-06 08:34:20 +00:00
Cyrus Leung and YunzhuLu
21997f45b1
[Redo] #33110 with threading limit ( #33502 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Co-authored-by: YunzhuLu <lucia.yunzhu@gmail.com >
2026-02-01 09:18:11 +00:00
Angela Yi
4197168ea5
[ez] Remove checks for torch version <= 2.8 ( #33209 )
...
Signed-off-by: angelayi <yiangela7@gmail.com >
2026-01-28 16:03:56 -05:00
7. Sun
0ccecf8833
[Tests] Standardize RNG seed utility across test files ( #32982 )
...
Signed-off-by: 7. Sun <jhao.sun@gmail.com >
2026-01-24 06:47:14 +00:00
a28b94e6ef
[Performance] Split FlashAttn attention and cache update ( #25954 )
...
Signed-off-by: ElizaWszola <ewszola@redhat.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Luka Govedič <luka.govedic@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
Signed-off-by: Luka Govedič <lgovedic@redhat.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
Co-authored-by: Luka Govedič <luka.govedic@gmail.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Luka Govedič <lgovedic@redhat.com >
2026-01-23 17:28:06 -08:00
Angela Yi
7933638051
[misc] Remove is_torch_equal_or_newer(2.4) cases ( #32296 )
...
Signed-off-by: angelayi <yiangela7@gmail.com >
2026-01-13 23:22:07 -08:00
Vadim Gimpelson
e15a5ff07b
[MISC] Add strict contiguity check for FlashInfer attention tensors ( #32008 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-01-10 12:40:05 -08:00
roikoren755
0c9614876e
Update modelopt KV cache quantization resolution to new scheme ( #31895 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
2026-01-10 04:54:13 +00:00
wangxiyuan
bb4337b34c
[Platform] Deprecate seed_everything ( #31659 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-01-04 18:34:04 -08:00
Amir Samani and youkaichao
030fc44914
use the same stream for cuda graph catpure and replay for NCCL ( #29207 )
...
Signed-off-by: Amir Samani <asamani@nvidia.com >
Signed-off-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2025-12-25 19:10:03 +08:00
danielafrimi and root
7b966ae2ba
[Fix]Load kv-cache dtype from hf_quant_config.json automatically (fix for reverted PR) ( #30785 )
...
Signed-off-by: <>
Co-authored-by: root <root@gpu-937.slurm-workers-slurm.slurm.svc.cluster.local >
2025-12-17 01:56:38 -08:00
Robert Shaw
e2ed238885
Revert "[Fix]Load kv-cache dtype from hf_quant_config.json automatically" ( #30653 )
2025-12-14 19:33:41 -05:00