MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
13dbf96093
[Model Loader] Apply EP weight filtering in InstantTensor ( #61167 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-14 09:10:10 -07:00
Ronald and aoshen02
19aba3e4d5
[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload ( #57797 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-14 10:17:58 +00:00
Woosuk Kwon and Claude Opus 5.5
f47792af7f
[Model Loader] Make dummy weight init dtype-aware ( #61134 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 18:33:26 -07:00
f38c6778ce
[quantization] Add per-token NVFP4 quantization MoE support for ReLU2 ( #56740 )
...
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-07 22:01:09 +00:00
e572e02d21
[UX] Skip checkpoint shards an MTP head does not load ( #60152 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 15:16:19 -04:00
6936e77ba4
[Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages ( #56926 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 10:52:02 +08:00
90a9515006
[Cleanup] Remove online quantization support in fp8.py in favor of online shorthands ( #53585 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-24 12:48:14 -07:00
1291bdb85e
[EPD][Model Loader] Skip language-model checkpoint shards for --mm-encoder-only ( #58086 )
...
Signed-off-by: grYe99 <guorongye99@gmail.com >
Co-authored-by: grYe99 <guorongye99@gmail.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-09-24 00:41:15 -07:00
Wentao Ye
fae5cbfc8b
[Refactor] Remove dead code multiple places ( #58002 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-22 16:36:03 -04:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
0f8fa53acf
[Bugfix][KV Offload] Check cgroup memory before SHM allocation ( #54014 )
...
Signed-off-by: Alex <jihui.huang@daocloud.io >
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com >
Signed-off-by: Alex <alex.tech.lab@outlook.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
2026-09-16 09:17:50 +03:00
Woosuk Kwon and OpenAI Codex
71888f507a
[Perf] Avoid redundant conversions in FP8 dummy initialization ( #56688 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-13 12:06:13 -07:00
Woosuk Kwon
e7a3963339
[Bugfix] Avoid repeated dummy initialization and random CPU Engram fills ( #56682 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-13 02:23:20 -07:00
Roderick Wu and mgoin
2c9d68f80b
[Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering ( #54809 )
...
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com >
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-08 12:45:12 -07:00
Wentao Ye
6b15bea080
[Refactor] Remove utils dead code ( #53941 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-08 11:22:09 -04:00
5690b02c03
[Quantization] Support online quantization with partially pre-quantized checkpoints ( #51392 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-04 11:46:06 -07:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
Harry Mellor
70b84f0bcb
Fix Gemma 4 for upcoming Transformers version ( #49797 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-10 07:48:26 +00:00
Isotr0py
073c510c91
[Migration] Migrate bitsandbytes support to OOT plugin ( #43529 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-08-09 10:27:31 +08:00
Chauncey and Andreas Karatzas
03a2d03367
[Bugfix] Respect cgroup memory limits on all platforms ( #49966 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-28 14:38:20 +08:00
Michael Goin and OpenAI Codex
cc25f028b7
[Loader] Improve InstantTensor loading ( #46868 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-17 16:30:02 -04:00
Andreas Karatzas
3ad85e0de4
[CI][AMD] Configure MI300 tests for native execution without DinD ( #48387 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-15 02:14:16 +00:00
85c09e9885
fix: correct load_weights track logic and enable weight integrity for… ( #41811 )
...
Signed-off-by: Yipeng Hu <i26268@metax-tech.com >
Signed-off-by: HuYiPeng <144002351+MynameFelix@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Yipeng Hu <i26268@metax-tech.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-07-10 14:08:20 +00:00
9d808e2309
[Core] Use fastsafetensors ParallelLoader for weight loading ( #40183 )
...
Signed-off-by: Git Bisector <gitbisector@gmail.com >
Signed-off-by: gitbisector <gitbisector@gmail.com >
Signed-off-by: git bisector <gitbisector@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-06-15 22:32:05 -07:00
Isotr0py
6635279d8a
[Migration] Migrate GGUF quantization support to plugin ( #39612 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-06-12 12:02:21 -07:00
Harry Mellor
18d87a87dc
Deprecate Transformers v4 support ( #45161 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-11 11:04:01 +08:00
Tyler Michael Smith and Claude Opus 4.6
fa8c868a3c
[Bugfix] Fix Llama4 weight loading ( #45047 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-06-10 13:40:45 -04:00
bnellnm
dc68bd8c41
[MoE Refactor] FusedMoE/MoERunner inversion refactor ( #41184 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-06-08 10:42:58 -04:00
Harry Mellor and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
62215e72c6
Remove KV cache scale boilerplate from model weight loading methods ( #43167 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-05 05:19:04 -07:00
Lucain and Claude Opus 4.6
11dfa3169d
Add vLLM library info to Hugging Face Hub requests ( #43857 )
...
Signed-off-by: Wauplin <lucainp@gmail.com >
Signed-off-by: Lucain Pouget <lucain@huggingface.co >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-05-29 14:04:58 +00:00
Benjamin Bartels
05eec7120e
Fix RunAI streamer tensor buffer reuse during weight loading ( #43464 )
...
Signed-off-by: bbartels <benjamin@bartels.dev >
2026-05-27 19:16:52 -07:00
Michael Goin
8efd508204
[Quantization] Rework quantization_config to use QuantKey and allow for activation override ( #41566 )
2026-05-13 16:58:32 -04:00
Mohammad Miadh Angkad
21943d4c25
[Performance] Make safetensors checkpoint prefetch settings configurable ( #41499 )
...
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu >
2026-05-10 15:55:15 +00:00
stevenkuang and Jee Jee Li
d0009ddb0b
[Model] Support Hy3 preview ( #40681 )
...
Signed-off-by: stevenkuang <stevenkuang@tencent.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-23 22:08:26 +08:00
Asaf Gardin
19fa90ed0d
[Quantization] - Layerwise reloading of Attention/KV quantized models ( #38995 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
2026-04-15 18:03:32 -07:00
Artem Perevedentsev
bb6047db13
[Model][Perf] Enable checkpoints prefetching for Lustre FS by default ( #39422 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-04-10 00:47:59 +00:00
Vasiliy Kuznetsov
7b1a7423be
[Frontend] new online quantization frontend ( #38138 )
...
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com >
2026-04-03 11:58:39 -04:00
Asaf Gardin
1fc69f59bb
[Bug fix][Quantization] Fix dummy weight loading ( #38478 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
2026-03-30 16:38:02 -04:00
PikaPikachu
63babd17f1
[Model][Quantization] Add GGUF support for MiniMax-M2.1 ( #36965 )
...
Signed-off-by: kangletian <Letian.Kang@amd.com >
2026-03-30 14:24:06 +08:00
Kyle Sayers
648edcf729
[QeRL] Compose online quantization with quantized reloading ( #38032 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-03-27 13:22:33 -07:00
Andreas Karatzas
f2d16207c7
[ROCm][CI] Fix flaky GPTQ compile correctness test ( #38161 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-03-26 19:57:00 +08:00
Artem Perevedentsev
a93a53f8a1
[Performance] Auto-enable prefetch on NFS with RAM guard ( #37673 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-03-24 17:31:14 -07:00
Harry Mellor
b3601da6e7
[Mypy] Fix mypy for vllm/model_executor (except vllm/model_executor/layers) ( #37904 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-24 17:14:01 +00:00
Artem Perevedentsev
f5e59ee7a6
[Performance] Add prefetch for checkpoints to OS page cache ( #36012 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-03-16 11:32:02 +00:00
Roy Wang
821eb80c0d
[Performance][Model Loader] Skip non-local expert weights during EP model loading ( #37136 )
...
Signed-off-by: esmeetu <jasonailu87@gmail.com >
2026-03-16 01:33:36 -07:00
arlo
8c29042bb9
[Feature] Add InstantTensor weight loader ( #36139 )
2026-03-14 18:05:23 +01:00
Isotr0py
a8e8d62dd8
[Misc] Clean up Kimi-audio whisper encoder loading ( #36903 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-03-14 23:37:52 +08:00
Shaun Kotek
90512b2e8b
fix: Use iterator as not to store all the file loads in memory at once ( #36149 )
...
Signed-off-by: Shaun Kotek - Nvidia <skotek@nvidia.com >
2026-03-08 20:25:21 -07:00