lesj0610 and Claude Opus 5
3b1911c0ca
[Bugfix][Model] Fix Qwen4Exp MTP buffer placement, config parsing, and Triton warmup
...
Three initialization and config defects found serving Qwen4Exp, each with a
regression test.
**The MTP hidden buffer lands on the ambient device.** torch.empty for the
spec-decode hidden buffer takes no device, so it follows whatever device is
current rather than vllm_config.device_config.device. The allocation was
duplicated verbatim in the NVIDIA and AMD backends; it moves to
common/mtp.py so the two cannot drift, and the caller passes is_last_rank
rather than reaching for the process group from shared code.
**M-RoPE keys are reported as unrecognized.** Qwen4Exp drives M-RoPE through
mrope_section and mrope_interleaved, which transformers' rope validator does
not know for rope_type="default", so every load warns about a configuration
the model genuinely uses. Declare them at rope validation on the text config
and on the outer config that mirrors it; every other key still warns.
**transformers' layer_types spelling is unread.** Its Qwen4Exp config
canonicalizes "full_attention" entries to "qwen_sparse_attention" in memory,
so checkpoints re-exported through it serialize that name. vLLM keys QSA off
"full_attention" plus indexer_n_heads, so map the spelling back.
**Qwen4Exp is missing from the Triton warmup set.** Its GDN prefill and QSA
run on other backends, but the kernels this warmup covers -- the gated RMS
norm, the causal conv, the post-conv preparation, the decode state update --
are Triton whichever prefill kernel is chosen, so they compile on the first
prompt instead. qwen4_exp_mtp stays out: kernel_warmup passes the worker's
own model_config, which is the target model's.
tests/models/qwen4_exp/test_config.py: 15 passed.
tests/model_executor/test_qwen_triton_warmup.py: 6 passed.
ruff check, ruff format --check and mypy (Python 3.10) report what they
already report on the base commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com >
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-09-14 07:25:01 +09:00
Taneem Ibrahim
2e9f7bb454
[Pooling] Report actual input token usage for scoring APIs ( #56573 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-13 09:20:21 -04:00
Ylean and Codex
a987777755
[Bugfix][Gemma4] Keep image kwargs out of video preprocessing ( #56652 )
...
Signed-off-by: Ylean <61880605+Woolgathererer@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-13 09:07:12 +00:00
Kevin H. Luu and OpenAI Codex
1ee4be4dbc
[CI] Update DeepSeek V4.1 MegaMoE routing test ( #56599 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-12 11:31:13 -07:00
Taneem Ibrahim
1b29c508bc
[Bugfix] Initialize data parser in Nano-Nemotron audio test ( #56401 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-12 09:07:20 -04:00
Taneem Ibrahim
a0914ab7d0
[Nano-Nemotron] Fix Nano-Nemotron precomputed multimodal embeddings ( #56398 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-12 09:07:10 -04:00
Wei Zhao
9d88ceb026
[KDA] Update flashKDA to support bf16 checkpoint state ( #56485 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-09-11 23:53:24 +00:00
Andreas Karatzas and OpenAI Codex
9dd969da09
[Model][ROCm] Enable DeepSeek V4 Vision ( #55107 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-11 09:17:30 -07:00
Shanshan Shen
127143e27f
Revert "[Rocm][Kimi-k3] Fix pipeline_parallel support for the kimik3 DCP mode ( #53664 )" ( #56429 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-11 09:17:26 -07:00
Andreas Karatzas
eb7e894432
[ROCm][CI] Add HY-V4 generation coverage ( #55667 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-09-11 17:24:28 +08:00
e77daef89e
[Model] Support DeepSeek-V4.1-Flash ( #56214 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-11 02:11:19 -07:00
Tyler Michael Smith and OpenAI Codex
a89de950a4
[CI/Build] Pin HyperCLOVAX V2 test model revision ( #56335 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-10 16:26:11 -07:00
siyu and Isotr0py
73fb19151f
[Fast Start] Support fp4 ( #55465 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 13:22:17 +00:00
9b959b8657
[Model] DeepSeek-V4.1-Flash Model Definitions ( #56228 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-10 20:16:26 +08:00
Wentao Ye
86aca66191
[Kimi K3 Perf] Avoid KDA mixed-batch gather/scatter, 5.2%~7.7% E2E Throughput Improvement ( #56159 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-10 17:57:01 +08:00
fe26c705f7
[Model] Add Bailing V3 VL support ( #55921 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 02:46:56 -07:00
Jared Wen and Claude Fable 5.1
1768273c13
[Perf][GLM-5.3-Flash] Decode hot-path cleanups: strided KDA recurrent inputs, NoPE MQA query without concat, no duplicate router GEMM ( #55736 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-10 16:28:33 +08:00
9521c60bdc
[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill reland ( #54038 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Signed-off-by: kliuae <17350011+kliuae@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-09-10 15:37:54 +08:00
Yunzhu Lu and Isotr0py
442d36031c
[MM][CG] Enable encoder CUDA Graph for MiniCPM-V ( #42785 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 06:28:48 +00:00
Li, Jiang
c3ccc0e957
[Model] Add DeepSeek-V4 CPU backend ( #55355 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-10 14:14:48 +08:00
Qiming Zhang
6e1d051fc9
[CI][XPU] skip test_models_text on XPU ( #56038 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
2026-09-10 11:03:09 +08:00
djramic
cce50657b7
[ROCm][CI] Use a platform-independent GEMM in the merged-column fuser test ( #56130 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-09-09 15:08:39 -05:00
22d95d1adc
[Bugfix] Make mm_device_do_normalize encoder-cudagraph safe ( #55370 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 01:43:22 +08:00
Harry Mellor and Claude Opus 5
719284fe15
[Core][Model] Unify XD-RoPE into M-RoPE and derive the channel count ( #56078 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-09 09:23:30 -07:00
d8d53f17c2
[Rocm][Kimi-k3] Add pipeline_parallel support for the kimik3 model ( #53664 )
...
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-09 07:50:24 -07:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
3116c5d06b
[Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism ( #54371 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 07:32:34 -07:00
Thomas Ortner and Harry Mellor
e509d32b5a
[Transformers backend] Enable QKV-Fuser for Gemma4 ( #55690 )
...
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com >
Signed-off-by: Thomas Ortner <bohnstinglthomas@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-09 14:46:31 +01:00
Michael Lazos and OpenAI Codex
28a73ccfba
[Kernel] Add NVFP4 support to the torch linear backend ( #53319 )
...
Signed-off-by: Michael Lazos <mlazos@meta.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 20:30:05 -04:00
Taneem Ibrahim
5af4cc33ec
[Bugfix] Fix OpenPangu multimodal embedding merge ( #55941 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-08 23:53:59 +00:00
c0d8d5413e
[Transformers] Generalize merged-column linear fusion ( #55301 )
...
Signed-off-by: BANANASJIM <bananasjim1@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-08 23:49:37 +00:00
60ad959b6f
Fix block FP8 MTP in ModelOpt mixed checkpoints ( #55513 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-08 22:47:21 +00:00
Andreas Karatzas
cb22234687
[CI] Reuse ColQwen3 models across pooling tests ( #55889 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-09-08 14:03:54 -07:00
Roderick Wu and mgoin
2c9d68f80b
[Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering ( #54809 )
...
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com >
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-08 12:45:12 -07:00
Wentao Ye
6b15bea080
[Refactor] Remove utils dead code ( #53941 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-08 11:22:09 -04:00
Sherif Waly and Nicolò Lucchesi
f998862d46
[Bugfix] Fix Kimi K3 loading with interleaved weight streams ( #53379 )
...
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
2026-09-08 16:08:51 +02:00
414057a3d3
[Bugfix] Apply attention sinks in the Transformers backend ( #52156 )
...
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-08 13:54:07 +00:00
Kevin H. Luu
34b9899c8f
[CI] Increase ColQwen3 pooling test memory budget on H200 MIG ( #55878 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-09-08 11:39:01 +00:00
Wei Zhao
a69402aaa8
[Perf] Integrate FlashInfer KDA kernels ( #55364 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-09-07 20:29:30 -07:00
Andreas Karatzas
6b5a12c0f8
[ROCm][CI] Enable HY-V4 model initialization on ROCm ( #54405 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-09-07 20:55:44 -05:00
Sahil Patel
51da0ca66c
[CI/Build] Unskip ColQwen3 multimodal pooling tests on Transformers v5 ( #55588 )
...
Signed-off-by: Sahil <sip4818@gmail.com >
2026-09-07 15:15:39 +00:00
70584f69b1
[Model] Add Cohere Compass model ( #54774 )
...
Signed-off-by: am-cohere <312513976+am-cohere@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 07:36:49 -07:00
Abhi and Antigravity
7cc89a7dda
[Tests] Update SarvamMLA transformers v5 compatibility reason to hf ( #55728 )
...
Signed-off-by: Abhi <108084481+Aj2280@users.noreply.github.com >
Co-authored-by: Antigravity <antigravity@google.com >
2026-09-07 14:27:07 +00:00
f2d45f26bd
[Bugfix][Gemma] Conditionally create KV projections/norms on KV-shared layers ( #54917 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
Co-authored-by: minyichen <96753147+cyc00518@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-07 15:04:00 +01:00
42801b3a6b
[3/N][warmup][DSv4] Migrate FA4 MLA and shared CuTeDSL kernels ( #53565 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 09:36:35 -04:00
b339d75a41
[Bugfix][Spec Decode] Resolve n_predict from text_config for Qwen3.5 multimodal MTP ( #55369 )
...
Signed-off-by: SOUMYAJIT GHOSH <23051387@kiit.ac.in >
Signed-off-by: Soumyajit Ghosh <jobsoumyajit6124@gmail.com >
Co-authored-by: SOUMYAJIT GHOSH <23051387@kiit.ac.in >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 13:28:26 +00:00
Thien Tran
94e26dd3dd
[Qwen3.8-Flash-Next] Support FP8 indexer cache for QSA ( #54890 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-07 12:15:45 +00:00
Andreas Karatzas
195bc9c4a1
[CI][ROCm] Temporarily skip unsupported HY-V4 initialization ( #55653 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-09-07 15:17:36 +08:00
Thien Tran
d9105ea800
[Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation ( #55272 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-06 21:57:56 -07:00
Ziming Huang and OpenAI Codex
144e79c810
[Kimi K3] Support internal prefix checkpoints with partial prefix caching and spec-decoding ( #53614 )
...
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-06 13:18:15 +08:00
Harry Huang
28e605fb33
[Bugfix][Qwen4Exp] fix state index strides in fused PLE conv ( #55375 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
2026-09-05 14:02:20 +00:00
