lesj0610
a66a3e1b4e
Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-09-14 14:36:05 +09:00
wang.yuqi
67111973ee
[Frontend] Move grpc_server to launchers. ( #56746 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-09-14 05:03:08 +00:00
Kevin H. Luu and OpenAI Codex
dc36fcce90
[CI] Collect GPU memory telemetry for MIG slices ( #56747 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-13 21:30:29 -07:00
Kevin H. Luu and OpenAI Codex
214248c7d8
[CI] Move smaller H200 workloads to 18GB MIG slices ( #56695 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-13 21:30:11 -07:00
Hotragn Pettugani and Isotr0py
a2685f2cda
[Bugfix][Multimodal] Validate base64 video payloads, matching image and audio ( #54323 )
...
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in >
Signed-off-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
2026-09-14 04:14:26 +00:00
Chaojun Zhang
39545e475d
Revert "[CI][XPU] Disable model runner V2 for XPU quantization test for some partially pre-quantized models" ( #56394 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-09-14 11:40:13 +08:00
7632f767f0
[Rust Frontend] Support HTTP RL weight synchronization ( #56567 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Codex <noreply@openai.com >
Signed-off-by: Wang Zupeng <zupenwang@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-14 03:37:04 +00:00
Kevin H. Luu
2d9c30a7b5
[CI][Bugfix] Complete FSE fixture R-SWA contract ( #56710 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-09-14 03:30:25 +00:00
siyu and Isotr0py
52dd0d7562
[Fast Start] Fast loader support nnode>1 ( #55468 )
...
Signed-off-by: siyu <mg21330037@smail.nju.edu.cn >
Co-authored-by: Isotr0py <2037008807@qq.com >
2026-09-14 03:05:54 +00:00
CedricHuang and OpenAI Codex
eb42686a30
[Bugfix][Frontend] Support Responses text types in DeepSeek V4.1 ( #56299 )
...
Signed-off-by: CedricHuang <38417461+CedricHwong@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-14 10:43:09 +08:00
Fangzhou Ai
dfa1984e58
[ROCm][Perf] Tune MiniMax-M3 decode top-k for short contexts ( #55235 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
2026-09-13 21:36:27 -05:00
58d45fd767
[Bugfix][KV Connector] MooncakeStore: exclude non-prefix-cacheable (QSA ring) groups; fix align-mode check ( #55027 )
...
Signed-off-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-13 19:18:16 -07:00
liuzhenwei
fe49665eb3
[XPU][CI] update test requirements ( #56704 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-09-14 02:01:50 +00:00
Woosuk Kwon
3b533197a9
[Perf] Parallelize mHC pre-norm JIT warmup ( #56683 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-13 18:58:20 -07:00
Wentao Ye
9f03b510c3
[DSv4 Bug] fix dsv4 start up error NotImplementedError: DeepSeek V4 MegaMoE currently requires expert parallel ( #55914 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-13 20:22:08 -04:00
lesj0610 and Claude Opus 5
3b1911c0ca
[Bugfix][Model] Fix Qwen4Exp MTP buffer placement, config parsing, and Triton warmup
...
Three initialization and config defects found serving Qwen4Exp, each with a
regression test.
**The MTP hidden buffer lands on the ambient device.** torch.empty for the
spec-decode hidden buffer takes no device, so it follows whatever device is
current rather than vllm_config.device_config.device. The allocation was
duplicated verbatim in the NVIDIA and AMD backends; it moves to
common/mtp.py so the two cannot drift, and the caller passes is_last_rank
rather than reaching for the process group from shared code.
**M-RoPE keys are reported as unrecognized.** Qwen4Exp drives M-RoPE through
mrope_section and mrope_interleaved, which transformers' rope validator does
not know for rope_type="default", so every load warns about a configuration
the model genuinely uses. Declare them at rope validation on the text config
and on the outer config that mirrors it; every other key still warns.
**transformers' layer_types spelling is unread.** Its Qwen4Exp config
canonicalizes "full_attention" entries to "qwen_sparse_attention" in memory,
so checkpoints re-exported through it serialize that name. vLLM keys QSA off
"full_attention" plus indexer_n_heads, so map the spelling back.
**Qwen4Exp is missing from the Triton warmup set.** Its GDN prefill and QSA
run on other backends, but the kernels this warmup covers -- the gated RMS
norm, the causal conv, the post-conv preparation, the decode state update --
are Triton whichever prefill kernel is chosen, so they compile on the first
prompt instead. qwen4_exp_mtp stays out: kernel_warmup passes the worker's
own model_config, which is the target model's.
tests/models/qwen4_exp/test_config.py: 15 passed.
tests/model_executor/test_qwen_triton_warmup.py: 6 passed.
ruff check, ruff format --check and mypy (Python 3.10) report what they
already report on the base commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com >
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-09-14 07:25:01 +09:00
Tianyu Guo
319cc5ef19
[Performance][EPD] Reduce Python proxy serialization overhead ( #56657 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-09-13 15:07:12 -07:00
b23433088b
[Attention] Remove DCP indexer interleave guard and test TP1 output parity ( #55802 )
...
Signed-off-by: Stu Cao <stucao@Stus-MacBook-Pro.local >
Signed-off-by: <>
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Stu Cao <stucao@Stus-MacBook-Pro.local >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-13 13:56:28 -07:00
Roberto L. Castro and Codex
186a1e6222
[Warmup] Gemma 4 de-JITification ( #55768 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-13 16:47:21 -04:00
Lucas Wilkinson and OpenAI Codex
586f652f8d
[Bugfix][PCP][DCP] Respect interleave in indexer KV gather mapping ( #56715 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-13 16:45:17 -04:00
Lucas Wilkinson and OpenAI Codex
7fe8fc803d
[NIXL][PCP][DCP] Expose PCP producer KV shards as transfer ranks ( #56645 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-13 20:23:15 +00:00
Woosuk Kwon and OpenAI Codex
71888f507a
[Perf] Avoid redundant conversions in FP8 dummy initialization ( #56688 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-13 12:06:13 -07:00
Lucas Wilkinson and OpenAI Codex
b6e2aa748b
[CI] Initialize warmup registry in V2 QSA runner fixture ( #56707 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-13 13:34:09 -04:00
Kevin H. Luu
dec0b5d63f
[Bugfix] Skip Triton autotune inspection without Triton ( #56676 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-09-13 17:05:29 +00:00
f09c52a587
[ROCm][Performance] Avoid blocking MiniMax M3 scalar upload ( #56170 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-13 16:56:02 +00:00
b7e0cdac5d
[Bugfix] Detect unloaded NVFP4 weight scales with a NaN sentinel ( #52501 )
...
Signed-off-by: pavelzak <pavel.zakharov@gmail.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-13 14:58:09 +00:00
82a85dc1d2
[Attention] Add Triton/FlashInfer composite for multimodal prefix attention ( #56305 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-13 14:29:41 +00:00
Shenglei Fu and Claude Sonnet 5
8cf9de9080
[Test][Determinism] Cover VLM batch invariance in default execution mode ( #56528 )
...
Signed-off-by: Shenglei Fu <sfu@confluent.io >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-09-13 10:22:03 -04:00
linitra24
dd4c841070
[LoRA] Add LoRA support for DeepSeek-V4 Flash Vision ( #55897 )
...
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io >
2026-09-13 07:10:54 -07:00
liuzhenwei
de500290ff
[XPU][CI] fix jit_warmup_triton_launcher ( #56670 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-09-13 21:44:24 +08:00
Taneem Ibrahim
2e9f7bb454
[Pooling] Report actual input token usage for scoring APIs ( #56573 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-13 09:20:21 -04:00
Kebe
c351fd3c64
[Core] Fix ValueError on KV load failure with a hybrid KV cache ( #50388 )
...
Signed-off-by: Kebe <mail@kebe7jun.com >
2026-09-13 21:03:05 +08:00
Tianyu Guo
fa1b3b1922
[Bugfix][EPD] Preserve explicit multimodal UUIDs with caches disabled ( #56432 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-09-13 10:08:30 +00:00
d2d649e674
[DS V4.1][Engram] Support async prefetch for offloaded engram lookups and engram DP sharding ( #56512 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: ywang96 <>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-13 02:44:23 -07:00
Woosuk Kwon
e7a3963339
[Bugfix] Avoid repeated dummy initialization and random CPU Engram fills ( #56682 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-13 02:23:20 -07:00
Ylean and Codex
a987777755
[Bugfix][Gemma4] Keep image kwargs out of video preprocessing ( #56652 )
...
Signed-off-by: Ylean <61880605+Woolgathererer@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-13 09:07:12 +00:00
fa008bdccf
[Perf][Kernel] Integrate DeepSelect TopK for the DSA sparse indexer ( #56464 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-09-13 00:44:37 -07:00
Lucas Wilkinson
1cfd972816
[CI] Fix NIXL transfer-rank geometry fixture ( #56672 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-09-13 07:36:06 +00:00
Lucas Wilkinson
410f6da5c4
[Bugfix][KV Offload] Submit CPU stores on no-forward steps ( #56621 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-09-13 05:56:14 +00:00
Taneem Ibrahim
0cf266a469
[Pooling] Support prompt embeddings in MRV2 decoder pooling ( #56639 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-13 05:49:45 +00:00
Suraj and Claude Opus 5
93e051e290
[Bugfix][Benchmark] Make streaming TTFT/E2E latency accounting consistent across endpoints ( #55508 )
...
Signed-off-by: surajm20061998 <surajm20061998@gmail.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-13 05:20:54 +00:00
Woosuk Kwon and OpenAI Codex
2671fedfc7
[MRV2] Revert explicit Triton JIT warmup migration ( #56654 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-13 04:07:56 +00:00
Markus Hartikainen and Jee Jee Li
c711f740b7
[ROCm] Auto-enable breakable CUDA graphs for DeepseekV41ForCausalLM ( #56349 )
...
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-09-12 20:47:07 -07:00
Ayushman Singh and Ayushman Singh
c2ea9f1a5f
[Kernel][Perf][Quantization] Fix odd-row performance cliff in per-token-group quantization ( #56478 )
...
Signed-off-by: Ayushman Singh <ayush1399@users.noreply.github.com >
Co-authored-by: Ayushman Singh <ayush1399@users.noreply.github.com >
2026-09-12 20:27:08 -07:00
Isotr0py
533dbe424b
[Misc] Clean up fast loader daemon quant method verification ( #55656 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-12 20:16:39 -07:00
Roberto L. Castro and Codex
e52be1a62d
[9/N][warmup][DSv4] Migrate MHC TileLang kernels ( #50178 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-12 17:25:23 -07:00
aed894c190
[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels ( #56323 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-12 17:24:24 -07:00
e19a3e172e
[5/N] Share HiSparse host cache across TP ranks ( reopens #52760 ) ( #56629 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Chao Lei <leichao139636@163.com >
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Chao Lei <leichao139636@163.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-12 19:48:25 -04:00
ebe1dec2da
[PCP][DCP] Enable PCP+DCP on sparse-MLA models ( #56157 )
...
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai >
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai >
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai >
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Patryk Saffer <PatrykSaffer@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-12 23:24:44 +00:00
Matt and Claude Opus 4.8
72d4d83157
[ROCm][Bugfix] Fix elastic EP scaling deadlock ( #56610 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-09-12 23:09:19 +00:00
lesj0610