iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,277 Commits 3 Branches 0 Tags
a66a3e1b4e3653005e8d4ffb8043f3e93592da56
Commit Graph
7951 Commits
Author SHA1 Message Date
lesj0610 a66a3e1b4e Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-14 14:36:05 +09:00
wang.yuqi 67111973ee [Frontend] Move grpc_server to launchers. (#56746) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-09-14 05:03:08 +00:00
Kevin H. Luu 2d9c30a7b5 [CI][Bugfix] Complete FSE fixture R-SWA contract (#56710) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-09-14 03:30:25 +00:00
CedricHuangandOpenAI Codex eb42686a30 [Bugfix][Frontend] Support Responses text types in DeepSeek V4.1 (#56299) 
Signed-off-by: CedricHuang <38417461+CedricHwong@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-14 10:43:09 +08:00
Fangzhou Ai dfa1984e58 [ROCm][Perf] Tune MiniMax-M3 decode top-k for short contexts (#55235) 
Signed-off-by: fai <fangzhouai@gmail.com>
 2026-09-13 21:36:27 -05:00
Zhewen LiZhewen LiOpenAI CodexClaude Fable 5.1
 		58d45fd767 [Bugfix][KV Connector] MooncakeStore: exclude non-prefix-cacheable (QSA ring) groups; fix align-mode check (#55027) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-13 19:18:16 -07:00
lesj0610andClaude Opus 5 3b1911c0ca [Bugfix][Model] Fix Qwen4Exp MTP buffer placement, config parsing, and Triton warmup 
Three initialization and config defects found serving Qwen4Exp, each with a
regression test.

**The MTP hidden buffer lands on the ambient device.** torch.empty for the
spec-decode hidden buffer takes no device, so it follows whatever device is
current rather than vllm_config.device_config.device. The allocation was
duplicated verbatim in the NVIDIA and AMD backends; it moves to
common/mtp.py so the two cannot drift, and the caller passes is_last_rank
rather than reaching for the process group from shared code.

**M-RoPE keys are reported as unrecognized.** Qwen4Exp drives M-RoPE through
mrope_section and mrope_interleaved, which transformers' rope validator does
not know for rope_type="default", so every load warns about a configuration
the model genuinely uses. Declare them at rope validation on the text config
and on the outer config that mirrors it; every other key still warns.

**transformers' layer_types spelling is unread.** Its Qwen4Exp config
canonicalizes "full_attention" entries to "qwen_sparse_attention" in memory,
so checkpoints re-exported through it serialize that name. vLLM keys QSA off
"full_attention" plus indexer_n_heads, so map the spelling back.

**Qwen4Exp is missing from the Triton warmup set.** Its GDN prefill and QSA
run on other backends, but the kernels this warmup covers -- the gated RMS
norm, the causal conv, the post-conv preparation, the decode state update --
are Triton whichever prefill kernel is chosen, so they compile on the first
prompt instead. qwen4_exp_mtp stays out: kernel_warmup passes the worker's
own model_config, which is the target model's.

tests/models/qwen4_exp/test_config.py: 15 passed.
tests/model_executor/test_qwen_triton_warmup.py: 6 passed.
ruff check, ruff format --check and mypy (Python 3.10) report what they
already report on the base commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-14 07:25:01 +09:00
Tianyu Guo 319cc5ef19 [Performance][EPD] Reduce Python proxy serialization overhead (#56657) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-09-13 15:07:12 -07:00
Lucas WilkinsonandOpenAI Codex 586f652f8d [Bugfix][PCP][DCP] Respect interleave in indexer KV gather mapping (#56715) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-13 16:45:17 -04:00
Lucas WilkinsonandOpenAI Codex 7fe8fc803d [NIXL][PCP][DCP] Expose PCP producer KV shards as transfer ranks (#56645) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-13 20:23:15 +00:00
Lucas WilkinsonandOpenAI Codex b6e2aa748b [CI] Initialize warmup registry in V2 QSA runner fixture (#56707) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-13 13:34:09 -04:00
Kevin H. Luu dec0b5d63f [Bugfix] Skip Triton autotune inspection without Triton (#56676) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-09-13 17:05:29 +00:00
Matthew BonanniCodexLucas WilkinsonLucas WilkinsonOpenAI Codex
 		82a85dc1d2 [Attention] Add Triton/FlashInfer composite for multimodal prefix attention (#56305) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-13 14:29:41 +00:00
Shenglei FuandClaude Sonnet 5 8cf9de9080 [Test][Determinism] Cover VLM batch invariance in default execution mode (#56528) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-13 10:22:03 -04:00
liuzhenwei de500290ff [XPU][CI] fix jit_warmup_triton_launcher (#56670) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-09-13 21:44:24 +08:00
Taneem Ibrahim 2e9f7bb454 [Pooling] Report actual input token usage for scoring APIs (#56573) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-13 09:20:21 -04:00
Kebe c351fd3c64 [Core] Fix ValueError on KV load failure with a hybrid KV cache (#50388) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2026-09-13 21:03:05 +08:00
Tianyu Guo fa1b3b1922 [Bugfix][EPD] Preserve explicit multimodal UUIDs with caches disabled (#56432) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-09-13 10:08:30 +00:00
Juntian LiuRoger Wangywang96 <>Woosuk KwonCodex
 		d2d649e674 [DS V4.1][Engram] Support async prefetch for offloaded engram lookups and engram DP sharding (#56512) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: ywang96 <>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-13 02:44:23 -07:00
YleanandCodex a987777755 [Bugfix][Gemma4] Keep image kwargs out of video preprocessing (#56652) 
Signed-off-by: Ylean <61880605+Woolgathererer@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-13 09:07:12 +00:00
Jiangyun ZhuKimi CodeRoger Wang
 		fa008bdccf [Perf][Kernel] Integrate DeepSelect TopK for the DSA sparse indexer (#56464) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-09-13 00:44:37 -07:00
Lucas Wilkinson 1cfd972816 [CI] Fix NIXL transfer-rank geometry fixture (#56672) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-13 07:36:06 +00:00
Lucas Wilkinson 410f6da5c4 [Bugfix][KV Offload] Submit CPU stores on no-forward steps (#56621) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-13 05:56:14 +00:00
SurajandClaude Opus 5 93e051e290 [Bugfix][Benchmark] Make streaming TTFT/E2E latency accounting consistent across endpoints (#55508) 
Signed-off-by: surajm20061998 <surajm20061998@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-13 05:20:54 +00:00
Woosuk KwonandOpenAI Codex 2671fedfc7 [MRV2] Revert explicit Triton JIT warmup migration (#56654) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-13 04:07:56 +00:00
Markus HartikainenandJee Jee Li c711f740b7 [ROCm] Auto-enable breakable CUDA graphs for DeepseekV41ForCausalLM (#56349) 
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-12 20:47:07 -07:00
Ayushman SinghandAyushman Singh c2ea9f1a5f [Kernel][Perf][Quantization] Fix odd-row performance cliff in per-token-group quantization (#56478) 
Signed-off-by: Ayushman Singh <ayush1399@users.noreply.github.com>
Co-authored-by: Ayushman Singh <ayush1399@users.noreply.github.com>
 2026-09-12 20:27:08 -07:00
Roberto L. CastroandCodex e52be1a62d [9/N][warmup][DSv4] Migrate MHC TileLang kernels (#50178) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-12 17:25:23 -07:00
Roberto L. CastroLucas WilkinsonOpenAI Codex
 		aed894c190 [6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-12 17:24:24 -07:00
Lucas WilkinsonChao LeiClaude Fable 5Matthew BonanniOpenAI CodexOpenAI Codex
 		e19a3e172e [5/N] Share HiSparse host cache across TP ranks (reopens #52760) (#56629) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Chao Lei <leichao139636@163.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Chao Lei <leichao139636@163.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-12 19:48:25 -04:00
PatrykSafferpatryk.safferpatryk.safferLucas WilkinsonPatryk SafferClaude Fable 5.1
 		ebe1dec2da [PCP][DCP] Enable PCP+DCP on sparse-MLA models (#56157) 
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patryk Saffer <PatrykSaffer@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-12 23:24:44 +00:00
Raphaël RiallandOpenAI CodexSimon VeitnerTomas Ruiz
 		7ee8a6dd01 [watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
 2026-09-12 18:42:01 +00:00
Kevin H. LuuandOpenAI Codex 1ee4be4dbc [CI] Update DeepSeek V4.1 MegaMoE routing test (#56599) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-12 11:31:13 -07:00
Kevin H. LuuandOpenAI Codex fd7cd4b883 [CI] Fix Qwen3 Omni DSpark load test config (#56600) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-12 11:29:58 -07:00
Ayushman Singhandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 7bcca16729 [Bugfix] Fix DeepGEMM FP8 warmup coverage (#56452) 
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-12 14:20:38 -04:00
Nicolò LucchesiLucas WilkinsonOpenAI Codex
 		29332cf936 [4/N] Expose HiSparse cache metrics via KV connector stats (#56061) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-12 10:47:55 -07:00
Roberto L. Castro c377114636 [Bugfix][CI] Update CuTeDSL indexer Q sentinel test for migrated wrappers (#56594) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
 2026-09-12 09:12:58 -07:00
Itay Alroy 658c8131c7 [Elastic EP] Reuse CUDA graphs across reconfiguration (#54985) 2026-09-12 10:26:39 -04:00
Woosuk Kwon 13e221f830 [Perf] Fuse DSV4.1 input metadata preparation with Triton (#56562) 2026-09-12 07:20:04 -07:00
liuzhenwei 2f59050eda [XPU][CI] Skip ROCm test on non-ROCm platforms (#56555) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-09-12 21:26:55 +08:00
JartX 986e2f870a [Refactor][ROCm] Migrate the RDNA3 W4A16 MoE to the oracle/experts pa… (#55522) 
Signed-off-by: JartX <sagformas@epdcenter.es>
 2026-09-12 21:20:26 +08:00
Taneem Ibrahim 1b29c508bc [Bugfix] Initialize data parser in Nano-Nemotron audio test (#56401) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-12 09:07:20 -04:00
Taneem Ibrahim a0914ab7d0 [Nano-Nemotron] Fix Nano-Nemotron precomputed multimodal embeddings (#56398) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-12 09:07:10 -04:00
Walter Beller-MoralesandFlora Feng 756794a9a7 [Frontend] create unified Cohere parser (#56392) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-12 04:04:46 -04:00
Canlin GuoandCodex dca96bf97b [Bugfix][Spec Decode] Avoid fastsafetensors deadlock for PP draft models (#54416) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-12 14:22:30 +08:00
Matthew BonanniOpenAI CodexLucas WilkinsonLucas WilkinsonNicolò LucchesiOpenAI CodexClaude Opus 5Lucas Wilkinson
 		d43bb2f37f [3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-12 00:36:47 -04:00
Roberto L. CastroandCodex 120ec4ebd2 [5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels (#53566) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-12 00:05:59 -04:00
Yizheng Jiao 56d7270138 [Bugfix][NIXL] Don't evict a remote engine a transfer is still reading from (#54689) 
Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
 2026-09-11 18:06:09 -07:00
Fangzhou AiandFangzhou Ai 46d2b23ac5 [ROCm][DSV4.1][Perf] Use AITER mHC for the delayed pre block (#56503) 
Signed-off-by: Fangzhou Ai <fangzhou@semianalysis.ai>
Co-authored-by: Fangzhou Ai <fangzhou@semianalysis.ai>
 2026-09-12 00:35:48 +00:00
Wei Zhao 9d88ceb026 [KDA] Update flashKDA to support bf16 checkpoint state (#56485) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-09-11 23:53:24 +00:00