Commit Graph
23158 Commits
Author SHA1 Message Date
djramic 8882b83b34 [Bugfix] Bump tokenizers to 0.23.2 for duplicate-pattern support (#59796)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
2026-10-02 18:33:50 +00:00
simon-veitner-redhatandLucas Wilkinson 202d497aa8 [Bugfix][Watermarking] Keep draft prompt lengths valid under CUDA graphs (#59779)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-10-02 14:32:11 -04:00
Shijin ZhangandFlora Feng d1a974c140 [Frontend] Constrain non-strict GLM-4.7 tool calls with a shallow structural tag (#56403)
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
2026-10-02 14:31:53 -04:00
Wentao Ye f700178f90 [Refactor] Remove dead env and config (#59781)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-10-02 18:07:20 +00:00
Wentao Ye 4e4d75c459 [GLM5.3 Perf] Reuse sparse MLA index conversion across layers, 3.5~3.9x kernel performance improvement (#59464)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-10-02 17:37:45 +00:00
Thang NguyenandClaude Opus 5.5 7cfd233538 Revert "[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it" (#53020) (#59805)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 10:19:25 -07:00
Stefano CastagnettaandBenjamin Chislett f590eb2448 [Perf][Qwen4Exp] Add SM121 TP=1 skinny-GEMM plans (#59753)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
2026-10-02 17:17:45 +00:00
9cf087fd14 [Perf] Tune MoE weighted-sum kernel launch configuration (#59731)
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-10-02 17:09:29 +00:00
8442117446 Upgrade tpu-inference to v0.30.0 (#59396)
Signed-off-by: Jeff Tseng <tsengjeff@google.com>
Co-authored-by: Jeff Tseng <tsengjeff@google.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-10-02 09:39:03 -07:00
Misha GoinandClaude 1ab18599ca [Agents] Add pre-commit check for agent files and skills (#59657)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-02 09:38:22 -07:00
rmhaskarandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e42d35d4f3 [Minimax-M3] Keep the native FP8 MMA in the Triton indexer scorers (#59481)
Signed-off-by: Rajas Mhaskar <rmhaskar@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 12:30:39 -04:00
djramic 167a81fa9c [CI] Fix ModelExpress handling in weight transfer tests (#59772)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
2026-10-02 11:30:01 -05:00
Divakar Verma 6abadc2aa7 [ROCm][CI] Add test coverage for VLLM_ROCM_MOE_PADDING memory-stride padding transparency (#59332)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
2026-10-02 11:23:22 -05:00
Divakar Verma 2a537887de [CI] Use vllm_runner in fusions_e2e conftest for reliable GPU cleanup (#59525)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
2026-10-02 11:21:21 -05:00
+8 7b6be2f1f4 [Docs] Add Reviewers page (#59459)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-10-02 16:11:31 +00:00
4c4003bb5f [Bugfix] Bind routed-experts capture to the MoE layer, not the kernel (#59455)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 15:50:38 +00:00
0979892992 [Bugfix][Multimodal] Fix GLM-5.3-Flash vision tower crashes on image input (#59126)
Signed-off-by: Juan Calderon-Perez <835733+gaby@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-10-02 15:25:37 +00:00
Taneem Ibrahim f0c44cc7d5 [Bugfix] Fix stalled local-only Elastic EP scale-up (#57820)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-10-02 10:47:55 -04:00
592c6f3fbc [Bugfix][Pooling] Handle multimodal cache misses without crashing (#58975)
Signed-off-by: linchaoying6 <linchaoying6@jd.com>
Co-authored-by: linchaoying6 <linchaoying6@jd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-02 10:14:33 -04:00
tenotekandClaude Fable 5.1 443dccd3c9 [Bugfix][Multimodal] Fix reference counting of the SHM processor cache (#59695)
Signed-off-by: tenotek <50898317+hdq66666@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:14:13 +08:00
aoshen02andClaude Opus 5.5 c91dccc080 [Bugfix][Quark] Pass grouped-routing arguments to OCP MX monolithic kernels (#59752)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 13:46:48 +00:00
e4294144a8 [Perf] Keep non-speculative GDN decode on the standard path (#59735)
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
2026-10-02 13:44:53 +00:00
24df2d3fc7 [Model] Add Qwen4Exp to the Qwen GDN Triton warmup (#56742)
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-10-02 21:38:15 +08:00
Rajeev Warrierandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 30e956fad3 [Docs] Fix stale W4A16 NVFP4 default kernel in ModelOpt docs (#59135)
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 13:21:43 +00:00
Jiangyun ZhuandClaude Opus 5.5 d648847fe4 [CI][Kimi-K3] Test prefix cache reuse with KV offload, P/D and DCP (#59229)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 20:58:35 +08:00
Stefano CastagnettaandClaude Opus 5 f0b4b36890 [Bugfix][Quantization] Detect NVFP4 in ModelOpt mixed-precision checkpoints (#56050)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-02 12:30:22 +00:00
Abhay ShuklaandClaude Opus 5.5 fa212b8dc5 [Test] Re-enable Ovis2.5 and Ovis2.6-MoE vLLM tests on Transformers v5 (#59763)
Signed-off-by: AbhayShuklaIIT <abhay.shukla@live.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 11:59:40 +00:00
pmanczakandPawel Manczak bdfb02674f [XPU][Kernel] Tune Triton W8A8 block-FP8 GEMM for Intel B70 (#56063)
Signed-off-by: Pawel Manczak <203671809+pmanczak@users.noreply.github.com>
Co-authored-by: Pawel Manczak <203671809+pmanczak@users.noreply.github.com>
2026-10-02 12:56:11 +01:00
Turner JabbourandClaude Sonnet 5 e2cb5ee46c [CI] Replace shellcheck-suppressed patterns flagged in #52572 with clean equivalents (#53341)
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-02 11:30:04 +00:00
Harry MellorandClaude Opus 5.5 6e4efafc6e [Model] Migrate GPT-NeoX, Phi, Seed-OSS and Jais2 to the Transformers modeling backend (#59701)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 11:18:35 +00:00
Harry MellorandClaude Opus 5.5 5d20979fc9 [Misc] Remove code paths for Transformers < 5.16.1 (#59762)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 11:17:24 +00:00
xiaoyu-xyzRaina Zhongmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8c60714da5 [Bugfix][LoRA] Fall back for high-rank MoE LoRA (#55161)
Signed-off-by: Raina Zhong <yuweiz@nvidia.com>
Co-authored-by: Raina Zhong <yuweiz@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 13:59:28 +03:00
Simon Danielssonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 98b29aa99a [ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes (#58167)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 11:47:41 +01:00
Simon Danielssonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 751fa96a49 [ROCm][Perf][GLM-5.3-Flash] Remove redundant copy after ragged sparse MLA (#58569)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 11:45:51 +01:00
5688a4dd4a [Bugfix][GLM-5.3] Size the image encoder cache from the exact token ceiling (#59565)
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-10-02 10:43:29 +00:00
af2a582d5c [Bugfix][Spec Decode] Qualify Transformers backend attention layer names with the model prefix (#59293)
Signed-off-by: Farhad Al-Amin <99006654+Legend-2727@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-10-02 11:21:20 +01:00
aoshen02andClaude Opus 5.5 8e58ac22af [Feature] Offload KV-init runtime state on sleep (#59158)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 09:49:21 +00:00
Juan Pérez de Algaba c15672cfb4 [Security] Accept zero-sum DeepSeek-OCR pixel tensors (#59417)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
2026-10-02 09:17:12 +00:00
Isotr0pyandHarry Mellor 58b3298457 [Misc] Add Transformers version upper bound in requirements (#59614)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-10-02 08:27:11 +00:00
9d2a6f52b2 [Rust Frontend] Expose gRPC port in Python vllm serve (#59659)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-10-02 08:14:37 +00:00
Isotr0py b558f160a2 [Bugfix] Fix minimax-m3 multimodal processor compatability with Transformers v5.18 (#59613)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-10-02 07:29:49 +00:00
kliuae 0154979656 [ROCm][Perf] Kimi-K3 enable prefill checkpoints on ROCm (#58344)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
2026-10-02 06:58:04 +00:00
4056c8ac1f [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-10-01 23:04:21 -07:00
0cbac6cd13 [Model][Spec Decode] Enable EAGLE3/DSpark pipeline parallelism for Sarvam MLA (#55902)
Signed-off-by: mohit-sarvam <mohit@sarvam.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-10-02 04:28:19 +00:00
f7999d2e44 [Bugfix][HiSparse] Size the KV cache from the groups HiSparse allocates (#59450)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-10-02 04:11:55 +00:00
Teng MaandTeng Ma 1a9eaa3a68 [Doc] Document ECMooncakeConnector for EPD disaggregation (#59196)
Signed-off-by: Teng Ma <stmatengss@gmail.com>
Co-authored-by: Teng Ma <stmatengss@users.noreply.github.com>
2026-10-02 12:10:33 +08:00
Qiu ChunshuoandLucas Wilkinson 8486f1ebfe [Perf][PCP] Shard decode requests across PCP ranks (#52162)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-10-01 23:50:13 -04:00
a810316c08 [Bugfix] Keep batch-invariance NCCL pins out of the weight-transfer group (#59500)
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
2026-10-02 11:19:28 +08:00
Martin Hickey fcfacc3499 [Bugfix][Frontend] Seed derender detokenization from the prompt (#59046)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-10-02 03:17:56 +00:00
Harry MellorandClaude Opus 5.5 8faacd707f [Model] Migrate Glm, Arcee, CWM and Mellum to the Transformers modeling backend (#59679)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 03:15:14 +00:00