iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,882 Commits 3 Branches 0 Tags
721d0e5c112a2e4e0bed9078fc8d953eaaf5abc2
Commit Graph
21882 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Fangzhou AiandClaude Opus 5.5 721d0e5c11 [ROCm][DSv4.1][Perf] Emit MXFP8 from the sparse decode reduce and run wo_a as a grouped FP8 GEMM (#58456) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 11:58:14 +00:00
liuzhenwei dcfc17e0b1 [XPU] enable XPU GRAPH by default (#51600) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-09-24 19:17:25 +08:00
Zijing Liu f5a78f2ad7 [Multimodal] Reuse the supplied tokenizer in the MiniMax-M3 VL processor (#58460) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
 2026-09-24 10:47:50 +00:00
SHREY GAJJAR 5747d4500a [Perf][Frontend] Offload streaming derender detokenization (#57528) 
Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com>
 2026-09-24 10:39:21 +00:00
Ganesh RCursorLi, Jiang <jiang1.li@intel.com>
 		f34a0e0797 [CPU] Gate the AVX10.2 paths on compiler support (#58133) 
Signed-off-by: R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-24 18:07:53 +08:00
Ziyang Ma cfd5c20286 [XPU][UT] Align HF and vLLM inputs for Qwen2 embedding test by preventing Sentence Transformers from applying chat template (#58117) 
Signed-off-by: RyanMa29 <ziyang.ma@intel.com>
 2026-09-24 10:07:37 +00:00
ReidBugen ZhaoClaude Opus 5.5
 		e430f7421f [Bugfix][Rust Frontend] Prevent MM timing from enabling debug tracing (#58378) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 09:22:23 +00:00
Taneem Ibrahim bbd7c24d6e [MRV2] Validate MRV2 entrypoint logits processors (#57728) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-24 09:07:26 +00:00
Priyansh JainCursorAndreas Karatzas
 		5963795ec7 [Attention][CPU] Run Zen CPU encoder attention on zentorch SDPA (#54508) 
Signed-off-by: priyansh jain <priyansh.jain2@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 02:05:44 -07:00
Yejing LaiandKunshang Ji 14f98cbe5d [Bugfix][XPU] store the pointer raw bit pattern instead of its numeric value (#54514) 
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-24 02:05:26 -07:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		c0c35ddd8e [Bugfix][CI] Fix the flaky sharded-sampling tests, and the engine teardown need (#58342) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 02:05:11 -07:00
Andreas KaratzasandOpenAI Codex 23110a0c9e [Bugfix] Capture prefill kernels for mixed FULL graphs (#58275) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-24 02:05:07 -07:00
jackLeiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> bbc4ddeca9 [Bugfix][V1] Reject encoder-cache hits with mismatched embedding counts (#57696) 
Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 08:40:47 +00:00
Li, Jiang b44895cf93 [CPU] Use pre-built triton (#58140) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-09-24 16:12:34 +08:00
Talor Abramovich 28282ffe34 [Feature][Frontend] Request JSON body debug logging on --enable-log-requests flag (#58163) 
Signed-off-by: talora <talora@nvidia.com>
 2026-09-24 08:08:40 +00:00
Thien TranandOpenAI Codex 00b7847c80 [Perf] Use Conv3dLayer for MiniMax M3 patch embedding (#58512) 
Signed-off-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-24 00:41:36 -07:00
GuorongYegrYe99Cursor Agent
 		1291bdb85e [EPD][Model Loader] Skip language-model checkpoint shards for --mm-encoder-only (#58086) 
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-09-24 00:41:15 -07:00
Andreas KaratzasandOpenAI Codex c3f5270270 [ROCm][CI] Add the MI355 TurboQuant t3nc mirror (#58432) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-24 02:16:09 -05:00
Tianmu LiOpenAI CodexLi, Jiang <jiang1.li@intel.com>
 		67ddac3ea8 [CPU][GDN] Support NIXL DS convolution-state layout (#53300) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-24 14:55:20 +08:00
Andreas KaratzasandCodex 153f2ba1b9 [Compile][CI] Honor Triton cache overrides and add AMD timeout headroom (#58474) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-24 01:27:07 -05:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e028554a1f [Rust Frontend] Support --sse-keep-alive-interval (#58306) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 06:19:14 +00:00
Biswa PandaandCodex cf78797cb7 [Rust Frontend] Pass vision preprocessing context for Nemotron-H (#57634) 
Pass the remaining engine context-length budget to model-owned vision processors through VisionPreprocessingContext. Preserve Nemotron batched engine fields and recognize llm_config as a text_config alias.

Use the merged upstream llm-multimodal revision f0985ef65967615db2c79279aa07818499301bfd.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 14:08:26 +08:00
lucamotzandClaude Opus 5.5 ee2f3cdf12 [Bugfix][Spec Decode] Separate DSpark width from MTP stage validation (#54631) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 06:08:22 +00:00
Wei ZhaoCodexYongye ZhuKevin H. Luu
 		44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-09-23 22:45:23 -07:00
Bugen ZhaoandClaude Opus 5.5 a1b6763df6 [Rust Frontend] Accept custom chat roles for HF templates (#58311) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 13:43:34 +08:00
Bugen ZhaoandClaude Opus 5.5 a5603f21ae [Rust Frontend] Recognize new frontend-owned serve args as unsupported or no-op (#58330) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 13:43:24 +08:00
Farzad AbdolhosseiniClaudeLi, Jiang <jiang1.li@intel.com>
 		8b06a343ec [Bugfix][CPU][MoE] Fix out-of-bounds write and segfault when router weights are fp32 (#56168) 
Signed-off-by: Farzad Abdolhosseini <farzad@elastix.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-24 13:15:17 +08:00
aoshen02 7f1a5398e9 Fix full logprobs in token-in/token-out responses (#58488) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 04:46:16 +00:00
Kapil AryaandOr Ozeri 70fc359d25 [Bugfix][KV Offload] Retain offload event metadata through batch translation (#57453) 
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapil.arya.17@gmail.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-09-24 07:09:12 +03:00
Jhao-Ting Chen 4fb767ff0a [Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection (#53175) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
 2026-09-23 20:29:06 -07:00
stefankoncarevicAndreas KaratzasCodex
 		e78e367c6e [ROCm] Give turboquant boundary layers a layout-compatible backend (#54988) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-23 22:00:07 -05:00
Lucas WilkinsonQiuChunshuoClaude Fable 5.1CodexRobert ShawNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e6dc16cebd [PCP] Support prefill context parallelism with data parallelism (#57075) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-23 22:32:36 -04:00
Robert Shaw 6697a7dd30 [Scheduler] Tune --long-prefill-token-threshold adaptiveness (#58459) 
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-09-23 22:23:18 -04:00
stefankoncarevicAndreas KaratzasAndreas KaratzasCodex
 		e9f167129c [CI][ROCM] Add the Fusion E2E TP2 Quick group on MI355, and the AITER MLA fix it needs (#58369) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-23 21:01:58 -05:00
Bob OkandWentao Ye bcdacfc1ff [Test][Determinism] Cover chunked prefill in the batch-invariance suite (#55612) 
Signed-off-by: Bob Ok <49168652+blipbyte@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-23 21:21:56 -04:00
Juntian LiuandClaude Opus 5.5 9ef37771be [Bugfix][DSA] Bound DeepSelect sentinel columns in the sparse top-k remap (#58215) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 00:54:27 +00:00
Aarushi JainandAndreas Karatzas e0f03eb4f4 [CI] Share BF16 baselines across quantization comparison tests (#58469) 
Signed-off-by: Aarushi Jain <Aarushi.Jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-23 19:26:09 -05:00
Kevin H. LuuandClaude Opus 5.5 7ef0d69ea0 [5/12][ci-selector][CI] Skip the Proton GPU test when another CUPTI tool is injected (#58455) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 17:25:41 -07:00
Nick Hill 310f15d354 [Bugfix][MRV2] Align dummy idx_mapping dtype to avoid runtime jit (#58462) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-23 16:54:48 -07:00
Misha Goin 89d9fd83e2 [CI] Disable JIT warmup by default in VllmRunner (#58452) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-23 18:53:58 -05:00
Kevin H. LuuandClaude Opus 5.5 21ee743f47 [CI][Bugfix] Limit MRV2 sampler JIT warmup registration to ROCm (#58465) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 16:11:50 -07:00
djramicandAndreas Karatzas 5fcc6e7c73 [ROCm][Bugfix] Fix TileLang mHC fused RMSNorm on 64-wide wavefronts (#58419) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-23 23:03:45 +00:00
Andreas Karatzas 8b660ce96b [ROCm][Test] Cover MoRI graph replay and output lifetime (#58093) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-23 14:16:51 -07:00
陈志谦andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 97b1b12117 [Docs] Fix docstring typos (output_dytpe, kwrags, Abbrivations) (#55936) 
Signed-off-by: simpleqt <89645338+simpleqt@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-23 16:36:57 -04:00
DivyandClaude Opus 5 e26547adfb [Bugfix] Pick a KV block size supported by every attention backend (#49845) 
Signed-off-by: Divy <divy@coralbricks.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-23 16:36:35 -04:00
Thang NguyenandClaude Opus 5.5 34b069080d [CI] Select one GPU for the H200 initialized snapshot E2E step (#58351) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-23 13:08:51 -07:00
Lio Einaudi 711fc55c10 [Perf] Use breakable CUDA graphs (no torch.compile) by default under VLLM_BATCH_INVARIANT so the tuned matmul configs see the runtime M (#57586) 
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
 2026-09-23 14:39:38 -04:00
CBPandMisha Goin f9ad9dd6b4 Doc: add DiffusionGemma to supported models (#46466) 
Signed-off-by: Bruce <Bruce798858117@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-23 17:14:30 +00:00
Simon Danielsson b5ea0c7107 [Bugfix][ROCm] Fix startup OOM in AITER MLA FP8 prefill workspace sizing (#57923) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2026-09-23 11:44:12 -05:00
Dao007foreverandCodex c4f6ce4874 [Bugfix][NIXL] Restore successful push completion reporting (#58188) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-23 09:39:04 -07:00