iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,896 Commits 3 Branches 0 Tags
e773247f8aee272d0262928117e3049cd2902b5e
Commit Graph
21896 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xun Sun e773247f8a [Fast Start] Wait for weight cache daemon readiness (#58370) 2026-09-24 19:18:04 +00:00
Nick HillandKimi Code e30559b58f [Core] Skip JIT monitor when JIT warmup is disabled (#58590) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-24 12:08:16 -07:00
Nick HillandKimi 074d57fce1 [Bugfix] Keep JIT warmup under enforce-eager when fault tolerance is on (#58593) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
 2026-09-24 12:07:32 -07:00
nightcitybladenightcitybladeCyrus Leung
 		5b3f280a6d [Bugfix] Count unsplit Idefics3 image patches (#48760) 
Signed-off-by: nightcityblade <nightcityblade@gmail.com>
Co-authored-by: nightcityblade <nightcityblade@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-24 11:36:26 -07:00
Thomas OrtnerandHarry Mellor 8b84e15066 [transformer] RMSNorm matching for alternative rsqrt (#54461) 
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-24 16:33:06 +00:00
Nick Hill 09fe178dba Revert "[DSpark] Support pipeline-parallel targets in aggregated serving (#56956)" (#58484) 2026-09-24 08:48:36 -07:00
Harry MellorandClaude Opus 5.5 d636456b1f [Frontend] Remove the slow tokenizer mode (#58545) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 08:48:28 -07:00
LucainClaude Opus 5Harry Mellormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		344fcc252a [Bugfix] Resolve the Hub revision once per repo (#56092) 
Signed-off-by: Wauplin <lucainp@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 08:48:24 -07:00
hclClaude Opus 4.8Harry Mellor
 		a7d203efe5 fix(config): apply presence_penalty/frequency_penalty from override-generation-config (#50769) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: hclsys <chenglunhu@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-24 08:48:18 -07:00
Yueh-Ting (eop) Chen 26f49e336a [Perf][Attention] Bound FlashInfer prefill dequantization scratch (#57918) 
Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com>
 2026-09-24 10:42:08 -04:00
Wentao Ye 3c42385d7d [Refactor] Remove dead or duplicate tests (#58446) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-24 10:29:56 -04:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
Linze Shi 484c211fe8 [Bugfix][Pooling] Fix JinaVL label configuration and restore multimodal tests (#57347) 
Signed-off-by: Linze-Shi <linzeshi0@gmail.com>
 2026-09-24 13:51:25 +00:00
Harry MellorandClaude Opus 5.5 cccf7e1376 Remove .gemini/ and CLAUDE.md (#58541) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 12:57:23 +00:00
Fangzhou AiandClaude Opus 5.5 721d0e5c11 [ROCm][DSv4.1][Perf] Emit MXFP8 from the sparse decode reduce and run wo_a as a grouped FP8 GEMM (#58456) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 11:58:14 +00:00
liuzhenwei dcfc17e0b1 [XPU] enable XPU GRAPH by default (#51600) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-09-24 19:17:25 +08:00
Zijing Liu f5a78f2ad7 [Multimodal] Reuse the supplied tokenizer in the MiniMax-M3 VL processor (#58460) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
 2026-09-24 10:47:50 +00:00
SHREY GAJJAR 5747d4500a [Perf][Frontend] Offload streaming derender detokenization (#57528) 
Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com>
 2026-09-24 10:39:21 +00:00
Ganesh RCursorLi, Jiang <jiang1.li@intel.com>
 		f34a0e0797 [CPU] Gate the AVX10.2 paths on compiler support (#58133) 
Signed-off-by: R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-24 18:07:53 +08:00
Ziyang Ma cfd5c20286 [XPU][UT] Align HF and vLLM inputs for Qwen2 embedding test by preventing Sentence Transformers from applying chat template (#58117) 
Signed-off-by: RyanMa29 <ziyang.ma@intel.com>
 2026-09-24 10:07:37 +00:00
ReidBugen ZhaoClaude Opus 5.5
 		e430f7421f [Bugfix][Rust Frontend] Prevent MM timing from enabling debug tracing (#58378) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 09:22:23 +00:00
Taneem Ibrahim bbd7c24d6e [MRV2] Validate MRV2 entrypoint logits processors (#57728) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-24 09:07:26 +00:00
Priyansh JainCursorAndreas Karatzas
 		5963795ec7 [Attention][CPU] Run Zen CPU encoder attention on zentorch SDPA (#54508) 
Signed-off-by: priyansh jain <priyansh.jain2@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 02:05:44 -07:00
Yejing LaiandKunshang Ji 14f98cbe5d [Bugfix][XPU] store the pointer raw bit pattern instead of its numeric value (#54514) 
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-24 02:05:26 -07:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		c0c35ddd8e [Bugfix][CI] Fix the flaky sharded-sampling tests, and the engine teardown need (#58342) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 02:05:11 -07:00
Andreas KaratzasandOpenAI Codex 23110a0c9e [Bugfix] Capture prefill kernels for mixed FULL graphs (#58275) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-24 02:05:07 -07:00
jackLeiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> bbc4ddeca9 [Bugfix][V1] Reject encoder-cache hits with mismatched embedding counts (#57696) 
Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 08:40:47 +00:00
Li, Jiang b44895cf93 [CPU] Use pre-built triton (#58140) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-09-24 16:12:34 +08:00
Talor Abramovich 28282ffe34 [Feature][Frontend] Request JSON body debug logging on --enable-log-requests flag (#58163) 
Signed-off-by: talora <talora@nvidia.com>
 2026-09-24 08:08:40 +00:00
Thien TranandOpenAI Codex 00b7847c80 [Perf] Use Conv3dLayer for MiniMax M3 patch embedding (#58512) 
Signed-off-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-24 00:41:36 -07:00
GuorongYegrYe99Cursor Agent
 		1291bdb85e [EPD][Model Loader] Skip language-model checkpoint shards for --mm-encoder-only (#58086) 
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-09-24 00:41:15 -07:00
Andreas KaratzasandOpenAI Codex c3f5270270 [ROCm][CI] Add the MI355 TurboQuant t3nc mirror (#58432) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-24 02:16:09 -05:00
Tianmu LiOpenAI CodexLi, Jiang <jiang1.li@intel.com>
 		67ddac3ea8 [CPU][GDN] Support NIXL DS convolution-state layout (#53300) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-24 14:55:20 +08:00
Andreas KaratzasandCodex 153f2ba1b9 [Compile][CI] Honor Triton cache overrides and add AMD timeout headroom (#58474) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-24 01:27:07 -05:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e028554a1f [Rust Frontend] Support --sse-keep-alive-interval (#58306) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 06:19:14 +00:00
Biswa PandaandCodex cf78797cb7 [Rust Frontend] Pass vision preprocessing context for Nemotron-H (#57634) 
Pass the remaining engine context-length budget to model-owned vision processors through VisionPreprocessingContext. Preserve Nemotron batched engine fields and recognize llm_config as a text_config alias.

Use the merged upstream llm-multimodal revision f0985ef65967615db2c79279aa07818499301bfd.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 14:08:26 +08:00
lucamotzandClaude Opus 5.5 ee2f3cdf12 [Bugfix][Spec Decode] Separate DSpark width from MTP stage validation (#54631) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 06:08:22 +00:00
Wei ZhaoCodexYongye ZhuKevin H. Luu
 		44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-09-23 22:45:23 -07:00
Bugen ZhaoandClaude Opus 5.5 a1b6763df6 [Rust Frontend] Accept custom chat roles for HF templates (#58311) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 13:43:34 +08:00
Bugen ZhaoandClaude Opus 5.5 a5603f21ae [Rust Frontend] Recognize new frontend-owned serve args as unsupported or no-op (#58330) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 13:43:24 +08:00
Farzad AbdolhosseiniClaudeLi, Jiang <jiang1.li@intel.com>
 		8b06a343ec [Bugfix][CPU][MoE] Fix out-of-bounds write and segfault when router weights are fp32 (#56168) 
Signed-off-by: Farzad Abdolhosseini <farzad@elastix.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-24 13:15:17 +08:00
aoshen02 7f1a5398e9 Fix full logprobs in token-in/token-out responses (#58488) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 04:46:16 +00:00
Kapil AryaandOr Ozeri 70fc359d25 [Bugfix][KV Offload] Retain offload event metadata through batch translation (#57453) 
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapil.arya.17@gmail.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-09-24 07:09:12 +03:00
Jhao-Ting Chen 4fb767ff0a [Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection (#53175) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
 2026-09-23 20:29:06 -07:00
stefankoncarevicAndreas KaratzasCodex
 		e78e367c6e [ROCm] Give turboquant boundary layers a layout-compatible backend (#54988) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-23 22:00:07 -05:00
Lucas WilkinsonQiuChunshuoClaude Fable 5.1CodexRobert ShawNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e6dc16cebd [PCP] Support prefill context parallelism with data parallelism (#57075) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-23 22:32:36 -04:00
Robert Shaw 6697a7dd30 [Scheduler] Tune --long-prefill-token-threshold adaptiveness (#58459) 
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-09-23 22:23:18 -04:00
stefankoncarevicAndreas KaratzasAndreas KaratzasCodex
 		e9f167129c [CI][ROCM] Add the Fusion E2E TP2 Quick group on MI355, and the AITER MLA fix it needs (#58369) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-23 21:01:58 -05:00
Bob OkandWentao Ye bcdacfc1ff [Test][Determinism] Cover chunked prefill in the batch-invariance suite (#55612) 
Signed-off-by: Bob Ok <49168652+blipbyte@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-23 21:21:56 -04:00
Juntian LiuandClaude Opus 5.5 9ef37771be [Bugfix][DSA] Bound DeepSelect sentinel columns in the sparse top-k remap (#58215) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 00:54:27 +00:00