iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,950 Commits 3 Branches 0 Tags
29468dde8b515031dc6d4d9d06bf0a2fa0442098
Commit Graph
21950 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
lzOpenAI CodexLucas WilkinsonRobert Shawmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		29468dde8b [Bugfix][KV Cache][MLA] Align packed block strides for V3.2 sparse MLA (#55528) 
Signed-off-by: lz <145014769+200lz@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 05:33:36 +00:00
shallow10 3a58aeb1a1 [Bugfix] Stop leaking the internal field name in the max_tokens validation error (#58336) 
Signed-off-by: shallow10 <495593563@qq.com>
 2026-09-24 21:55:16 -07:00
Robert ShawjrybergRobert ShawClaude Opus 5.5
 		5a2ed2e7ee [Frontend] Handle Disable Thinking in /v1/messages (#58613) 
Signed-off-by: jryberg <johan.ryberg@security.ntt>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: jryberg <johan.ryberg@security.ntt>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-25 12:52:08 +08:00
Lingpeng JinandAndreas Karatzas 35f92d2abc [ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-launch split-K (#58659) 
Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 21:37:41 -07:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1f0bc49ee6 [Structured Outputs] Parse Lark grammars natively in the xgrammar backend (#58321) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-25 04:35:10 +00:00
wang.yuqiIsotr0pyIsotr0py
 		8a269919f4 [MM] Add Triton kernel for mm_input_normal. (#56798) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 04:31:20 +00:00
Mathew Oddenopencode+deepseek-v4-flash+vllmAndreas KaratzasOpenAI Codex
 		267eee54bf [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497) 
Signed-off-by: Mathew Odden <modden@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-24 23:28:20 -05:00
Andreas KaratzasandCodex fa94450b74 [CI][ROCm] Prevent Model Executor apt stalls (#58607) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-24 23:18:45 -05:00
mkunreddMahesh KunreddiClaudeAndreas KaratzasCodexAndreas Karatzas
 		4013f3ad6c [ROCm] Fix CI runtime and tests for MI355 DPX (#58244) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 23:15:11 -05:00
wangxiyuan 44257c2e35 [Feature] Triton kernel dispatcher (#43048) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-09-25 04:07:43 +00:00
YiSheng5andKunshang Ji b05f7d5fbc [PP][XPU]Add the flag to control microbatch feature on MRV2+PP (#55145) 
Signed-off-by: yisheng <yi.sheng@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-25 12:05:35 +08:00
Yongye ZhuandSummer Yang 442bd00d9f [Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-09-25 03:27:21 +00:00
Reidandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8b365ff949 [ROCm][Build] Filter crate tags from vLLM version detection (#57744) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-09-25 03:06:05 +00:00
fululi12Claude Opus 5Nick Hill
 		4d79d5c638 [ROCm] Cut 69 wasted contiguous copies per decode step from the skinny GEMM path (#58566) 
Signed-off-by: lifulu <fululi12@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-24 22:00:58 -05:00
Samuel Kimandmgoin f55419c43c [Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-25 02:54:11 +00:00
Juntian LiuKimi CodeClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6936e77ba4 [Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:52:02 +08:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		48d8880d09 [Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool (#58489) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:51:50 +08:00
Fangzhou AiandClaude Opus 5.5 3059155b47 [ROCm][Perf] MXFP8 GEMM on native 32x32 block scales for gfx950 (#58510) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 21:26:30 -05:00
Bugen Zhaojthomson04Claude Opus 5.5
 		055f150df5 [Perf][Rust Frontend] Make histogram observations lock-free (#58574) 
Co-authored-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-25 10:25:10 +08:00
gongwei-130andClaude Fable 5.1 34e104f6fe [gRPC] Fix ping tolerance so long non-streaming RPCs are not dropped (#55102) 
Signed-off-by: Wei Gong <wei@together.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-25 02:14:32 +00:00
Divakar Verma 65c5e24689 [ROCm][CI][AITER Coverage] Harden MoE sorting-backend/dispatch env-var test matrix (#58393) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-09-24 21:14:02 -05:00
김재억 32fc82343f [Bugfix][LogitsProcessor] Validate ':' separator in custom logits processor FQCN (#56020) 
Signed-off-by: 100milliongold <gadian88@gmail.com>
 2026-09-25 01:49:20 +00:00
Juntian Liu 609731e8f5 [Perf][DSv4.1] Restore the fused query RMSNorm + MXFP8 quantization path (#57679) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
 2026-09-25 01:39:03 +00:00
Matej Sirovatkaaoshen02Claude Opus 5aoshen02
 		3652f35e7c [Bugfix][Quantization] Refresh online NVFP4 scales before reload post-processing (#57954) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 18:36:08 -07:00
Thien TranKimiIsotr0py
 		16070ed7e9 [Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope (#58526) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-24 18:30:32 -07:00
Thien Tran ade1056ee5 [Kimi-K3][Perf] Dispatch GEMM for vision patch embedder (#58527) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-24 18:30:11 -07:00
Nick HillandClaude Opus 5.5 7dbd0a8d22 [Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models (#58434) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 18:29:53 -07:00
Kevin H. LuuClaude Opus 5.5Misha Goin
 		155b988373 [CI] Run DFlash2 NVFP4 acceptance test on B200; skip it on H200 35GB MIG (#58496) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-24 19:29:44 -06:00
Samuel KimandClaude 34ee85d5ca [Perf] Batch Mamba2 prefill SSM state saves, removing GPU<->CPU syncs (#49371) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-25 01:11:19 +00:00
SIDDARTHA REDDY a811738a60 [Bugfix] Accept EOS after grammar finish in outlines backend; reject json_object at validation (#57743) 
Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com>
 2026-09-24 17:17:10 -07:00
Yifan QiaoandClaude Fable 5.1 39724758e7 [Bugfix] GLM-5.3-Flash: launch the kpool paged MQA logits in the varlen mode its schedule was built with (#55270) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-25 00:13:55 +00:00
Samyabrata MajiandFlora Feng 064bc747ab [Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers (#58626) 
Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 00:12:29 +00:00
Yifan QiaoOpenAI CodexKimi Code
 		b3a87b2d04 [Bugfix][KVConnector] Finalize saves on steps without a forward (#57775) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-09-25 00:06:36 +00:00
Khushali Desai 7833ff25dd Release prompt_embeds tensor when its InputBatch slot is freed (#57988) 
Signed-off-by: khushali9 <khushali.desai9@gmail.com>
 2026-09-25 00:00:44 +00:00
Zijing LiuYifan QiaoCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7a9406690e [KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode (#58472) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 23:55:47 +00:00
Misha GoinandCodex 71891bdb94 [Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-24 19:52:00 -04:00
Netanel HaberClaude Opus 5.5Benjamin Chislett
 		d5051abaf1 [Bugfix][Mamba] Restore prompt-tail prefix-cache hits with MTP (#58368) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-09-24 23:46:42 +00:00
Misha Goin 4f72bd0152 [Bugfix][Quantization] Give LM heads standard linear metadata (#58444) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-24 16:39:49 -07:00
stefankoncarevicandAndreas Karatzas 9ac3b7c69f [ROCm][CI] Mirror the DSv4-Flash disaggregated DP EP group on MI355 (#58558) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 18:38:25 -05:00
Sense_wanghaosenwang1018Claude Opus 5
 		f6aa2919bc [Bugfix][Core] Keep every multimodal feature in the partial-block KV event (#58288) 
Signed-off-by: haosenwang1018 <haosenwang1018@users.noreply.github.com>
Co-authored-by: haosenwang1018 <haosenwang1018@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-24 23:25:01 +00:00
Miłosz Grunwald bd4cb3fe4e [BUGFIX] fix ovis2_5 multimodal tokens (#52623) 
Signed-off-by: Milosz Grunwald <milosz.grunwald@intel.com>
 2026-09-25 07:18:54 +08:00
snadampalandNicolò Lucchesi 6a86bf626e [PD][PushConnector] Record last activity of remotes on the D side (#52245) 
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-09-24 23:15:32 +00:00
Andrey Talman f5b40fcdfe [CI] Report to CRCR after all jobs finish, gated on the build's long pole (#58628) 2026-09-24 16:02:30 -07:00
Fanli LinandKunshang Ji 7edb27f68e [XPU][CI] enable prompt embeds tests on XPU (#58283) 
Signed-off-by: Lin, Fanli <fanli.lin@intel.com>
Signed-off-by: Fanli Lin <fanli.lin@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-25 06:56:59 +08:00
Jellow e6c07ea576 [Bugfix][KV Cache] Fix incremental multimodal block hashing (#51694) 
Signed-off-by: Jellow <49915976+CZT0@users.noreply.github.com>
Signed-off-by: Jellow <dvdx@foxmail.com>
 2026-09-24 15:38:03 -07:00
Nick Hill 36f94d5fe8 [Bugfix][Outlines] Fix EOS termination and unconstrained masks after rejected drafts (#58612) 2026-09-24 21:55:56 +00:00
Juntian LiuandClaude Opus 5.5 04730e8270 [DFlash] Capture the context K/V precompute in the draft CUDA graph (#57632) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 21:38:04 +00:00
stefankoncarevicandAndreas Karatzas e33de821c0 [ROCm][CI] skip the ROCm MRV1 default where MRV1 cannot serve the config (#58535) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 15:36:36 -05:00
Aaron KangandMisha Goin 0908116dd9 [Bugfix] Pass quant_config to DiffusionGemma's ParallelLMHead (#48521) 
Signed-off-by: Aaron Kang <aaron.h.kang@icloud.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-24 13:17:02 -07:00
Matt MastracciClaude Fable 5.1Lucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Misha Goin
 		ab3de6edf2 [Perf] DiffusionGemma: constrained reads over the request's logprob_token_ids (#58216) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-24 13:15:08 -07:00