iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,963 Commits 3 Branches 0 Tags
38cc054fc2269941cd35ecdfd198aacbb2eabfb8
Commit Graph
8396 Commits
Author SHA1 Message Date
MonishverClaude Fable 5.1Misha Goin
 		38cc054fc2 [Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 quantization (#58194) 
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-25 09:19:48 -06:00
chuan932andMisha Goin f550ad8bf7 [Kernel][Perf] Register-resident path for per-token-group 8-bit quant (#55330) 
Signed-off-by: chao.huan <chao.huan@nio.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-25 08:11:42 -07:00
Taneem IbrahimandWentao Ye bb35e2587b [Pooling] Preserve reranker tokenization with document limits (#57666) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-25 11:04:41 -04:00
Nick Hill 234df712ed [MRV2] Minor model_runner.py code cleanup (#58610) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-25 07:59:27 -07:00
Andrii SkliarAndrii SkliarCodexBenjamin Chislett
 		73a78e6f1f [SpecDecode] Add LiLiCorr drafter (#57934) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-09-25 10:08:28 -04:00
djramicandAndreas Karatzas 25b0add7b8 [ROCm][CI] Pass weight_shape in MXFP8 block32 linear tests (#58698) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 10:52:15 +00:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Yongye ZhuandClaude Opus 5.5 378504a544 [MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-25 15:38:29 +08:00
Isotr0py 24928a4d62 [UX][Frontend] Introduce vllm preload cli for fast restart (#56680) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 06:56:39 +00:00
Yongye ZhuandClaude Opus 5.5 5ff3bbfb08 [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 23:26:50 -07:00
lzOpenAI CodexLucas WilkinsonRobert Shawmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		29468dde8b [Bugfix][KV Cache][MLA] Align packed block strides for V3.2 sparse MLA (#55528) 
Signed-off-by: lz <145014769+200lz@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 05:33:36 +00:00
Robert ShawjrybergRobert ShawClaude Opus 5.5
 		5a2ed2e7ee [Frontend] Handle Disable Thinking in /v1/messages (#58613) 
Signed-off-by: jryberg <johan.ryberg@security.ntt>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: jryberg <johan.ryberg@security.ntt>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-25 12:52:08 +08:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1f0bc49ee6 [Structured Outputs] Parse Lark grammars natively in the xgrammar backend (#58321) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-25 04:35:10 +00:00
wang.yuqiIsotr0pyIsotr0py
 		8a269919f4 [MM] Add Triton kernel for mm_input_normal. (#56798) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 04:31:20 +00:00
Mathew Oddenopencode+deepseek-v4-flash+vllmAndreas KaratzasOpenAI Codex
 		267eee54bf [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497) 
Signed-off-by: Mathew Odden <modden@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-24 23:28:20 -05:00
mkunreddMahesh KunreddiClaudeAndreas KaratzasCodexAndreas Karatzas
 		4013f3ad6c [ROCm] Fix CI runtime and tests for MI355 DPX (#58244) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 23:15:11 -05:00
wangxiyuan 44257c2e35 [Feature] Triton kernel dispatcher (#43048) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-09-25 04:07:43 +00:00
Yongye ZhuandSummer Yang 442bd00d9f [Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-09-25 03:27:21 +00:00
Samuel Kimandmgoin f55419c43c [Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-25 02:54:11 +00:00
Juntian LiuKimi CodeClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6936e77ba4 [Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:52:02 +08:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		48d8880d09 [Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool (#58489) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:51:50 +08:00
Fangzhou AiandClaude Opus 5.5 3059155b47 [ROCm][Perf] MXFP8 GEMM on native 32x32 block scales for gfx950 (#58510) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 21:26:30 -05:00
Divakar Verma 65c5e24689 [ROCm][CI][AITER Coverage] Harden MoE sorting-backend/dispatch env-var test matrix (#58393) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-09-24 21:14:02 -05:00
김재억 32fc82343f [Bugfix][LogitsProcessor] Validate ':' separator in custom logits processor FQCN (#56020) 
Signed-off-by: 100milliongold <gadian88@gmail.com>
 2026-09-25 01:49:20 +00:00
Matej Sirovatkaaoshen02Claude Opus 5aoshen02
 		3652f35e7c [Bugfix][Quantization] Refresh online NVFP4 scales before reload post-processing (#57954) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 18:36:08 -07:00
Thien TranKimiIsotr0py
 		16070ed7e9 [Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope (#58526) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-24 18:30:32 -07:00
Thien Tran ade1056ee5 [Kimi-K3][Perf] Dispatch GEMM for vision patch embedder (#58527) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-24 18:30:11 -07:00
Nick HillandClaude Opus 5.5 7dbd0a8d22 [Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models (#58434) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 18:29:53 -07:00
Samuel KimandClaude 34ee85d5ca [Perf] Batch Mamba2 prefill SSM state saves, removing GPU<->CPU syncs (#49371) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-25 01:11:19 +00:00
Samyabrata MajiandFlora Feng 064bc747ab [Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers (#58626) 
Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 00:12:29 +00:00
Yifan QiaoOpenAI CodexKimi Code
 		b3a87b2d04 [Bugfix][KVConnector] Finalize saves on steps without a forward (#57775) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-09-25 00:06:36 +00:00
Khushali Desai 7833ff25dd Release prompt_embeds tensor when its InputBatch slot is freed (#57988) 
Signed-off-by: khushali9 <khushali.desai9@gmail.com>
 2026-09-25 00:00:44 +00:00
Zijing LiuYifan QiaoCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7a9406690e [KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode (#58472) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 23:55:47 +00:00
Misha GoinandCodex 71891bdb94 [Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-24 19:52:00 -04:00
Netanel HaberClaude Opus 5.5Benjamin Chislett
 		d5051abaf1 [Bugfix][Mamba] Restore prompt-tail prefix-cache hits with MTP (#58368) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-09-24 23:46:42 +00:00
Misha Goin 4f72bd0152 [Bugfix][Quantization] Give LM heads standard linear metadata (#58444) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-24 16:39:49 -07:00
stefankoncarevicandAndreas Karatzas 9ac3b7c69f [ROCm][CI] Mirror the DSv4-Flash disaggregated DP EP group on MI355 (#58558) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 18:38:25 -05:00
Sense_wanghaosenwang1018Claude Opus 5
 		f6aa2919bc [Bugfix][Core] Keep every multimodal feature in the partial-block KV event (#58288) 
Signed-off-by: haosenwang1018 <haosenwang1018@users.noreply.github.com>
Co-authored-by: haosenwang1018 <haosenwang1018@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-24 23:25:01 +00:00
snadampalandNicolò Lucchesi 6a86bf626e [PD][PushConnector] Record last activity of remotes on the D side (#52245) 
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-09-24 23:15:32 +00:00
Fanli LinandKunshang Ji 7edb27f68e [XPU][CI] enable prompt embeds tests on XPU (#58283) 
Signed-off-by: Lin, Fanli <fanli.lin@intel.com>
Signed-off-by: Fanli Lin <fanli.lin@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-25 06:56:59 +08:00
Jellow e6c07ea576 [Bugfix][KV Cache] Fix incremental multimodal block hashing (#51694) 
Signed-off-by: Jellow <49915976+CZT0@users.noreply.github.com>
Signed-off-by: Jellow <dvdx@foxmail.com>
 2026-09-24 15:38:03 -07:00
Nick Hill 36f94d5fe8 [Bugfix][Outlines] Fix EOS termination and unconstrained masks after rejected drafts (#58612) 2026-09-24 21:55:56 +00:00
stefankoncarevicandAndreas Karatzas e33de821c0 [ROCm][CI] skip the ROCm MRV1 default where MRV1 cannot serve the config (#58535) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 15:36:36 -05:00
Matt MastracciClaude Fable 5.1Lucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Misha Goin
 		ab3de6edf2 [Perf] DiffusionGemma: constrained reads over the request's logprob_token_ids (#58216) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-24 13:15:08 -07:00
Vincent CavéShiksha PatelDouglas LehrAndreas Karatzas
 		92e9c6bf13 [ROCm] Fix misrouting race-condition in multi-decode P/D disagg with mori-io (#51681) 
Signed-off-by: Vincent Cave <vincent.cave@amd.com>
Signed-off-by: Shiksha Patel <shikpate@amd.com>
Co-authored-by: Shiksha Patel <shikpate@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 19:52:43 +00:00
fxmarty-amdmgoinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		90a9515006 [Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 12:48:14 -07:00
Misha Goin 58aa2f1ef1 [Refactor] Move auxiliary files out of the repository root (#58572) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-24 12:38:56 -07:00
Xun Sun e773247f8a [Fast Start] Wait for weight cache daemon readiness (#58370) 2026-09-24 19:18:04 +00:00
Nick HillandKimi Code e30559b58f [Core] Skip JIT monitor when JIT warmup is disabled (#58590) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-24 12:08:16 -07:00
Nick HillandKimi 074d57fce1 [Bugfix] Keep JIT warmup under enforce-eager when fault tolerance is on (#58593) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
 2026-09-24 12:07:32 -07:00