iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,990 Commits 3 Branches 0 Tags
6ec2f84e1a42cdca59779ffae20aa62c8beda993
Commit Graph
21990 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Taneem IbrahimandWentao Ye 6ec2f84e1a [Pooling] Preserve BERT-family heads for raw logits (#57664) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-25 19:10:14 -04:00
Linkun 614b55f0b6 [AuxOutput] Only require Model Runner V2 on GPU platform (#58205) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2026-09-25 15:55:21 -07:00
Oxana Korzh e55d076f89 [Bugfix] Don't drop the rest of the allocator config when toggling expandable segments (#57982) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
 2026-09-25 17:34:19 -05:00
Raphaël RiallandSimon VeitnerOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		4ccfe12398 [watermarking] golden tests for backwards compatibility (#56809) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 22:01:48 +00:00
yzong-rhandBen Browning b6761e8ded [Bugfix] Default missing detail for Responses API input images (#57241) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-09-25 14:08:55 -07:00
Cheng RuiCodexFlora Feng
 		8b47e8b22c [Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams (#56067) 
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 14:08:23 -07:00
Nick HillandKimi Code de2d478d9a [Bugfix][Frontend] Document 404 response for /generative_scoring (#58788) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-25 14:08:00 -07:00
Denis ErmakovDu BinClaude Opus 5.5Flora Fengmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7871963fcc [Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop (#58551) 
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 14:00:39 -07:00
Yousafandsfeng33 83db8c8390 [Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957) 
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-09-25 20:43:08 +00:00
Rohan PotdarClaude Fable 5jpvillam
 		e440b75bb6 [ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision (#50605) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: jpvillam <juan.villamizar@amd.com>
 2026-09-25 14:55:52 -05:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		31cc226401 [ROCm][CI] Run the MLA attention+quant fusion test on ROCm (#58717) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 14:45:32 -05:00
stefankoncarevicandAndreas Karatzas 2d7dd0e5c1 [Bugfix][CI] Report subprocess test skips as skips, not passes (#58701) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 14:44:49 -05:00
Taneem Ibrahim 2bc902eb0f [CI] [MRV2] Restore MRV2 pp dp coverage (#57735) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-25 19:13:59 +00:00
Sheral KumarAndreas KaratzasCodexAndreas KaratzasOpenAI CodexAndreas Karatzas
 		22bbe3f102 [ROCm][CI] Expand single-GPU coverage on MI355 DPX (#57599) 
Signed-off-by: Sheral Kumar <shekumar@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
 2026-09-25 13:27:53 -05:00
Matt MastracciandClaude Fable 5.1 353e491efd [Perf] DiffusionGemma: one-pass sampler statistics kernel (#58226) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-25 14:14:55 -04:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		e3dd5b5a75 [ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch reference (#58740) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 18:05:54 +00:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		1b922ffcbe [ROCm][CI] Add quantized MoE serving test for gfx950 (#58748) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 18:02:15 +00:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		b21757555a [ROCm][CI] Cover the AITER MQA logits dispatch on gfx950 (#58724) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 17:43:40 +00:00
jiacao-amdOpenAI CodexAndreas Karatzas
 		1417022c3d [ROCm][Kimi-K3] Optimize low-concurrency speculative KDA (#58045) 
Signed-off-by: jiacao-amd <jiahui.cao@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 12:33:30 -05:00
GokayAIandGokayAI c89b5b4176 [Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head (#58292) 
Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com>
Co-authored-by: GokayAI <gokay-ai@users.noreply.github.com>
 2026-09-25 17:23:44 +00:00
Wentao Ye c88f4824ff [GLM5.3 Perf] Optimize glm 5.3 metadata op, 1.6~4.8x kernel level performance improvement (#58450) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-25 13:15:18 -04:00
Matt MastracciYifan QiaoClaude Opus 5.5Robert Shawmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2617fe9383 [Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding (#58454) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 12:51:34 -04:00
Yashasvi Chaurasia f83ea61cfe [Benchmark] Record model_id in bench latency/throughput --output-json (#58112) 
Signed-off-by: yashasvi <yashasvi@ibm.com>
 2026-09-25 16:43:37 +00:00
Matt MastracciClaude Fable 5.1Robert Shaw
 		2edbb32e58 [Bugfix][GLM-5.3-Flash] SM90 sparse MLA: index_kpool mismatch leads to corruption via unread query token (#58704) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-09-25 12:36:36 -04:00
Denis ErmakovYanxiao ZhaoPrakhar AgarwalFlora Feng
 		f339e0e757 [Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks (#58583) 
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com>
Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 16:33:19 +00:00
Robert ShawRobert ShawRobert ShawAndreas KaratzasAndreas Karatzas
 		6491f481a7 [Bugfix] Stop allocator fragmentation from shrinking the KV cache during memory profiling (#58430) 
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 12:16:27 -04:00
Mark McLoughlin a44d7b5151 [Core][Logging] Fix JSON logging process decoration (#57957) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-09-25 10:12:28 -06:00
MonishverClaude Fable 5.1Misha Goin
 		38cc054fc2 [Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 quantization (#58194) 
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-25 09:19:48 -06:00
chuan932andMisha Goin f550ad8bf7 [Kernel][Perf] Register-resident path for per-token-group 8-bit quant (#55330) 
Signed-off-by: chao.huan <chao.huan@nio.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-25 08:11:42 -07:00
Taneem IbrahimandWentao Ye bb35e2587b [Pooling] Preserve reranker tokenization with document limits (#57666) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-25 11:04:41 -04:00
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 50f78084da [Bugfix] Fix generative scoring body cancellation (#57729) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 11:01:31 -04:00
Nick Hill 234df712ed [MRV2] Minor model_runner.py code cleanup (#58610) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-25 07:59:27 -07:00
Andrii SkliarAndrii SkliarCodexBenjamin Chislett
 		73a78e6f1f [SpecDecode] Add LiLiCorr drafter (#57934) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-09-25 10:08:28 -04:00
djramicandAndreas Karatzas 25b0add7b8 [ROCm][CI] Pass weight_shape in MXFP8 block32 linear tests (#58698) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 10:52:15 +00:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Thang NguyenandClaude Opus 5.5 fa6c407d15 [CI] Shard (H100) Helion Kernels five ways (#58645) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-25 03:25:11 -07:00
Harry MellorandClaude Opus 5.5 afea5c20c7 [Docs] Add return annotation to fused_mm_input_norm_triton (#58687) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-25 10:01:11 +00:00
Yongye ZhuandClaude Opus 5.5 378504a544 [MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-25 15:38:29 +08:00
Isotr0py 24928a4d62 [UX][Frontend] Introduce vllm preload cli for fast restart (#56680) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 06:56:39 +00:00
Yongye ZhuandClaude Opus 5.5 5ff3bbfb08 [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 23:26:50 -07:00
lzOpenAI CodexLucas WilkinsonRobert Shawmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		29468dde8b [Bugfix][KV Cache][MLA] Align packed block strides for V3.2 sparse MLA (#55528) 
Signed-off-by: lz <145014769+200lz@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 05:33:36 +00:00
shallow10 3a58aeb1a1 [Bugfix] Stop leaking the internal field name in the max_tokens validation error (#58336) 
Signed-off-by: shallow10 <495593563@qq.com>
 2026-09-24 21:55:16 -07:00
Robert ShawjrybergRobert ShawClaude Opus 5.5
 		5a2ed2e7ee [Frontend] Handle Disable Thinking in /v1/messages (#58613) 
Signed-off-by: jryberg <johan.ryberg@security.ntt>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: jryberg <johan.ryberg@security.ntt>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-25 12:52:08 +08:00
Lingpeng JinandAndreas Karatzas 35f92d2abc [ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-launch split-K (#58659) 
Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 21:37:41 -07:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1f0bc49ee6 [Structured Outputs] Parse Lark grammars natively in the xgrammar backend (#58321) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-25 04:35:10 +00:00
wang.yuqiIsotr0pyIsotr0py
 		8a269919f4 [MM] Add Triton kernel for mm_input_normal. (#56798) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 04:31:20 +00:00
Mathew Oddenopencode+deepseek-v4-flash+vllmAndreas KaratzasOpenAI Codex
 		267eee54bf [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497) 
Signed-off-by: Mathew Odden <modden@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-24 23:28:20 -05:00
Andreas KaratzasandCodex fa94450b74 [CI][ROCm] Prevent Model Executor apt stalls (#58607) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-24 23:18:45 -05:00
mkunreddMahesh KunreddiClaudeAndreas KaratzasCodexAndreas Karatzas
 		4013f3ad6c [ROCm] Fix CI runtime and tests for MI355 DPX (#58244) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 23:15:11 -05:00
wangxiyuan 44257c2e35 [Feature] Triton kernel dispatcher (#43048) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-09-25 04:07:43 +00:00