iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,018 Commits 3 Branches 0 Tags
03cae7837fec21c776a0331ffce40d7f202364d2
Commit Graph
22018 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jürgen SchmiedandClaude 03cae7837f [Spec Decode][Qwen4Exp] RecoverSSM for Qwen GDN and the PLE short conv 
With --use-replayssm and speculative decoding, Qwen4Exp's Gated DeltaNet layers verify the
draft window from a read-only checkpoint and keep a per-token replay record
[correction(V), normalized key(K), gate(1)] in fp32. After sampling, one commit replays the
accepted tokens and writes the recurrent state once (plus the block-boundary state in align
mode), instead of snapshotting the state after every draft position. The PLE dilated short
conv follows the same protocol (accepted-suffix compaction). num_speculative_blocks = 0,
so no per-draft Mamba blocks are reserved.

- GDN verify/commit kernels: model_executor/layers/mamba/gdn/recoverssm_gdn.py
- metadata/backends: v1/attention/backends/gdn_recoverssm.py, ple_recoverssm.py
- the verify reads strided q/k/v views of the conv output and writes in place
- builders declare AttentionCGSupport.NEVER: FULL_AND_PIECEWISE falls back to PIECEWISE;
  the fallback now also counts breakable CUDA graphs (empty splitting_ops, with or
  without torch.compile) as piecewise
  attention instead of dropping to NONE
- the generic Mamba metadata builder skips the Mamba2 ReplaySSM ring-buffer workspaces
  when --use-replayssm selects RecoverSSM (the PLE short conv uses that builder)
- tests: GDN verify/commit vs the native kernels (fp32 and bf16 state, align boundaries,
  mixed and ragged batches, extreme gates), PLE vs token-by-token decode, config selection

Signed-off-by: Jürgen Schmied <juergenschmied70@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-27 01:17:24 +02:00
Jürgen SchmiedandClaude a80e41d690 [Mamba] Share RecoverSSM commit kernels and helpers across models 
RecoverSSM's commit-plan and conv-window compaction kernels are model independent.
Move them from models/kimi_k3 into model_executor/layers/mamba/ops/recoverssm.py
(Kimi-K3 keeps its private names as import aliases; kernel source unchanged), add
mamba/recoverssm_utils.py (activation predicate, argument validation) and the
generic spec-row/request mapping helpers to v1/attention/backends/recoverssm_metadata.py.

CacheConfig.use_kda_recoverssm becomes use_recoverssm: the switch now selects RecoverSSM
for any model whose recurrent layers implement it, not only KDA.

No functional change for Kimi-K3.

Signed-off-by: Jürgen Schmied <juergenschmied70@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-27 00:07:04 +02:00
Yongye ZhuandClaude Opus 5.5 77871126f9 [Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary (#58586) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 03:08:36 +08:00
Cyrus Leung 379e9a1ea8 [Security] Harden message sanitization (#58832) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-26 16:28:32 +00:00
Kevin H. Luu 7d8c5fe9a9 [Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata (#58499) 2026-09-26 11:37:36 -04:00
Andreas Karatzas 0239bd2549 [CI] Stabilize batch submission in full CUDA graph tests (#58810) 2026-09-26 11:35:56 -04:00
Wentao Ye 3576691c42 [GLM5.3 Bug] Fix sparse indexer attn topk backend selection (#58594) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-26 11:35:28 -04:00
Misha Goin 79ac28c5bd [CI] Reduce CUDA graph mode test overhead (#58749) 2026-09-26 11:22:33 -04:00
Ashraf Bhuiyan 5113cf9d16 [Mypy] Fix mypy typing for Qwen and Qianfan models (#58046) 
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com>
 2026-09-26 14:31:51 +00:00
Robert ShawandRobert Shaw dfab504333 [Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-26 14:00:56 +00:00
Robert ShawandRobert Shaw d64f277c54 [Bugfix] Fix Anthropic Thinking Disabled with P/D (#58786) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-26 13:50:57 +00:00
Willian Zandmgoin ad6817b68d [Perf][MoE] Index expert mapping lookups in RoutedExperts.load_weights (#58720) 
Signed-off-by: Willian <willian@willian.email>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-26 06:48:24 -07:00
Wentao Ye bf93e1f92c [Refactor] Remove dead tests utils (#58803) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-26 09:16:53 -04:00
Itay AlroyandTyler Michael Smith 2bb7605ae1 [Elastic EP] Fix EPLB load statistics during scaling (#58473) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-09-26 12:05:37 +00:00
Juan Pérez de AlgabaandCursor 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-26 11:10:35 +00:00
Thang NguyenClaude Opus 5.5Kevin H. Luu
 		2f41e00235 [CI] Split (B200) Miscellaneous Kernels into mHC, FLA Ops and Misc named jobs (#58609) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-09-26 04:10:14 -07:00
qizixiOpenAI CodexClaude Opus 5.5Nick Hill
 		a4eb3f25d6 [Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-26 17:01:17 +08:00
Aarushi JainandAndreas Karatzas 3b4566c5cf [CI] Only isolate the registry tests that need a fresh process (#58764) 
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-26 14:03:45 +08:00
zhecClaudeBugen Zhao
 		ddd6fbca14 [Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: zhec <chengyunfei@ruc.edu.cn>
 2026-09-26 02:18:58 +00:00
xiao-llmandCursor 5840d95284 [Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071) 
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-25 20:51:18 -05:00
Ke Wenmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>aoshen02
 		b6d2b4fc25 [RL] Add sharding-aware NCCL M2N weight transfer (#51520) 
Signed-off-by: Ke Wen <kwen@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-26 01:38:53 +00:00
Shuolei Wang 4bb804cc5b [Perf][DSv4.1] Shard the Engram wkv projection across TP ranks (#58678) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
 2026-09-26 01:15:57 +00:00
Chandra MouryaandVS Chandra Mourya 8a2364605c [Core] Bound draft-token RPC waits by the execute-model timeout (#58779) 
Signed-off-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com>
Co-authored-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com>
 2026-09-26 00:40:22 +00:00
Baljinder HothiandFlora Feng fdcce47e9b [Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning (#58792) 
Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-26 00:38:52 +00:00
Thang NguyenClaude Opus 5.5Andreas KaratzasCodex
 		da1aaec31f [CI] Split (H200 MIG/MI300) Basic Correctness into named jobs (#57054) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-25 17:05:33 -07:00
Elvir CrnčevićandFlora Feng 9bf44c4013 [Bugfix][Reasoning] Count Kimi K3 reasoning tokens (#58372) 
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 23:29:56 +00:00
Rui ZhuandClaude 974cb65155 [Bugfix] V1: fix allowed_token_ids_mask aliasing in InputBatch.swap_states (#48419) 
Signed-off-by: Rui Zhu <rui.zhu.rz399@yale.edu>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-25 23:23:11 +00:00
Ricardo-M-LandClaude Opus 4.6 b040321b33 fix: perf: use startswith(x, i) instead of string slicing to avoid O(N^2) (#52580) 
Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-09-25 23:22:13 +00:00
Taneem IbrahimandWentao Ye 6ec2f84e1a [Pooling] Preserve BERT-family heads for raw logits (#57664) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-25 19:10:14 -04:00
Linkun 614b55f0b6 [AuxOutput] Only require Model Runner V2 on GPU platform (#58205) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2026-09-25 15:55:21 -07:00
Oxana Korzh e55d076f89 [Bugfix] Don't drop the rest of the allocator config when toggling expandable segments (#57982) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
 2026-09-25 17:34:19 -05:00
Raphaël RiallandSimon VeitnerOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		4ccfe12398 [watermarking] golden tests for backwards compatibility (#56809) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 22:01:48 +00:00
yzong-rhandBen Browning b6761e8ded [Bugfix] Default missing detail for Responses API input images (#57241) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-09-25 14:08:55 -07:00
Cheng RuiCodexFlora Feng
 		8b47e8b22c [Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams (#56067) 
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 14:08:23 -07:00
Nick HillandKimi Code de2d478d9a [Bugfix][Frontend] Document 404 response for /generative_scoring (#58788) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-25 14:08:00 -07:00
Denis ErmakovDu BinClaude Opus 5.5Flora Fengmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7871963fcc [Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop (#58551) 
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 14:00:39 -07:00
Yousafandsfeng33 83db8c8390 [Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957) 
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-09-25 20:43:08 +00:00
Rohan PotdarClaude Fable 5jpvillam
 		e440b75bb6 [ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision (#50605) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: jpvillam <juan.villamizar@amd.com>
 2026-09-25 14:55:52 -05:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		31cc226401 [ROCm][CI] Run the MLA attention+quant fusion test on ROCm (#58717) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 14:45:32 -05:00
stefankoncarevicandAndreas Karatzas 2d7dd0e5c1 [Bugfix][CI] Report subprocess test skips as skips, not passes (#58701) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 14:44:49 -05:00
Taneem Ibrahim 2bc902eb0f [CI] [MRV2] Restore MRV2 pp dp coverage (#57735) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-25 19:13:59 +00:00
Sheral KumarAndreas KaratzasCodexAndreas KaratzasOpenAI CodexAndreas Karatzas
 		22bbe3f102 [ROCm][CI] Expand single-GPU coverage on MI355 DPX (#57599) 
Signed-off-by: Sheral Kumar <shekumar@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
 2026-09-25 13:27:53 -05:00
Matt MastracciandClaude Fable 5.1 353e491efd [Perf] DiffusionGemma: one-pass sampler statistics kernel (#58226) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-25 14:14:55 -04:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		e3dd5b5a75 [ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch reference (#58740) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 18:05:54 +00:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		1b922ffcbe [ROCm][CI] Add quantized MoE serving test for gfx950 (#58748) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 18:02:15 +00:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		b21757555a [ROCm][CI] Cover the AITER MQA logits dispatch on gfx950 (#58724) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 17:43:40 +00:00
jiacao-amdOpenAI CodexAndreas Karatzas
 		1417022c3d [ROCm][Kimi-K3] Optimize low-concurrency speculative KDA (#58045) 
Signed-off-by: jiacao-amd <jiahui.cao@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 12:33:30 -05:00
GokayAIandGokayAI c89b5b4176 [Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head (#58292) 
Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com>
Co-authored-by: GokayAI <gokay-ai@users.noreply.github.com>
 2026-09-25 17:23:44 +00:00
Wentao Ye c88f4824ff [GLM5.3 Perf] Optimize glm 5.3 metadata op, 1.6~4.8x kernel level performance improvement (#58450) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-25 13:15:18 -04:00
Matt MastracciYifan QiaoClaude Opus 5.5Robert Shawmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2617fe9383 [Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding (#58454) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 12:51:34 -04:00