4e59d53ab5
[MRV2] Support stock torch.compile mode ( #59079 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-28 20:17:48 -07:00
Mehmet Cagri
70ae0b7435
[ROCm][MLA] Enable sparse MLA Gluon kernel from Aiter ( #53492 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
Signed-off-by: Mehmet Cagri <mehmet.kaymak@amd.com >
2026-09-29 09:08:18 +08:00
Almog Tavor
171d1b9ef6
[Elastic EP] Support Model Runner V2 ( #53934 )
...
Signed-off-by: almogtavor <almogtavor@gmail.com >
Signed-off-by: Almog Tavor <70065337+almogtavor@users.noreply.github.com >
2026-09-29 00:20:40 +00:00
Benedikt Falk
c68eb98b69
[Bugfix][Frontend] Keep in-flight requests on the same DP engine ( #59017 )
...
Signed-off-by: Benedikt Falk <bfalk@nvidia.com >
2026-09-28 23:52:29 +00:00
Fanli Lin and Kunshang Ji
6a6c2952fa
[XPU][CI] Make test_mamba_prefix_cache block-size agnostic ( #58269 )
...
Signed-off-by: Lin, Fanli <fanli.lin@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-29 07:12:43 +08:00
0xsensei and OpenAI Codex
d28795f1a7
[Bugfix][Scheduler] Refresh max tokens for streaming continuations ( #57676 )
...
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 20:31:57 +00:00
fedbc3b564
[Bugfix][MRV2][Spec Decode] Reject draft slots that were never proposed ( #58784 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-28 13:07:15 -07:00
Nick Hill and Robert Shaw
42dbe4578c
[Core] Rework scheduler skipped_waiting queue ( #58947 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-09-28 20:03:16 +00:00
0xsensei
2840ca7a33
[Bugfix][Scheduler] Preserve logprobs across streaming continuations ( #57447 )
...
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com >
2026-09-28 19:50:27 +00:00
e7c903609f
[Perf][MRV2] Allow FULL decode graphs for one-token prompt tails ( #58400 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-09-28 12:48:52 -07:00
yzong-rh
2f4a975901
[Bugfix] Fix resumable request + async scheduling handoff race ( #58259 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-09-28 15:26:57 -04:00
25624f65cf
[ROCm][MLA] Add an AITER ASM round-robin decode route for DCP multi-token verify ( #56861 )
...
Signed-off-by: Xiaohu Guo <Xiaohu.Guo@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-28 14:52:21 -04:00
qizixi
6d3ea3c2c9
[KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill ( #50499 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
2026-09-28 18:50:46 +02:00
b721a4c709
[Feature] Add fixed-token prefill scoring ( #54335 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-28 07:50:09 -07:00
BaoYunkai
f6877f563c
[Bugfix][Spec Decode] Implement get_top_tokens() on the ROCm DeepSeek V4 MTP drafter ( #57568 )
...
Signed-off-by: BaoYunkai <ybao@amd.com >
2026-09-28 20:16:38 +08:00
004e37ece2
[Misc] Name each backend and its kernel block sizes in block-size errors ( #58557 )
...
Signed-off-by: Eugenio "Jay" Zuccarelli <11176606+jayzuccarelli@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 14:33:18 +08:00
dd3bc9c72f
[Perf][MRV2] Reuse Mamba/GDN metadata across KV cache groups ( #58762 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Xin Yang <xyangx@amazon.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-28 04:36:58 +00:00
Tianyu Guo
53d2e16a97
[Bugfix][EPD] Skip sampling for encoder-only async steps ( #58490 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-09-27 20:35:09 -07:00
Chao Lei
386ac2573e
[Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout ( reopens #55763 ) ( #58919 )
2026-09-27 19:47:36 +00:00
0376f81530
[Perf][Pooling] Avoid blocking seq_lens GPU-to-CPU copy for pooling in FlashInfer metadata builder ( #57214 )
...
Signed-off-by: frankwang28 <frank.wbb@hotmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-09-27 16:06:38 +00:00
924707f1bf
[CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named jobs ( #57237 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Thang Nguyen <thang.nguyen@inferact.ai >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 05:01:31 -07:00
Nicolò Lucchesi
c8d7a7dd13
[Perf][Attention] Remove D2H sync from FlashInfer SM90 sparse MLA plan under async scheduling ( #58684 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-09-27 13:06:23 +02:00
24c9772d19
[Attention][CPU] Use zentorch SDPA for CPU MLA prefill ( #54967 )
...
Signed-off-by: Rakul Chauhan <rakul.chauhan@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-27 08:30:55 +00:00
Varshith and Claude
fba47397f3
[Bugfix] V1: clear stale allowed_token_ids mask in InputBatch.condense ( #43931 )
...
Signed-off-by: Varshith <kvarshithgowda@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-27 03:44:54 +00:00
YannikHinteregger and xijiade.aihemaiti
fff03267da
[Bugfix][NIXL] Release a dead peer's NIXL state without waiting for TTL ( #50047 )
...
Signed-off-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com >
Co-authored-by: xijiade.aihemaiti <3146335281@qq.com >
2026-09-26 20:30:09 -07:00
eb0f2ca37f
[mooncake] support CUSTOM_MEM_POOL in vllm ( #49300 )
...
Signed-off-by: bruce.xu <bruce.xb@alibaba-inc.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: bruce.xu <bruce.xb@alibaba-inc.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-27 00:26:05 +00:00
mevince
cd1daf58a9
[Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector ( #57251 )
...
Signed-off-by: Vincent <vincexxchan@gmail.com >
2026-09-26 17:17:11 -07:00
927c87b347
[PCP][DCP] Support DCP target model with non-DCP Dspark ( #56723 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-26 18:12:35 -04:00
Misha Goin
79ac28c5bd
[CI] Reduce CUDA graph mode test overhead ( #58749 )
2026-09-26 11:22:33 -04:00
Wentao Ye
bf93e1f92c
[Refactor] Remove dead tests utils ( #58803 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-26 09:16:53 -04:00
a4eb3f25d6
[Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer ( #57263 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-26 17:01:17 +08:00
Chandra Mourya and VS Chandra Mourya
8a2364605c
[Core] Bound draft-token RPC waits by the execute-model timeout ( #58779 )
...
Signed-off-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com >
Co-authored-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com >
2026-09-26 00:40:22 +00:00
da1aaec31f
[CI] Split (H200 MIG/MI300) Basic Correctness into named jobs ( #57054 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-25 17:05:33 -07:00
Rui Zhu and Claude
974cb65155
[Bugfix] V1: fix allowed_token_ids_mask aliasing in InputBatch.swap_states ( #48419 )
...
Signed-off-by: Rui Zhu <rui.zhu.rz399@yale.edu >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-25 23:23:11 +00:00
Matt Mastracci and Claude Fable 5.1
353e491efd
[Perf] DiffusionGemma: one-pass sampler statistics kernel ( #58226 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-25 14:14:55 -04:00
GokayAI and GokayAI
c89b5b4176
[Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head ( #58292 )
...
Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com >
Co-authored-by: GokayAI <gokay-ai@users.noreply.github.com >
2026-09-25 17:23:44 +00:00
Wentao Ye
c88f4824ff
[GLM5.3 Perf] Optimize glm 5.3 metadata op, 1.6~4.8x kernel level performance improvement ( #58450 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-25 13:15:18 -04:00
2617fe9383
[Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding ( #58454 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 12:51:34 -04:00
2edbb32e58
[Bugfix][GLM-5.3-Flash] SM90 sparse MLA: index_kpool mismatch leads to corruption via unread query token ( #58704 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-09-25 12:36:36 -04:00
6491f481a7
[Bugfix] Stop allocator fragmentation from shrinking the KV cache during memory profiling ( #58430 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Signed-off-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com >
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 12:16:27 -04:00
Nick Hill
234df712ed
[MRV2] Minor model_runner.py code cleanup ( #58610 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-25 07:59:27 -07:00
73a78e6f1f
[SpecDecode] Add LiLiCorr drafter ( #57934 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Andrii Skliar <andreyws96@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-09-25 10:08:28 -04:00
Mark McLoughlin and AI Assistant
45d56c0baf
[Core] Model console logging as CLI configuration ( #57205 )
...
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: AI Assistant <noreply@openai.com >
2026-09-25 11:36:59 +01:00
29468dde8b
[Bugfix][KV Cache][MLA] Align packed block strides for V3.2 sparse MLA ( #55528 )
...
Signed-off-by: lz <145014769+200lz@users.noreply.github.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 05:33:36 +00:00
1f0bc49ee6
[Structured Outputs] Parse Lark grammars natively in the xgrammar backend ( #58321 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-25 04:35:10 +00:00
4013f3ad6c
[ROCm] Fix CI runtime and tests for MI355 DPX ( #58244 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-24 23:15:11 -05:00
김재억
32fc82343f
[Bugfix][LogitsProcessor] Validate ':' separator in custom logits processor FQCN ( #56020 )
...
Signed-off-by: 100milliongold <gadian88@gmail.com >
2026-09-25 01:49:20 +00:00
Nick Hill and Claude Opus 5.5
7dbd0a8d22
[Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models ( #58434 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 18:29:53 -07:00
Samuel Kim and Claude
34ee85d5ca
[Perf] Batch Mamba2 prefill SSM state saves, removing GPU<->CPU syncs ( #49371 )
...
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-25 01:11:19 +00:00
b3a87b2d04
[Bugfix][KVConnector] Finalize saves on steps without a forward ( #57775 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
2026-09-25 00:06:36 +00:00
