|
483e0c24c7
|
[KV Offload] Split replicated-layout pre-faulting across workers (#59545)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-13 19:18:56 +00:00 |
|
|
Matthew BonanniandClaude Opus 5.5
|
624a5ddafd
|
[Attention] Fetch MLA prefill workspace at use instead of init (#60654)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 17:25:03 +00:00 |
|
|
Taksh Kothari
|
12b005eefa
|
[Bugfix] Don't split a multimodal item that a prefill chunk starts exactly on (#52412)
Signed-off-by: Taksh <takshkothari09@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-13 15:32:38 +00:00 |
|
|
|
de8291e24f
|
[Core] Add lifecycle hooks for worker extensions (#59463)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-13 21:51:05 +08:00 |
|
|
|
31e90ac370
|
[Attention] Enable FA4 sparse MQA decode on Blackwell (#55866)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-10-13 09:31:37 -04:00 |
|
|
|
818f70c295
|
[Bugfix] Mask special added tokens in xgrammar JSON, regex and EBNF grammars (#60749)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-10-13 14:38:23 +03:00 |
|
|
samayp01andArtem Perevedentsev
|
54db759eea
|
[Bugfix][Structured Output] Reject structural_tag and unhandled specs in backend validation (#58862)
Signed-off-by: Samay <samay.piduru@gmail.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-10-13 13:27:34 +03:00 |
|
|
|
2cc2e68d1c
|
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-13 00:58:59 -07:00 |
|
|
Yongye ZhuandClaude Opus 5.5
|
a6009ea85b
|
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer (#61149)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-12 23:24:43 -07:00 |
|
|
Nicolò LucchesiandClaude Opus 5.5
|
86b1c2ef15
|
[Attention][DCP] Support a2a DCP comm backend for FlashMLA sparse (#61285)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 13:51:22 +08:00 |
|
|
|
0d2689e0df
|
[ROCm][Perf] Skip redundant sparse index remap on non-indexer layers (#51309)
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
|
2026-10-13 05:25:56 +00:00 |
|
|
|
2ce8012fa0
|
[ROCm][Perf] Skip cleaning sparse prefill MQA logits (#51314)
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-13 00:21:52 -05:00 |
|
|
|
df6c7ad031
|
[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py (#61183)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 03:05:39 +00:00 |
|
|
|
c6566bb9f6
|
[ROCm][MLA] Keep DCP prefill context in FP8 through AllGather (#59070)
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
|
2026-10-13 09:20:38 +08:00 |
|
|
|
98f7211162
|
[Attention][Spec Decode] Multi-query Triton split-K for spec-decode verify, with empty-segment/padding hardening (#56148)
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Signed-off-by: Venky <23023424+venkywonka@users.noreply.github.com>
Co-authored-by: OpenAI GPT-6 Astra <noreply@openai.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-13 04:37:56 +04:00 |
|
|
|
1484266938
|
[Perf] Improve _topk_topp_kernel() for masked logits (#61213)
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-12 16:01:11 -07:00 |
|
|
Matthew BonanniandClaude Opus 5.5
|
fe341a53dd
|
[HiSparse] Report host pool in KV cache capacity logs (#60285)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 20:49:49 +00:00 |
|
|
ztysthandOpenAI Codex
|
1bdd3fa2d5
|
[Bugfix][Core] Check multimodal chunk boundaries before encoder cache reuse (#59721)
Signed-off-by: ztysth <rnk.liulw@gmail.com>
Signed-off-by: ztysth <166926442+ztysth@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-12 19:25:35 +00:00 |
|
|
Wentao Ye
|
d0948f754d
|
[Bug] Honor spec decode draft model config (#61240)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-12 15:13:23 -04:00 |
|
|
Maroon AyoubandClaude
|
d6aca837c3
|
[Attention] Make the sparse MLA prefill workspace size configurable (#61023)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-12 14:56:51 -04:00 |
|
|
|
8055293353
|
[ROCm][MLA] Cut ~21x host dispatch from the MLA metadata build (#58381)
Signed-off-by: lifulu <fululi12@amd.com>
Signed-off-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com>
|
2026-10-12 13:11:20 -05:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
e1a5086256
|
[Attention] Remove cascade attention (#61126)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 10:41:07 -07:00 |
|
|
|
a7b7bc1fbe
|
[Metrics] Count finished requests by watermark status (#58715)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-12 17:21:22 +00:00 |
|
|
Misha GoinandClaude
|
d79b52b182
|
[Misc] Consolidate csrc shared headers and prune dead code (#60721)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-12 09:46:22 -07:00 |
|
|
|
e0317b28ce
|
[Feature] Support batch invariance on ROCm (#52231)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-12 10:53:26 -05:00 |
|
|
TJian
|
50b21a2667
|
[Perf] [Feat] [ROCm] Add densemha support to ROCm AITER Sparse MLA (#49263)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-10-12 15:50:28 +00:00 |
|
|
Ilya Markov
|
a281ad32f0
|
[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) (#57549)
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
|
2026-10-12 10:03:09 -04:00 |
|
|
DimensionSTPandArtem Perevedentsev
|
314ab8cffd
|
[Bugfix][Structured Output] Drain compilation tasks before releasing the backend (#61047)
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-10-12 16:25:36 +03:00 |
|
|
Chauncey
|
ac28877f0e
|
[Bugfix] Fix out-of-bounds lookahead in adaptive block verification (#60945)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-10-12 09:21:35 -04:00 |
|
|
Tianyu Guo
|
258b97ff6b
|
[Bugfix][EC Connector] Size transfers by measured encoder output widths (#57904)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
|
2026-10-12 12:32:39 +00:00 |
|
|
Thuongvu HoandNicolò Lucchesi
|
99b44c7a42
|
[Metrics] Add admission rejection metrics (#55812)
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
|
2026-10-12 08:31:07 +00:00 |
|
|
Giancarlo DelfinandRishi Puri
|
36f3972812
|
[Spec Decode] Skip metadata build for dflash/dspark when all model backends support it (#60692)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com>
|
2026-10-12 00:09:10 -07:00 |
|
|
Yan Ma
|
f147949be8
|
[XPU] limit FA kernel block size to multipleOf 64 (#57297)
Signed-off-by: Yan Ma <yan.ma@intel.com>
|
2026-10-12 10:25:32 +08:00 |
|
|
dingdangmaoandCodex
|
124b89d5b7
|
[Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries (#59962)
Signed-off-by: apex-mochen <2756823972@qq.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-12 02:08:14 +00:00 |
|
|
Nick Hill
|
3fe917abb1
|
[Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 (#61013)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-11 16:04:41 -07:00 |
|
|
|
93d2cab35e
|
[Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions (#51504)
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
|
2026-10-11 11:39:11 -04:00 |
|
|
aoshen02andCodex
|
239a3fa3a8
|
[Sleep] Retain draft-private weights with existing parameter selector (#61046)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-11 23:30:56 +08:00 |
|
|
|
3e8e1364ff
|
[Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP (#54951)
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com>
Signed-off-by: Zheng Cai <caizheng1993@gmail.com>
Signed-off-by: Codex <noreply@openai.com>
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-10-11 10:43:37 -04:00 |
|
|
aoshen02
|
bd5bbce6e4
|
[RL] Release weight-transfer communicators without stopping inference (#61060)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
|
2026-10-11 21:15:14 +08:00 |
|
|
|
2339bd39d6
|
[Bugfix][Mooncake] Use the EAGLE replay boundary when retaining sliding-window tails (#61012)
Signed-off-by: Stu Cao <stu@primeintellect.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-11 09:15:18 +00:00 |
|
|
Xun Sun
|
26a488ad93
|
[Fast Start][Bugfix] Snapshot GPU memory after external weights are ready (#60966)
|
2026-10-11 07:10:04 +00:00 |
|
|
Nick Hill
|
f8471164bf
|
[MRV2] Detatch weight-reload logic from MRV1 (#60937)
|
2026-10-10 21:15:05 -07:00 |
|
|
bzsuni
|
01abe975b5
|
[Bugfix][KV Connector] Fix speculative block reservations across KV cache groups (#59880)
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
|
2026-10-10 20:55:41 -07:00 |
|
|
Shuolei Wangandaoshen02
|
b89852aad0
|
[RL][Frontend] Return token-level weight-version spans on final outputs (#59582)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-11 11:33:29 +08:00 |
|
|
Wentao Ye
|
276fbcff27
|
[Deprecation] Remove scheduled deprecated items (#60923)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-10 15:45:33 -04:00 |
|
|
|
f2376cdc4d
|
[Fast Start] Preload the FlashInfer autotune table on the weight cache daemon (#60085)
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-10 15:48:06 +00:00 |
|
|
Tony Lin
|
2bea22002d
|
[BUG] Fix KV offload shared-memory slot selection using worker rank (#58497)
Signed-off-by: Tony Lin <tony.lin@intel.com>
|
2026-10-10 20:25:11 +08:00 |
|
|
Chauncey
|
fa5cfd48d5
|
[Refactor] Deduplicate trace header handling in BaseServing (#60828)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-10-10 05:27:58 +00:00 |
|
|
Matthew BonanniandClaude Opus 5.5
|
d5f356dae0
|
[HiSparse] Own GPU residency instead of editing core block-table rows (#60647)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 21:25:44 -07:00 |
|
|
SeongJun LeeandMisha Goin
|
9bd074a996
|
[Attention] Decode an NVFP4 KV cache with XQA on SM12x (#60452)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-09 22:24:00 -06:00 |
|