|
a4d2d9d95e
|
[ROCm][Kimi-K3] Enable a4w4 flydsl kernels for KimiK3 (#53940)
Signed-off-by: ppalanga <ppalanga@amd.com>
Signed-off-by: Poovaiah Palangappa <poovaiah.palangappa@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Poovaiah Palangappa <poovaiah.palangappa@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
|
2026-09-16 01:38:02 +00:00 |
|
|
Colin ZandAndreas Karatzas
|
c8d1cf077a
|
[ROCm] [Bugfix] Enable Load and Inference of GLM-5.3-Flash Quark MXFP4 Checkpoint (#56176)
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-16 01:25:29 +00:00 |
|
|
Giancarlo Delfin
|
85c1f58d50
|
[BugFix][Model Runner V2][Spec Decode] Fix decode instance's multi-layer MTP kv caches during P/D (#55055)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-09-15 17:31:25 -07:00 |
|
|
|
0fefffc934
|
[KVConnector][NIXL] Support attention-HMA layouts in pipeline-parallel push prefill (#50494)
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-15 15:43:56 -07:00 |
|
|
Misha GoinandOpenAI Codex
|
c6fa1f05d1
|
[Perf][Kernel] Add sampled filtering for persistent top-k (#56346)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-15 14:15:59 -07:00 |
|
|
Jared WenandClaude Fable 5.1
|
1257512305
|
[DSA] Wire DeepGEMM sparse MQA logits into the DeepSeek V4.1 indexer (#56254)
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-15 14:07:14 -07:00 |
|
|
Nick Hill
|
f2aad6aa70
|
[MRV2] Buffer util simplifications (#56888)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-15 20:36:51 +00:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
38ca7a899c
|
[Bugfix][HiSparse] Preserve per-layer offsets in KV cache bindings (#57027)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-15 20:27:11 +00:00 |
|
|
Nicolò Lucchesi
|
df42d112ee
|
[Config] Infer HiSparse attention config from HiSparseConnector (#57041)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-15 19:44:26 +00:00 |
|
|
Fangzhou Ai
|
dffbb714e4
|
[ROCm][Perf] Optimize DSV4.1 K=512 decode top-k on gfx950 (#56743)
Signed-off-by: fai <fangzhouai@gmail.com>
|
2026-09-15 18:34:24 +00:00 |
|
|
Kevin H. Luu
|
2f46a1d9c6
|
[CI] Keep Qwen3 Omni DSpark config fixture complete (#57044)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
|
2026-09-15 18:19:14 +00:00 |
|
|
|
35dc273072
|
[ROCm][Spec Decode] Add Aiter MLA decode support non-causal draft block (#55966)
Signed-off-by: Poovaiah Palangappa <poovaiah.palangappa@amd.com>
Co-authored-by: Poovaiah Palangappa <poovaiah.palangappa@amd.com>
Co-authored-by: John Qin <zhengsheng1997@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
|
2026-09-15 12:18:09 -05:00 |
|
|
|
241e9391ed
|
[Bugfix][Attention] Stabilize sparse-MLA DCP for GLM PCP evals (#55879)
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Jason Yao <wsyjh8@gmail.com>
|
2026-09-15 09:58:30 -07:00 |
|
|
Chauncey
|
836bb3839f
|
[Bugfix] Prevent out-of-bounds access in FlashInfer SM90 sparse MLA mixed batches (#56969)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-09-15 15:03:58 +00:00 |
|
|
Abhinav VermaandClaude Opus 5
|
1e47ec00d2
|
[Bugfix][Gemma 4] Don't read fft_length when profiling unified audio (#56721)
Signed-off-by: Abhinav Verma <verma.abhinav275@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-15 14:57:32 +00:00 |
|
|
Chauncey
|
f0fbc98c48
|
[Frontend] Share max_num_queued_reqs across API server processes (#54746)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-09-15 14:50:26 +00:00 |
|
|
zack
|
0136df94b0
|
[Bugfix][Spec Decode][MoE] Avoid uninitialized EPLB state in DeepSeek V4.1 DSpark drafter (#56387)
Signed-off-by: zack <51604064+luoyuctl@users.noreply.github.com>
|
2026-09-15 14:35:47 +00:00 |
|
|
yzong-rh
|
e6960af33b
|
[Refactor] StructuredOutputManager x Speculative Decoding Refactor (#48200)
|
2026-09-15 09:48:49 -04:00 |
|
|
|
bb55077411
|
[Perf][Kernel] Integrate Mega-mHC from DeepGEMM for DeepSeek V4.1 (reopen of #56255) (#56962)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-09-15 04:01:06 -07:00 |
|
|
ALEX LILUZandJiangyun Zhu
|
5be911ee87
|
[Bugfix] Load reasoning parser plugins before headless engine config (#53124)
Signed-off-by: alexliluz <49665315+alexliluz@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-09-15 18:31:25 +08:00 |
|
|
Flora Feng
|
24bfbd5d4d
|
[Refactor] Derive is_reasoning_end from the engine grammar (#56200)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-09-15 10:07:58 +00:00 |
|
|
|
073f883b56
|
[Perf][DSpark] Collapse DeepSeek-V4.1 draft states before SP all-gather (#56903)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-09-15 02:58:57 -07:00 |
|
|
Kevin H. Luu
|
2983fc6286
|
[CI] Fix NIXL push worker test stub after failure deferral (#56955)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
|
2026-09-15 02:00:58 -07:00 |
|
|
Yixin DongandFlora Feng
|
2c2cbd84fd
|
[Frontend] Use XGrammar schema constraints for DeepSeek V4.1 (#56408)
Signed-off-by: Ubospica <ubospica@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-09-15 04:36:16 -04:00 |
|
|
|
676650397f
|
[Feature][PCP] Support decode-only FULL CUDA graphs (#53867)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: QiuChunshuo <chunshuoq@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-15 16:27:46 +08:00 |
|
|
shaohuaxiandCodex
|
ca67438c08
|
[Bugfix][Frontend] Handle aborted requests in beam search (#56249)
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-15 08:17:30 +00:00 |
|
|
|
000c7df9ff
|
[KV Offload] Add KVCR secondary-tier adapter (#53624)
Signed-off-by: Moein Khazraee <moein@nvidia.com>
Signed-off-by: Moein <moein@nvidia.com>
Co-authored-by: Olga Andreeva <124622579+oandreeva-nv@users.noreply.github.com>
Co-authored-by: Olga Andreeva <oandreeva@nvidia.com>
|
2026-09-15 09:55:04 +03:00 |
|
|
Jared WenandKimi Code
|
51918e252a
|
[Bugfix][SparseMLA] Fix piecewise cudagraph capture crash in index group (#56825)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
|
2026-09-15 06:51:35 +00:00 |
|
|
|
f547c23ec9
|
[Bugfix][Kimi-K3] Keep transient checkpoints out of prefix-cache eviction (#56794)
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
|
2026-09-15 14:18:24 +08:00 |
|
|
Kevin H. Luu
|
ec6b14c044
|
[CI] Select the supported DCP backend for PCP eval (#56677)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
|
2026-09-14 23:05:59 -07:00 |
|
|
|
cd10ed6f9f
|
[feature] [xgrammar] support patternProperties/propertyNames/unevaluatedProperties kw for object types (#42904)
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
2026-09-15 12:58:52 +08:00 |
|
|
aoshen02
|
eb61707851
|
[Bugfix] Fix stale HPC QK-norm weights after weight refit (#56706)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
|
2026-09-14 21:34:41 -07:00 |
|
|
Nick Hill
|
7dbc3d62c3
|
[Cleanup] Remove vestigial tpu_input_batch.py (#56898)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-14 21:29:15 -07:00 |
|
|
Nicolò LucchesiandNick Hill
|
45779ef99f
|
[Bugfix][NIXL] Fix multiple handles xfer race (#56104)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-15 04:27:10 +00:00 |
|
|
|
caf56ce9da
|
[CI] Pass the scale-out endpoint flag in EC E2E (#56819)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-09-15 12:25:30 +08:00 |
|
|
|
9d14fd3095
|
[Kernel][MoE] DeepEP v2: async finalize to overlap shared experts with combine (#52781)
Signed-off-by: Mario <404mario@users.noreply.github.com>
Co-authored-by: Mario <404mario@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-15 00:04:22 -04:00 |
|
|
Yizheng JiaoandNick Hill
|
381c61e008
|
[Bugfix][NIXL] Report a full prefix cache hit as finished (#56640)
Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-14 20:15:19 -07:00 |
|
|
|
e6eb0d120c
|
[Model][DSv4.1] Store the whole KV in MXFP8 (FlashMLA V4.1 record) (#56893)
Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-14 20:12:55 -07:00 |
|
|
wuhangxianandChauncey
|
a529c1a748
|
[Bugfix] Handle bare and malformed tool call openers in Gemma4 parser (#53444)
Signed-off-by: wuhangxian <1391938827@qq.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-09-15 02:52:43 +00:00 |
|
|
Liqian Lin
|
e45bb43984
|
[Bugfix][KV Offload] Ignore pending chunks in invalid sliding windows (#56486)
Signed-off-by: Liqian Lin <jacklin78911@gmail.com>
|
2026-09-15 05:48:38 +03:00 |
|
|
liranschour
|
17bc3e106a
|
[KVConnector][P2P] Configurable unbound-store timeout and one-RTT rejection of a late fetch (#53453)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
|
2026-09-15 05:38:18 +03:00 |
|
|
|
84e5f3edac
|
[Bugfix] Validate routed-expert prompt offsets before engine submission (#56844)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bhushan Asati <bhushanasati25@gmail.com>
Co-authored-by: Megha Agarwal <19240983+meghaagr13@users.noreply.github.com>
|
2026-09-15 09:17:41 +08:00 |
|
|
JohnQinAMDandClaude Opus 5
|
ef5f7cd119
|
[ROCm][DSv4.1][Perf] Dequantize the MXFP8 weight once when dot_scaled cannot be used (#56560)
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 09:07:26 +08:00 |
|
|
Andreas KaratzasandCodex
|
a7576447b8
|
[ROCm][Bugfix] Initialize Ray NIXL agents for sharded RDT (#56687)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-15 00:34:21 +00:00 |
|
|
Giancarlo Delfin
|
3bb0a03f35
|
[Model Runner V2] Acceptance estimation for adaptive verification (#52228)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-09-14 15:58:48 -07:00 |
|
|
Fangzhou AiandCursor
|
00972dfd72
|
[ROCm][DSV4.1][Perf] Fold the mHC post step into the delayed pre projection (#56513)
Signed-off-by: Fangzhou Ai <fangzhou@semianalysis.ai>
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-14 16:44:56 -05:00 |
|
|
Fangzhou AiandCursor
|
dabc4362b4
|
[ROCm][DSV4.1][Perf] Stride the DSA decode candidate mask over the live context (#56628)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-14 20:47:32 +00:00 |
|
|
sychen52
|
7702ee87db
|
[Bugfix][DSA] Write nvfp4_ds_mla from the fused norm+rope kernel (#55538)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
|
2026-09-15 00:25:33 +04:00 |
|
|
Netanel Haber
|
d4ee7fe7a9
|
[Quantization] Select linear backends per quantization (#51204)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
|
2026-09-14 16:24:18 -04:00 |
|
|
Nicolò LucchesiandJorge Gallego Feliciano
|
995e8581f4
|
[Model] Voxtral Realtime: add support for CUDAGraphMode.FULL_DECODE_ONLY (#51167)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Jorge Gallego Feliciano <jorge.gallego@mistral.ai>
|
2026-09-14 19:16:39 +00:00 |
|