|
35dc273072
|
[ROCm][Spec Decode] Add Aiter MLA decode support non-causal draft block (#55966)
Signed-off-by: Poovaiah Palangappa <poovaiah.palangappa@amd.com>
Co-authored-by: Poovaiah Palangappa <poovaiah.palangappa@amd.com>
Co-authored-by: John Qin <zhengsheng1997@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
|
2026-09-15 12:18:09 -05:00 |
|
|
|
241e9391ed
|
[Bugfix][Attention] Stabilize sparse-MLA DCP for GLM PCP evals (#55879)
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Jason Yao <wsyjh8@gmail.com>
|
2026-09-15 09:58:30 -07:00 |
|
|
Chauncey
|
836bb3839f
|
[Bugfix] Prevent out-of-bounds access in FlashInfer SM90 sparse MLA mixed batches (#56969)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-09-15 15:03:58 +00:00 |
|
|
Chauncey
|
f0fbc98c48
|
[Frontend] Share max_num_queued_reqs across API server processes (#54746)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-09-15 14:50:26 +00:00 |
|
|
zack
|
0136df94b0
|
[Bugfix][Spec Decode][MoE] Avoid uninitialized EPLB state in DeepSeek V4.1 DSpark drafter (#56387)
Signed-off-by: zack <51604064+luoyuctl@users.noreply.github.com>
|
2026-09-15 14:35:47 +00:00 |
|
|
yzong-rh
|
e6960af33b
|
[Refactor] StructuredOutputManager x Speculative Decoding Refactor (#48200)
|
2026-09-15 09:48:49 -04:00 |
|
|
Kevin H. Luu
|
2983fc6286
|
[CI] Fix NIXL push worker test stub after failure deferral (#56955)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
|
2026-09-15 02:00:58 -07:00 |
|
|
|
676650397f
|
[Feature][PCP] Support decode-only FULL CUDA graphs (#53867)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: QiuChunshuo <chunshuoq@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-15 16:27:46 +08:00 |
|
|
|
000c7df9ff
|
[KV Offload] Add KVCR secondary-tier adapter (#53624)
Signed-off-by: Moein Khazraee <moein@nvidia.com>
Signed-off-by: Moein <moein@nvidia.com>
Co-authored-by: Olga Andreeva <124622579+oandreeva-nv@users.noreply.github.com>
Co-authored-by: Olga Andreeva <oandreeva@nvidia.com>
|
2026-09-15 09:55:04 +03:00 |
|
|
Jared WenandKimi Code
|
51918e252a
|
[Bugfix][SparseMLA] Fix piecewise cudagraph capture crash in index group (#56825)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
|
2026-09-15 06:51:35 +00:00 |
|
|
|
f547c23ec9
|
[Bugfix][Kimi-K3] Keep transient checkpoints out of prefix-cache eviction (#56794)
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
|
2026-09-15 14:18:24 +08:00 |
|
|
|
cd10ed6f9f
|
[feature] [xgrammar] support patternProperties/propertyNames/unevaluatedProperties kw for object types (#42904)
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
2026-09-15 12:58:52 +08:00 |
|
|
Nick Hill
|
7dbc3d62c3
|
[Cleanup] Remove vestigial tpu_input_batch.py (#56898)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-14 21:29:15 -07:00 |
|
|
Nicolò LucchesiandNick Hill
|
45779ef99f
|
[Bugfix][NIXL] Fix multiple handles xfer race (#56104)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-15 04:27:10 +00:00 |
|
|
Yizheng JiaoandNick Hill
|
381c61e008
|
[Bugfix][NIXL] Report a full prefix cache hit as finished (#56640)
Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-14 20:15:19 -07:00 |
|
|
Liqian Lin
|
e45bb43984
|
[Bugfix][KV Offload] Ignore pending chunks in invalid sliding windows (#56486)
Signed-off-by: Liqian Lin <jacklin78911@gmail.com>
|
2026-09-15 05:48:38 +03:00 |
|
|
liranschour
|
17bc3e106a
|
[KVConnector][P2P] Configurable unbound-store timeout and one-RTT rejection of a late fetch (#53453)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
|
2026-09-15 05:38:18 +03:00 |
|
|
|
84e5f3edac
|
[Bugfix] Validate routed-expert prompt offsets before engine submission (#56844)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bhushan Asati <bhushanasati25@gmail.com>
Co-authored-by: Megha Agarwal <19240983+meghaagr13@users.noreply.github.com>
|
2026-09-15 09:17:41 +08:00 |
|
|
Giancarlo Delfin
|
3bb0a03f35
|
[Model Runner V2] Acceptance estimation for adaptive verification (#52228)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-09-14 15:58:48 -07:00 |
|
|
鐘天楽andNick Hill
|
0a5747d410
|
[Profiler] Add Proton CUDA graph attribution for MRV2 (#51084)
Signed-off-by: Luossu <zhluosuu@outlook.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-14 19:02:07 +00:00 |
|
|
Kevin H. LuuandOpenAI Codex
|
56ca9904ce
|
[CI] Initialize ubatch runner in cudagraph unit test (#56808)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-14 13:37:08 -05:00 |
|
|
Thillai ChithambaramandMark McLoughlin
|
435c96f9db
|
[V1][Metrics] Support Sliding Window Attention (SWA) and hybrid layers in MFU/MBU estimation (#55624)
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
|
2026-09-14 16:04:55 +00:00 |
|
|
|
ff5f6d41b1
|
[Bugfix] Scale KV page size for hidden states extraction with TP (#50894)
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Orestis Zambounis <orestis-z@users.noreply.github.com>
Signed-off-by: Orestis Zambounis <23146389+orestis-z@users.noreply.github.com>
Signed-off-by: Speculators Bot <bot@speculators.ai>
Signed-off-by: Speculators Bot <speculatorsbot@gmail.com>
Co-authored-by: Orestis Zambounis <orestis-z@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Speculators Bot <speculatorsbot@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-14 10:07:09 -04:00 |
|
|
Jiangyun ZhuandKimi Code
|
47fbd36e6d
|
[Bugfix] Trim stale consequence claims from unannotated-eagle warning (#56791)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
|
2026-09-14 21:06:26 +08:00 |
|
|
Simon Danielsson
|
1d0d1081c4
|
[ROCm][Connector] SWA+HMA-support in MoRI-IO connector (Gemma4) (#53721)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
|
2026-09-14 20:52:56 +08:00 |
|
|
|
c612e2bff0
|
[Bugfix][KV Offload] Reuse in-flight async lookup probes (#55823)
Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-14 15:02:45 +03:00 |
|
|
|
a5f6f61a8a
|
[CI][CPU] Add speculative-decoding coverage to CPU CI (#54934)
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
|
2026-09-14 19:51:38 +08:00 |
|
|
Xiaozhen BiandClaude Opus 5
|
e82794f48f
|
[CI][Test] Mock CPU backend block sizes in kv_connector unit conftest (#56671)
Signed-off-by: Xiaozhen Bi <xiaozhen.bi@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-14 18:48:53 +08:00 |
|
|
Nicolò Lucchesi
|
0ca7aef4e3
|
[Bugfix][NixlPush] Guard _remote_agents read in _do_send_reg_notif (X1) (#56317)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-14 18:37:12 +08:00 |
|
|
jiangkuaixue123
|
1678b39627
|
[Bugfix][EPD] Preserve media processing options in encoder requests (#56786)
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
|
2026-09-14 10:05:13 +00:00 |
|
|
|
238cb2b191
|
[Perf][GLM-5.3-Flash] Dense/masked-MHA sparse prefill for the NoPE (256, 0, 256) layout + skip the NoPE K concat (#55738)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-14 09:36:39 +00:00 |
|
|
Bugen Zhao
|
cf1584f373
|
[Refactor] Normalize DeepSeek V4.1 model package naming (#56741)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-13 23:17:50 -07:00 |
|
|
Anjie HouandCodex
|
73d2a8cf86
|
[2/2][Model Runner V2] FULL CUDA graph capture for microbatched steps (DBO) (#51700)
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-13 22:42:33 -07:00 |
|
|
|
58d45fd767
|
[Bugfix][KV Connector] MooncakeStore: exclude non-prefix-cacheable (QSA ring) groups; fix align-mode check (#55027)
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-13 19:18:16 -07:00 |
|
|
Tianyu Guo
|
319cc5ef19
|
[Performance][EPD] Reduce Python proxy serialization overhead (#56657)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
|
2026-09-13 15:07:12 -07:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
586f652f8d
|
[Bugfix][PCP][DCP] Respect interleave in indexer KV gather mapping (#56715)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-13 16:45:17 -04:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
7fe8fc803d
|
[NIXL][PCP][DCP] Expose PCP producer KV shards as transfer ranks (#56645)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-13 20:23:15 +00:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
b6e2aa748b
|
[CI] Initialize warmup registry in V2 QSA runner fixture (#56707)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-13 13:34:09 -04:00 |
|
|
|
82a85dc1d2
|
[Attention] Add Triton/FlashInfer composite for multimodal prefix attention (#56305)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-13 14:29:41 +00:00 |
|
|
Shenglei FuandClaude Sonnet 5
|
8cf9de9080
|
[Test][Determinism] Cover VLM batch invariance in default execution mode (#56528)
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
2026-09-13 10:22:03 -04:00 |
|
|
Kebe
|
c351fd3c64
|
[Core] Fix ValueError on KV load failure with a hybrid KV cache (#50388)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2026-09-13 21:03:05 +08:00 |
|
|
Lucas Wilkinson
|
1cfd972816
|
[CI] Fix NIXL transfer-rank geometry fixture (#56672)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-09-13 07:36:06 +00:00 |
|
|
Lucas Wilkinson
|
410f6da5c4
|
[Bugfix][KV Offload] Submit CPU stores on no-forward steps (#56621)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-09-13 05:56:14 +00:00 |
|
|
Woosuk KwonandOpenAI Codex
|
2671fedfc7
|
[MRV2] Revert explicit Triton JIT warmup migration (#56654)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-13 04:07:56 +00:00 |
|
|
|
aed894c190
|
[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-12 17:24:24 -07:00 |
|
|
|
e19a3e172e
|
[5/N] Share HiSparse host cache across TP ranks (reopens #52760) (#56629)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Chao Lei <leichao139636@163.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Chao Lei <leichao139636@163.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-12 19:48:25 -04:00 |
|
|
|
ebe1dec2da
|
[PCP][DCP] Enable PCP+DCP on sparse-MLA models (#56157)
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patryk Saffer <PatrykSaffer@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-12 23:24:44 +00:00 |
|
|
|
7ee8a6dd01
|
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
|
2026-09-12 18:42:01 +00:00 |
|
|
|
29332cf936
|
[4/N] Expose HiSparse cache metrics via KV connector stats (#56061)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-12 10:47:55 -07:00 |
|
|
Woosuk Kwon
|
13e221f830
|
[Perf] Fuse DSV4.1 input metadata preparation with Triton (#56562)
|
2026-09-12 07:20:04 -07:00 |
|