 Connor CarpenterandNick Hill
|
f473870ecf
|
[Rust Frontend][gRPC] Add RL lifecycle control (#51316)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-14 11:32:39 -07:00 |
|
|
James E T SmithandAndreas Karatzas
|
9b0ab5dd53
|
[ROCm][AMD] Enable preshuffled sparse indexing for 16-token blocks (#51216)
Signed-off-by: jamesETsmith <james.smith9113@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-08-14 12:38:14 -05:00 |
|
|
Lucas WilkinsonandClaude
|
57bd0ed441
|
[5/N][KV-Cache Layout Refactor] Backend-published KV packing via customize_spec (#51704)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-14 09:47:55 -04:00 |
|
|
Zhuobin Huangandnodeeeeee
|
63a9a5010a
|
[Attention][DSA] Take the native decode path for MTP=3 on SM90 (#52164)
Signed-off-by: zobinHuang <zobin1999@gmail.com>
Co-authored-by: nodeeeeee <zhangkai.nodeee@gmail.com>
|
2026-08-14 10:09:51 +00:00 |
|
|
Tzu-Ling KanandClaude Opus 4.8
|
653cc6faca
|
[Bugfix][NIXL] Include transfer mode (push/pull) in the compatibility hash (#50620)
Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-14 12:11:22 +08:00 |
|
|
Qiming Zhang
|
d18bb7b390
|
[UT] fix device of test_outputs.py (#52237)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
|
2026-08-14 11:29:54 +08:00 |
|
|
Junhao ShenandClaude Opus 4.8
|
38f097fab8
|
[Bugfix] Reject NUL byte in structured_outputs.regex (#51796)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-08-14 00:08:36 +00:00 |
|
|
Giancarlo DelfinandClaude Fable 5
|
f80b66f548
|
[Model Runner V2][Spec Decode] Add KV cache support for multi-layer MTP (#50062)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-08-13 16:22:29 -07:00 |
|
|
|
b652dedd0c
|
[Attention] Fix FlashInfer SM12x prefill with sinks (#52148)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-08-13 14:25:09 -07:00 |
|
|
Ali JaseemandClaude
|
64ca614fe4
|
[Bugfix] Fix --data-parallel-start-rank 0 being treated as unset in create_engine_config (#47692)
Signed-off-by: Syed Ali Jaseem <syedali0057@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-13 15:12:26 +00:00 |
|
|
|
95c9144424
|
[LoRA][Gemma4] Support vision tower LoRA (#42662)
Signed-off-by: bk-201 <joy25810@foxmail.com>
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Signed-off-by: linitra24 <joy25810@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: linitra24 <joy25810@gmail.com>
|
2026-08-13 22:07:43 +08:00 |
|
|
|
50ba4bc6b2
|
[Feature] Mask Replay (#49577)
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-13 08:44:31 +00:00 |
|
|
|
903d2efe7e
|
[Attention][MLA] Fuse Kimi-K3 chunked-context K/V packing (#51772)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-12 22:42:57 -07:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
79f3183f86
|
[Bugfix] Bound KV block zeroing launch geometry (#52058)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-13 03:42:26 +00:00 |
|
|
Ziqi Fan
|
15227b934a
|
[Bugfix][KV Offload] Emit self-describing CPU events at KV-group block granularity (#51614)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2026-08-12 20:28:15 -07:00 |
|
|
Ziqi Fan
|
f9a0f629bd
|
[KV Offload] Expose data-parallel topology to offloading backends (#51879)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2026-08-12 20:27:26 -07:00 |
|
|
|
5936bac7d7
|
[Bugfix] Report FULL_ATTENTION for uniform-base UniformTypeKVCacheSpecs groups instead of UNKNOWN (#51218)
Signed-off-by: Yifan Jiang <19356972+yifjiang@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-13 01:25:23 +00:00 |
|
|
|
7f7a32cfec
|
[Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-12 12:27:48 -07:00 |
|
|
|
9035151d6c
|
[Model] Add native Dots3 NOTE multimodal support (#51255)
Signed-off-by: KurodaKanbei <mistergalahad@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-08-12 23:13:20 +08:00 |
|
|
Michael Goin
|
8e958902ee
|
[Bugfix] Disable fine-grained prefix-cache hits for incompatible hybrid KV layouts (#51843)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-08-12 04:08:40 -07:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
02ac17851d
|
[Refactor][MRV2] Unify uniform decode token count helper (#51917)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-12 05:58:43 +00:00 |
|
|
Andreas Karatzas
|
23f25aac14
|
[CI] Stabilize tensor IPC multiprocessing tests (#50804)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-08-12 10:16:43 +08:00 |
|
|
Varun Sundar RabindranathandVarun Sundar Rabindranath
|
f067737b2a
|
[Bugfix][TieredOffloading] : Return HIT_PENDING when KV promotion is triggered (#51840)
Signed-off-by: <>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
|
2026-08-12 05:07:16 +03:00 |
|
|
|
0a94d85a66
|
[Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch (#51865)
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
|
2026-08-11 17:15:06 -07:00 |
|
|
|
1d2d83a07f
|
[Attention] Add FlashInfer XQA decode support on SM12x (#49718)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-11 16:32:49 -07:00 |
|
|
鐘天楽
|
a367cdbca0
|
[Profiler] Add minimal Triton Proton profiling backend (#48789)
Signed-off-by: Luossu <zhluosuu@outlook.com>
|
2026-08-11 14:51:57 -07:00 |
|
|
stefankoncarevic
|
3e372c5ff2
|
[Bugfix][ROCm] Give KV-first attention blocks their own page in hybrid models (#51837)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
|
2026-08-11 16:52:30 -04:00 |
|
|
Andreas Karatzas
|
cc668c5b72
|
[CI][Bugfix][V1] Remove stale FlashAttention metadata arguments (#51854)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
|
2026-08-11 18:54:56 +00:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
1ab2801dde
|
[ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-11 14:19:16 -04:00 |
|
|
Michael Goinandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
e3fe212eaf
|
[Bugfix] Generalize KV block zeroing to AttentionSpec (#51749)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-11 12:13:09 -04:00 |
|
|
Nick Hill
|
513f83e7ec
|
[Bugfix] Take the sliding window from the layer, not the KV cache group (#51756)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-11 09:10:46 -07:00 |
|
|
Guan-Ming Chiuandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
457a5f34db
|
[Bugfix][MRV2] Support encoder timing stats in model runner V2 (#50020)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-12 00:00:49 +08:00 |
|
|
liuzhenweiandKunshang Ji
|
d648236199
|
[XPU][CI] Fix ExampleConnector KV cache device selection (#51806)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-08-11 23:40:29 +08:00 |
|
|
Wentao Ye
|
4988df2eb8
|
[Config] Update default _max_num_batched_tokens from 8192 to 16384 (#51726)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-11 11:09:34 -04:00 |
|
|
AlexHuang
|
0fec3d652b
|
[Bugfix][KV Offload] Centralize shared mmap cleanup in CPU worker (#51622)
Signed-off-by: Alex <jihui.huang@daocloud.io>
|
2026-08-11 13:22:15 +03:00 |
|
|
Dao007foreverandOpenAI Codex
|
ce07118669
|
[Bugfix][Core] Preserve Mamba running CoW after external hits (#51766)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-11 18:15:24 +08:00 |
|
|
Yizhou
|
a311916a29
|
[MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph (#46849)
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
|
2026-08-11 15:06:22 +08:00 |
|
|
Wentao Ye
|
0914ed2e81
|
[Perf] Adaptive budget for spec scheduled input tokens, ~60% better Kimi K3 DSpark TTFT (#51725)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-10 20:52:21 -07:00 |
|
|
Itay EtelisandItay Etelis
|
0f2ea973be
|
[KV Connector][Offloading] Keep per-layer KV registration when canonical_layout is requested (#51688)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
|
2026-08-11 06:03:59 +03:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
1a1727330a
|
[CI] Solidify speculative decoding E2E coverage (#50713)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-10 19:56:39 -06:00 |
|
|
Tristan Rice
|
98a4144a41
|
[Bugfix] Preserve non-logitproc entry points in tests (#51097)
Signed-off-by: Tristan Rice <rice@fn.lc>
|
2026-08-10 18:14:53 -04:00 |
|
|
|
63ac04a61e
|
[Kimi-K3] DCP support (#50484)
Signed-off-by: girasoley <girasoley@inferact.ai>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: foraxe <73625538+foraxe@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-08-10 12:54:26 -07:00 |
|
|
Flora Feng
|
05f0a80016
|
[Bugfix][Structured Output] Mask request stop tokens in xgrammar until grammar terminates (#49227)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-08-10 14:29:20 -04:00 |
|
|
fanxingran
|
37fbf52084
|
[ROCm][MLA] [K3] Fix fp8 KV cache decode on the AITER MLA backend (#51011)
Signed-off-by: fanxingran <xingran.fan@amd.com>
|
2026-08-10 11:42:10 -04:00 |
|
|
liuzhenwei
|
bd6536071c
|
[XPU][Test] Pin block size in test_multi_connector (#51213)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
|
2026-08-10 21:57:45 +08:00 |
|
|
|
243c63baf5
|
[V1][Scheduler] Apply Mamba alignment before encoder caps (#51603)
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Akshat Anand <40275336+cipheraxat@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-08-10 16:37:17 +08:00 |
|
|
Itay EtelisandItay Etelis
|
81840a172f
|
[KV Connector] Canonical CPU layout for parallelism-agnostic KV offload (#48414)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
|
2026-08-10 10:44:54 +03:00 |
|
|
|
61c1dd0966
|
[EC Connector] Call to EC Connector update_connector_output from scheduler (#49579)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-10 14:25:57 +08:00 |
|
|
Srinivas Krovvidiandsrinivas_oo7
|
a123159f7a
|
Add tiering offloading metrics (#48798)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-08-10 05:23:14 +03:00 |
|
|
Yifan QiaoandOpenAI Codex
|
d6941300fc
|
[BugFix] Scope divergent hybrid cache hits to capable connectors (#50344)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-09 15:38:13 -07:00 |
|