 Anerudhan GopalandOpenAI Codex
|
af28c171f5
|
[Dependency] Upgrade FlashInfer to 0.7.2 and restore KDA auto backend (#61913)
Signed-off-by: Anerudhan Gopal <agopal@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-15 17:04:24 -07:00 |
|
|
Juntian LiuandClaude Opus 5.5
|
19b0efce30
|
[Perf][Core] Skip the null head of sliding-window block tables (#61724)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 16:14:51 -07:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
4a89316efa
|
[Core] Drop the Ray driver-node GPU pin left over from the legacy executor (#61713)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 16:00:18 -07:00 |
|
|
|
3f1e57225a
|
[ROCm][Perf][Kimi-K3] Use AITER Gluon KDA kernels for decode and spec decode (#57392)
Signed-off-by: omuhamma <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
|
2026-10-15 17:54:24 -05:00 |
|
|
|
de7d70676d
|
[Perf] Low-latency CPU all-reduce for DP metadata syncs (#61767)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: hjjq <50634613+hjjq@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-15 15:29:13 -07:00 |
|
|
Stefano CastagnettaandKevin H. Luu
|
784d1fff0c
|
[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head (#59973)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 22:06:40 +00:00 |
|
|
|
d92a4f25f2
|
[watermarking] Add DFlash support to dual-key gumbel-max watermarking (#59554)
Signed-off-by: Shern Shiou Tan <shernshiou@gmail.com>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-15 22:06:06 +00:00 |
|
|
|
e711723352
|
[ROCm] gfx1250 optimizations and model enablement (#58800)
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
|
2026-10-15 14:51:57 -07:00 |
|
|
Thang NguyenandClaude Opus 5.5
|
9b99ccb56b
|
[CI] Shard the AMD mirrors of the DP, RayExecutorV2 and batch invariance steps automatically (#61861)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 16:48:19 -05:00 |
|
|
DimensionSTPandKevin H. Luu
|
9edcc7fa7e
|
[Bugfix][Spec Decode] Avoid reusing random draws across block verification rounds (#61161)
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 14:46:38 -07:00 |
|
|
Mikko Tukiainen
|
0c25849292
|
[ROCm][Perf][Bugfix] Probe the preshuffled FP4 GEMM config by logical K (#59406)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
|
2026-10-15 21:11:33 +00:00 |
|
|
|
f8460ef236
|
[Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache (#61371)
Signed-off-by: juntianl <juntianl@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-15 13:58:25 -07:00 |
|
|
|
33212db2c2
|
[KV Connector][NIXL] Reject NVFP4 KV cache with heterogeneous TP head split (#61865)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 13:57:37 -07:00 |
|
|
Thang NguyenandClaude Opus 5.5
|
c414bf0295
|
[CI] Shard four AMD mirrors automatically (#61817)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 13:57:35 -07:00 |
|
|
Wentao Ye
|
54d9b1cb48
|
[GLM5.3 Perf] Inplace update draft decode metadata, 1.7% E2E Throughput Improvement (#61873)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-15 20:20:12 +00:00 |
|
|
SeongJun LeeandWei Zhao
|
ee8c47961b
|
[CI] Use an ungated model as the attention test config default (#61732)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-10-15 20:19:09 +00:00 |
|
|
|
21f4b6dc84
|
[KV Offload] Split the CPU offload region across shm files (#59546)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 20:17:40 +00:00 |
|
|
Thang NguyenandClaude Opus 5.5
|
16532876e7
|
[CI] Shard four more AMD mirrors automatically (#61860)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 15:13:51 -05:00 |
|
|
|
d12ea8e309
|
[Bugfix][MRV2] Keep input_ids on later PP stages for raw-token models (#61016)
Signed-off-by: booth-algo <kevinlauofficial01@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 12:54:21 -07:00 |
|
|
|
d486219fc2
|
[Bugfix][Core] Uncount stop-trimmed spec tokens from num_computed_tokens (#58668)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-15 12:50:06 -07:00 |
|
|
|
49f944be3e
|
[Perf][HiSparse] Reuse pinned host pool after graph profiling (#61668)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 12:33:03 -07:00 |
|
|
Rohan PotdarandClaude Opus 5.5
|
105605b8e9
|
[ROCm] Support DeepSeek-V4 hash routing in AITER Triton W4A16 MoE (#60282)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 14:10:12 -05:00 |
|
|
Thien Tran
|
82b9ccaa79
|
[Perf][MoE] Skip the TRTLLM fp8-block activation-scale transpose copy (#61773)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-10-15 12:09:07 -07:00 |
|
|
|
4d58fa254a
|
[MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 (#61538)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 19:02:01 +00:00 |
|
|
Misha GoinandClaude Opus 5.5
|
4d9a8d6a41
|
[Bugfix] Make Humming DeviceInfo lookups traceable under torch.compile (#61858)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 11:12:21 -07:00 |
|
|
Mikko TukiainenandCursor
|
47fd76e4ce
|
[ROCm][Perf] Route softmax MoE top-k, including fused shared-expert sigmoid, through AITER topk_gating (#56226)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-10-15 17:53:18 +00:00 |
|
|
|
4f49b55c1f
|
[MoE] Shape-based padding for FlashInfer TRT-LLM MXFP8 experts (gated and non-gated) (#61293)
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-10-15 13:44:11 -04:00 |
|
|
|
7cd7982dfd
|
[Spec Decode] Don't reserve DFlash/DSpark draft slots in the scheduler token budget (#59468)
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-16 01:23:58 +08:00 |
|
|
bzsuniandFlora Feng
|
338dab077c
|
[Bugfix] Fix Apertus streaming scientific notation (#61812)
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-15 16:49:57 +00:00 |
|
|
|
957d512e96
|
[Bugfix][Frontend] Preserve streaming chunk boundaries in RequestOutputCollector (#58941)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Benedikt Falk <bfalk@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-15 16:38:28 +00:00 |
|
|
Jovan
|
a9e90b9d1d
|
[Bugfix][Frontend] Reject Cohere REQUIRED tool choice without tools (#61825)
Signed-off-by: Kevin <jia__kuan@163.com>
|
2026-10-15 16:37:08 +00:00 |
|
|
|
d6f0b7a416
|
[CI] Skip exited descendants when finding the worker in the FT e2e test (#61648)
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-15 16:08:14 +00:00 |
|
|
|
ed3050add4
|
[Core] Apply prefill_schedule_interval outside data parallelism (#54627)
Signed-off-by: lifulu <fululi12@amd.com>
Signed-off-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-10-15 11:03:13 -05:00 |
|
|
|
c825e1d235
|
[Perf] Fuse the mamba align block-table gather into one Triton kernel, 7 kernels -> 1, 6x speedup on all shapes for this kernel (#58737)
Signed-off-by: lifulu <fululi12@amd.com>
Signed-off-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com>
|
2026-10-15 11:02:53 -05:00 |
|
|
CharlesWuandYichao Zhu
|
a97d19d792
|
[UMBP][1/10] Add shared connector, runtime contracts, and request lifecycle (#61789)
Signed-off-by: CharlesWu <Charles.Wu@amd.com>
Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com>
|
2026-10-15 15:12:39 +00:00 |
|
|
|
3dcfaeb4e8
|
[Frontend] Apply each endpoint's top_logprobs cut in derender (#59518)
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-15 15:04:04 +00:00 |
|
|
Matthias Gehre
|
011fa68ca9
|
[ROCm][Perf] Pad fp16/bf16 GEMM weight row stride for L2 cache (#55090)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-10-15 14:26:32 +00:00 |
|
|
Ronaldandaoshen02
|
d12d5460a5
|
[Bugfix][MoE] Keep routing rows during DeepSeek V4.1 decoder replay (#61643)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-15 14:02:06 +00:00 |
|
|
|
24f632ca7e
|
[Bugfix] Engage indexer TP row sharding on total rows, not per-rank rows (#61647)
Signed-off-by: Zheng Cai <caizheng1993@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-10-15 13:54:14 +00:00 |
|
|
mosafariuk
|
69425846f8
|
[Bugfix][CPU] Report the real cause when the worker outgrows its memory reservation (#59472)
Signed-off-by: Mo <mosafariuk@gmail.com>
|
2026-10-15 21:52:05 +08:00 |
|
|
|
b7b20b1b8f
|
[Bugfix][Frontend] Leave the final stop token out of chat derender text (#61757)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-15 21:51:34 +08:00 |
|
|
littlecircle0730andShanshan Shen
|
71536bd5f8
|
[MM][CG] Support ViT full CUDA graph for DeepSeek-VL2 (#46005)
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-10-15 21:38:52 +08:00 |
|
|
|
0951fc183c
|
[ROCm][DSv4.1] Disambiguate ROCm attention class names (#58689)
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 21:38:39 +08:00 |
|
|
|
1463501553
|
[Bugfix] Terminate streaming sessions on ABORT/ERROR and prevent zombie sessions (#58942)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Benedikt Falk <bfalk@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-15 13:32:19 +00:00 |
|
|
Xunzhuo
|
766c837bd5
|
[Model] Support every Decision 3.0 size (d3-flash, d3-mini, d3-nano, d3-lite, d3-edge) (#61778)
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com>
|
2026-10-15 13:29:24 +00:00 |
|
|
Laith SakkaandNick Hill
|
9cede34a67
|
[Bugfix] Create cuBLAS handles before memory profiling to avoid OOM (#61044)
Signed-off-by: Laith Sakka <lsakka@meta.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-15 08:59:22 -04:00 |
|
|
Martin Hickey
|
252be28c18
|
[Frontend] Replace return_mm_kwargs with an mm_payload enum on render requests (#61427)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-15 12:41:46 +00:00 |
|
|
Martin Hickey
|
c6b772cf6b
|
[Bugfix][Frontend] Emit kv/ec transfer params at the top level of render output (#61241)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-15 12:34:15 +00:00 |
|
|
Micah Williamson
|
e8e1e96196
|
[ROCm][DSv4] AITER MegaMoE Fixed-Slot Dispatch For DeepSeek-V4 (#61262)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
|
2026-10-15 19:10:59 +08:00 |
|
|
|
8b27d27402
|
[Bugfix][KV Offload] Always attempt NIXL agent removal during peer teardown (#60279)
Signed-off-by: nej1gotnochill <divyanshlalotra2@gmail.com>
Co-authored-by: Codebuff <noreply@codebuff.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 14:07:23 +03:00 |
|