 Juntian LiuandClaude Opus 5.5
|
19b0efce30
|
[Perf][Core] Skip the null head of sliding-window block tables (#61724)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 16:14:51 -07:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
4a89316efa
|
[Core] Drop the Ray driver-node GPU pin left over from the legacy executor (#61713)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 16:00:18 -07:00 |
|
|
|
3f1e57225a
|
[ROCm][Perf][Kimi-K3] Use AITER Gluon KDA kernels for decode and spec decode (#57392)
Signed-off-by: omuhamma <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
|
2026-10-15 17:54:24 -05:00 |
|
|
|
de7d70676d
|
[Perf] Low-latency CPU all-reduce for DP metadata syncs (#61767)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: hjjq <50634613+hjjq@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-15 15:29:13 -07:00 |
|
|
Stefano CastagnettaandKevin H. Luu
|
784d1fff0c
|
[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head (#59973)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 22:06:40 +00:00 |
|
|
|
d92a4f25f2
|
[watermarking] Add DFlash support to dual-key gumbel-max watermarking (#59554)
Signed-off-by: Shern Shiou Tan <shernshiou@gmail.com>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-15 22:06:06 +00:00 |
|
|
|
e711723352
|
[ROCm] gfx1250 optimizations and model enablement (#58800)
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
|
2026-10-15 14:51:57 -07:00 |
|
|
DimensionSTPandKevin H. Luu
|
9edcc7fa7e
|
[Bugfix][Spec Decode] Avoid reusing random draws across block verification rounds (#61161)
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 14:46:38 -07:00 |
|
|
Mikko Tukiainen
|
0c25849292
|
[ROCm][Perf][Bugfix] Probe the preshuffled FP4 GEMM config by logical K (#59406)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
|
2026-10-15 21:11:33 +00:00 |
|
|
|
f8460ef236
|
[Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache (#61371)
Signed-off-by: juntianl <juntianl@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-15 13:58:25 -07:00 |
|
|
|
33212db2c2
|
[KV Connector][NIXL] Reject NVFP4 KV cache with heterogeneous TP head split (#61865)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 13:57:37 -07:00 |
|
|
SeongJun LeeandWei Zhao
|
ee8c47961b
|
[CI] Use an ungated model as the attention test config default (#61732)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-10-15 20:19:09 +00:00 |
|
|
|
21f4b6dc84
|
[KV Offload] Split the CPU offload region across shm files (#59546)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 20:17:40 +00:00 |
|
|
|
d12ea8e309
|
[Bugfix][MRV2] Keep input_ids on later PP stages for raw-token models (#61016)
Signed-off-by: booth-algo <kevinlauofficial01@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 12:54:21 -07:00 |
|
|
|
d486219fc2
|
[Bugfix][Core] Uncount stop-trimmed spec tokens from num_computed_tokens (#58668)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-15 12:50:06 -07:00 |
|
|
|
49f944be3e
|
[Perf][HiSparse] Reuse pinned host pool after graph profiling (#61668)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 12:33:03 -07:00 |
|
|
Rohan PotdarandClaude Opus 5.5
|
105605b8e9
|
[ROCm] Support DeepSeek-V4 hash routing in AITER Triton W4A16 MoE (#60282)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 14:10:12 -05:00 |
|
|
Thien Tran
|
82b9ccaa79
|
[Perf][MoE] Skip the TRTLLM fp8-block activation-scale transpose copy (#61773)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-10-15 12:09:07 -07:00 |
|
|
|
4d58fa254a
|
[MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 (#61538)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 19:02:01 +00:00 |
|
|
Misha GoinandClaude Opus 5.5
|
4d9a8d6a41
|
[Bugfix] Make Humming DeviceInfo lookups traceable under torch.compile (#61858)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 11:12:21 -07:00 |
|
|
Mikko TukiainenandCursor
|
47fd76e4ce
|
[ROCm][Perf] Route softmax MoE top-k, including fused shared-expert sigmoid, through AITER topk_gating (#56226)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-10-15 17:53:18 +00:00 |
|
|
|
4f49b55c1f
|
[MoE] Shape-based padding for FlashInfer TRT-LLM MXFP8 experts (gated and non-gated) (#61293)
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-10-15 13:44:11 -04:00 |
|
|
|
7cd7982dfd
|
[Spec Decode] Don't reserve DFlash/DSpark draft slots in the scheduler token budget (#59468)
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-16 01:23:58 +08:00 |
|
|
bzsuniandFlora Feng
|
338dab077c
|
[Bugfix] Fix Apertus streaming scientific notation (#61812)
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-15 16:49:57 +00:00 |
|
|
|
957d512e96
|
[Bugfix][Frontend] Preserve streaming chunk boundaries in RequestOutputCollector (#58941)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Benedikt Falk <bfalk@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-15 16:38:28 +00:00 |
|
|
Jovan
|
a9e90b9d1d
|
[Bugfix][Frontend] Reject Cohere REQUIRED tool choice without tools (#61825)
Signed-off-by: Kevin <jia__kuan@163.com>
|
2026-10-15 16:37:08 +00:00 |
|
|
|
d6f0b7a416
|
[CI] Skip exited descendants when finding the worker in the FT e2e test (#61648)
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-15 16:08:14 +00:00 |
|
|
|
ed3050add4
|
[Core] Apply prefill_schedule_interval outside data parallelism (#54627)
Signed-off-by: lifulu <fululi12@amd.com>
Signed-off-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-10-15 11:03:13 -05:00 |
|
|
|
c825e1d235
|
[Perf] Fuse the mamba align block-table gather into one Triton kernel, 7 kernels -> 1, 6x speedup on all shapes for this kernel (#58737)
Signed-off-by: lifulu <fululi12@amd.com>
Signed-off-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com>
|
2026-10-15 11:02:53 -05:00 |
|
|
|
3dcfaeb4e8
|
[Frontend] Apply each endpoint's top_logprobs cut in derender (#59518)
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-15 15:04:04 +00:00 |
|
|
Matthias Gehre
|
011fa68ca9
|
[ROCm][Perf] Pad fp16/bf16 GEMM weight row stride for L2 cache (#55090)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-10-15 14:26:32 +00:00 |
|
|
Ronaldandaoshen02
|
d12d5460a5
|
[Bugfix][MoE] Keep routing rows during DeepSeek V4.1 decoder replay (#61643)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-15 14:02:06 +00:00 |
|
|
|
24f632ca7e
|
[Bugfix] Engage indexer TP row sharding on total rows, not per-rank rows (#61647)
Signed-off-by: Zheng Cai <caizheng1993@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-10-15 13:54:14 +00:00 |
|
|
mosafariuk
|
69425846f8
|
[Bugfix][CPU] Report the real cause when the worker outgrows its memory reservation (#59472)
Signed-off-by: Mo <mosafariuk@gmail.com>
|
2026-10-15 21:52:05 +08:00 |
|
|
|
b7b20b1b8f
|
[Bugfix][Frontend] Leave the final stop token out of chat derender text (#61757)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-15 21:51:34 +08:00 |
|
|
littlecircle0730andShanshan Shen
|
71536bd5f8
|
[MM][CG] Support ViT full CUDA graph for DeepSeek-VL2 (#46005)
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-10-15 21:38:52 +08:00 |
|
|
|
0951fc183c
|
[ROCm][DSv4.1] Disambiguate ROCm attention class names (#58689)
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 21:38:39 +08:00 |
|
|
|
1463501553
|
[Bugfix] Terminate streaming sessions on ABORT/ERROR and prevent zombie sessions (#58942)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Benedikt Falk <bfalk@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-15 13:32:19 +00:00 |
|
|
Xunzhuo
|
766c837bd5
|
[Model] Support every Decision 3.0 size (d3-flash, d3-mini, d3-nano, d3-lite, d3-edge) (#61778)
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com>
|
2026-10-15 13:29:24 +00:00 |
|
|
Martin Hickey
|
252be28c18
|
[Frontend] Replace return_mm_kwargs with an mm_payload enum on render requests (#61427)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-15 12:41:46 +00:00 |
|
|
Martin Hickey
|
c6b772cf6b
|
[Bugfix][Frontend] Emit kv/ec transfer params at the top level of render output (#61241)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-15 12:34:15 +00:00 |
|
|
Micah Williamson
|
e8e1e96196
|
[ROCm][DSv4] AITER MegaMoE Fixed-Slot Dispatch For DeepSeek-V4 (#61262)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
|
2026-10-15 19:10:59 +08:00 |
|
|
|
8b27d27402
|
[Bugfix][KV Offload] Always attempt NIXL agent removal during peer teardown (#60279)
Signed-off-by: nej1gotnochill <divyanshlalotra2@gmail.com>
Co-authored-by: Codebuff <noreply@codebuff.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 14:07:23 +03:00 |
|
|
Talor Abramovich
|
81e42c9987
|
[Feature][Frontend] Control the log level of request JSON body using env var (#60630)
Signed-off-by: talora <talora@nvidia.com>
|
2026-10-15 11:07:19 +00:00 |
|
|
|
e10c10a644
|
[Core] Add PROMOTION_LATENCY histogram metric for tiering offload (#53910)
Signed-off-by: Wen Zhou <wenzhou@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-10-15 13:47:41 +03:00 |
|
|
TimofeyandCodex
|
716c5225ae
|
[Model] Add AliceAI support (#57963)
Signed-off-by: timka <timka.byzov@yandex.ru>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-15 10:15:03 +00:00 |
|
|
|
7020600b1d
|
[KV Connector][NIXL] Register packed MLA KV rows as a single region (#61074)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-15 09:18:15 +00:00 |
|
|
Flora FengandChauncey
|
80dd705c1d
|
[Frontend] Make finish_reason and Responses item status truthful (#59324)
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-10-15 10:12:02 +01:00 |
|
|
wang.yuqi
|
0f0bc4dbb1
|
[Test] Split the chat_utils tests. (#61765)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-10-15 08:26:22 +00:00 |
|
|
|
b965ee564d
|
[Chore] Use upstream Transformers DeepseekV4Config (#61392)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 01:08:08 -07:00 |
|