|
73a78e6f1f
|
[SpecDecode] Add LiLiCorr drafter (#57934)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2026-09-25 10:08:28 -04:00 |
|
|
Harry MellorandClaude Opus 5.5
|
afea5c20c7
|
[Docs] Add return annotation to fused_mm_input_norm_triton (#58687)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-25 10:01:11 +00:00 |
|
|
Yongye ZhuandClaude Opus 5.5
|
378504a544
|
[MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-25 15:38:29 +08:00 |
|
|
Isotr0py
|
24928a4d62
|
[UX][Frontend] Introduce vllm preload cli for fast restart (#56680)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-25 06:56:39 +00:00 |
|
|
|
29468dde8b
|
[Bugfix][KV Cache][MLA] Align packed block strides for V3.2 sparse MLA (#55528)
Signed-off-by: lz <145014769+200lz@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-25 05:33:36 +00:00 |
|
|
Lingpeng JinandAndreas Karatzas
|
35f92d2abc
|
[ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-launch split-K (#58659)
Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-24 21:37:41 -07:00 |
|
|
|
8a269919f4
|
[MM] Add Triton kernel for mm_input_normal. (#56798)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-25 04:31:20 +00:00 |
|
|
|
4d79d5c638
|
[ROCm] Cut 69 wasted contiguous copies per decode step from the skinny GEMM path (#58566)
Signed-off-by: lifulu <fululi12@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-24 22:00:58 -05:00 |
|
|
Samuel Kimandmgoin
|
f55419c43c
|
[Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223)
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-09-25 02:54:11 +00:00 |
|
|
|
6936e77ba4
|
[Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-25 10:52:02 +08:00 |
|
|
Fangzhou AiandClaude Opus 5.5
|
3059155b47
|
[ROCm][Perf] MXFP8 GEMM on native 32x32 block scales for gfx950 (#58510)
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-24 21:26:30 -05:00 |
|
|
Divakar Verma
|
65c5e24689
|
[ROCm][CI][AITER Coverage] Harden MoE sorting-backend/dispatch env-var test matrix (#58393)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
|
2026-09-24 21:14:02 -05:00 |
|
|
|
3652f35e7c
|
[Bugfix][Quantization] Refresh online NVFP4 scales before reload post-processing (#57954)
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-09-24 18:36:08 -07:00 |
|
|
|
16070ed7e9
|
[Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope (#58526)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-24 18:30:32 -07:00 |
|
|
Thien Tran
|
ade1056ee5
|
[Kimi-K3][Perf] Dispatch GEMM for vision patch embedder (#58527)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-09-24 18:30:11 -07:00 |
|
|
Samuel KimandClaude
|
34ee85d5ca
|
[Perf] Batch Mamba2 prefill SSM state saves, removing GPU<->CPU syncs (#49371)
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-25 01:11:19 +00:00 |
|
|
Misha GoinandCodex
|
71891bdb94
|
[Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-24 19:52:00 -04:00 |
|
|
Misha Goin
|
4f72bd0152
|
[Bugfix][Quantization] Give LM heads standard linear metadata (#58444)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-09-24 16:39:49 -07:00 |
|
|
Miłosz Grunwald
|
bd4cb3fe4e
|
[BUGFIX] fix ovis2_5 multimodal tokens (#52623)
Signed-off-by: Milosz Grunwald <milosz.grunwald@intel.com>
|
2026-09-25 07:18:54 +08:00 |
|
|
Aaron KangandMisha Goin
|
0908116dd9
|
[Bugfix] Pass quant_config to DiffusionGemma's ParallelLMHead (#48521)
Signed-off-by: Aaron Kang <aaron.h.kang@icloud.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-09-24 13:17:02 -07:00 |
|
|
|
ab3de6edf2
|
[Perf] DiffusionGemma: constrained reads over the request's logprob_token_ids (#58216)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-09-24 13:15:08 -07:00 |
|
|
|
90a9515006
|
[Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-24 12:48:14 -07:00 |
|
|
Misha Goin
|
cd06e81c41
|
[Bugfix][Quantization] Add Humming to the W4A8 (INT4xFP8) MoE oracle (#58427)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-09-24 12:37:33 -07:00 |
|
|
Xun Sun
|
e773247f8a
|
[Fast Start] Wait for weight cache daemon readiness (#58370)
|
2026-09-24 19:18:04 +00:00 |
|
|
|
5b3f280a6d
|
[Bugfix] Count unsplit Idefics3 image patches (#48760)
Signed-off-by: nightcityblade <nightcityblade@gmail.com>
Co-authored-by: nightcityblade <nightcityblade@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-24 11:36:26 -07:00 |
|
|
Thomas OrtnerandHarry Mellor
|
8b84e15066
|
[transformer] RMSNorm matching for alternative rsqrt (#54461)
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-09-24 16:33:06 +00:00 |
|
|
Nick Hill
|
09fe178dba
|
Revert "[DSpark] Support pipeline-parallel targets in aggregated serving (#56956)" (#58484)
|
2026-09-24 08:48:36 -07:00 |
|
|
|
344fcc252a
|
[Bugfix] Resolve the Hub revision once per repo (#56092)
Signed-off-by: Wauplin <lucainp@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-24 08:48:24 -07:00 |
|
|
Harry MellorandClaude Opus 5.5
|
d881812500
|
[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-09-24 14:14:10 +00:00 |
|
|
Linze Shi
|
484c211fe8
|
[Bugfix][Pooling] Fix JinaVL label configuration and restore multimodal tests (#57347)
Signed-off-by: Linze-Shi <linzeshi0@gmail.com>
|
2026-09-24 13:51:25 +00:00 |
|
|
|
1291bdb85e
|
[EPD][Model Loader] Skip language-model checkpoint shards for --mm-encoder-only (#58086)
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
|
2026-09-24 00:41:15 -07:00 |
|
|
|
67ddac3ea8
|
[CPU][GDN] Support NIXL DS convolution-state layout (#53300)
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-24 14:55:20 +08:00 |
|
|
|
44bf011f44
|
[Dependency] Upgrade FlashInfer version to 0.7.0 (#58069)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-09-23 22:45:23 -07:00 |
|
|
|
8b06a343ec
|
[Bugfix][CPU][MoE] Fix out-of-bounds write and segfault when router weights are fp32 (#56168)
Signed-off-by: Farzad Abdolhosseini <farzad@elastix.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-24 13:15:17 +08:00 |
|
|
Jhao-Ting Chen
|
4fb767ff0a
|
[Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection (#53175)
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
|
2026-09-23 20:29:06 -07:00 |
|
|
|
e6dc16cebd
|
[PCP] Support prefill context parallelism with data parallelism (#57075)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-23 22:32:36 -04:00 |
|
|
Juntian LiuandClaude Opus 5.5
|
9ef37771be
|
[Bugfix][DSA] Bound DeepSelect sentinel columns in the sparse top-k remap (#58215)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-09-24 00:54:27 +00:00 |
|
|
djramicandAndreas Karatzas
|
5fcc6e7c73
|
[ROCm][Bugfix] Fix TileLang mHC fused RMSNorm on 64-wide wavefronts (#58419)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-23 23:03:45 +00:00 |
|
|
|
b715699df4
|
[ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE (#50212)
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com>
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
|
2026-09-23 11:11:57 -05:00 |
|
|
fxmarty-amdandmgoin
|
0549e8d0ab
|
[Quark] Remove quark-specific silent online quantization (#51800)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-09-23 11:35:30 -04:00 |
|
|
HDCharles
|
9f07d023d0
|
[Quantization] Enable humming wNaM asymmetric quant (zero_point) with compressed-tensors (#46528)
|
2026-09-23 10:23:11 -04:00 |
|
|
Shuolei Wang
|
9ed81581a8
|
[Perf][MoE] Skip top-k slots routed to non-local experts in TritonExp… (#58051)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
|
2026-09-23 21:03:21 +08:00 |
|
|
|
157bcb7c48
|
[DSpark] Support pipeline-parallel targets in aggregated serving (#56956)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-09-23 04:52:28 -07:00 |
|
|
aoshen02andYongye Zhu
|
973a3be780
|
[Refactor][Quantization] Make FP8 and MLA weight transforms reusable pure functions (#57732)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-09-23 01:16:20 -07:00 |
|
|
|
9127170b88
|
[Quantization] Select per-token NVFP4 MoE backends explicitly (#57176)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-23 07:30:58 +00:00 |
|
|
00CC
|
eb2e91d3ba
|
[ROCm][Perf] Use wvSplitK for single-output GEMMs (#53283)
Signed-off-by: tangzzycc <3081129260@qq.com>
|
2026-09-23 06:38:17 +00:00 |
|
|
Jee Jee Li
|
de6674b002
|
[MoE] Use GateLinear for all MoE models (#58234)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-09-23 05:40:13 +00:00 |
|
|
|
88aa0d287d
|
[Spec decode] Support variable-length decode for Kimi-K3 adaptive ver (#52988)
Signed-off-by: Albert Cheng <albecheng@nvidia.com>
Signed-off-by: Albert Cheng (Engrg-Hardware 1) <albecheng@login-bia01.bia.clusters.nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-09-22 20:43:32 -07:00 |
|
|
Michał GanczarenkoandClaude
|
73c59d365b
|
[XPU] Wire up SYCL apply_rotary_emb kernel in ApplyRotaryEmb (#55721)
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Signed-off-by: Michał Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-22 20:43:24 -07:00 |
|
|
Tianyu GuoandCyrus Leung
|
3ee2906f47
|
[EPD] Support metadata-only audio inputs (#57887)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-23 09:03:50 +08:00 |
|