0b7f11a1ee
[AuxOutput] Add block-keyed storage for routed-expert outputs ( #45635 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-21 10:25:05 +00:00
Misha Goin and ErinYin
e5fce7b56b
[Core][Kernel] Share persistent workspaces for Marlin and Humming ( #57421 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: ErinYin <liang.yin@daocloud.io >
2026-09-20 17:28:33 +08:00
Woosuk Kwon
38537331d3
[Bugfix][DSA] Avoid runtime JIT for offset candidate end buffers ( #57667 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-19 00:27:11 -07:00
Andreas Karatzas and OpenAI Codex
2c3fb4c545
[CI][ROCm] Deprecate DinD for MI250 test groups ( #56162 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-18 14:13:35 -07:00
Woosuk Kwon and Codex
64563d0ec4
[Bugfix][DSv4.1] Preserve NaN-scored candidate block indices ( #57454 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-17 19:00:10 -07:00
b2f2bd71ba
[CI][Bugfix] Add tp_shard_with_padding to padded MoE reload test mock ( #57402 )
...
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-17 13:02:24 -04:00
78fdf4efb8
[Bugfix][MoE] Convert FlashInfer BF16 weights in place ( #54699 )
...
Signed-off-by: Yuchen Wang <93700456+yuchenwang3@users.noreply.github.com >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-17 22:00:04 +08:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Jared Wen and Claude Fable 5.1
1257512305
[DSA] Wire DeepGEMM sparse MQA logits into the DeepSeek V4.1 indexer ( #56254 )
...
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-15 14:07:14 -07:00
Kevin H. Luu
2f46a1d9c6
[CI] Keep Qwen3 Omni DSpark config fixture complete ( #57044 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-09-15 18:19:14 +00:00
aoshen02
eb61707851
[Bugfix] Fix stale HPC QK-norm weights after weight refit ( #56706 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
2026-09-14 21:34:41 -07:00
Netanel Haber
d4ee7fe7a9
[Quantization] Select linear backends per quantization ( #51204 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
2026-09-14 16:24:18 -04:00
03dc26e639
[Perf][Kernel][Quantization] Fuse ReLU2 with static FP8 activation quantization ( #53793 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: ElizaWszola <ewszola@redhat.com >
Signed-off-by: snordmann <snordmann@nvidia.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: ElizaWszola <ewszola@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-14 10:26:15 -07:00
c676e4930b
[Spec Decode] Fix Qwen3 DSpark d2t requirement for padded-vocab drafts ( #55133 )
...
Signed-off-by: Orestis Zambounis <23146389+orestis-z@users.noreply.github.com >
Signed-off-by: Orestis Zambounis <orestis.zambounis@gmail.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-14 10:07:07 -04:00
Isotr0py
dbf49dad11
[Fast Start] Use GPU uuid as socket folder identifier ( #56669 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-14 06:28:13 +00:00
Kevin H. Luu
2d9c30a7b5
[CI][Bugfix] Complete FSE fixture R-SWA contract ( #56710 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-09-14 03:30:25 +00:00
Kevin H. Luu
dec0b5d63f
[Bugfix] Skip Triton autotune inspection without Triton ( #56676 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-09-13 17:05:29 +00:00
liuzhenwei
de500290ff
[XPU][CI] fix jit_warmup_triton_launcher ( #56670 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-09-13 21:44:24 +08:00
aed894c190
[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels ( #56323 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-12 17:24:24 -07:00
7ee8a6dd01
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support ( #56122 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
2026-09-12 18:42:01 +00:00
Kevin H. Luu and OpenAI Codex
fd7cd4b883
[CI] Fix Qwen3 Omni DSpark load test config ( #56600 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-12 11:29:58 -07:00
Ayushman Singh and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7bcca16729
[Bugfix] Fix DeepGEMM FP8 warmup coverage ( #56452 )
...
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-12 14:20:38 -04:00
d43bb2f37f
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering ( #53781 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com >
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-12 00:36:47 -04:00
Roberto L. Castro and Codex
120ec4ebd2
[5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels ( #53566 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-12 00:05:59 -04:00
e77daef89e
[Model] Support DeepSeek-V4.1-Flash ( #56214 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-11 02:11:19 -07:00
siyu and Isotr0py
73fb19151f
[Fast Start] Support fp4 ( #55465 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 13:22:17 +00:00
fe26c705f7
[Model] Add Bailing V3 VL support ( #55921 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 02:46:56 -07:00
Qiming Zhang
49632fb3ac
[CI][XPU] Reject CUDA-IPC weight cache on non-CUDA/ROCm platforms ( #56010 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
2026-09-10 14:05:07 +08:00
Jing Wang
95cf420ac1
[CI][Test][Spec Decode] Fix CI failure of Qwen3 Omni DSpark loader mock ( #56011 )
...
Signed-off-by: Jing Wang <jingwang96@qq.com >
2026-09-08 23:06:15 -07:00
Roderick Wu and mgoin
2c9d68f80b
[Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering ( #54809 )
...
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com >
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-08 12:45:12 -07:00
Roberto L. Castro and Codex
6ddbab03de
[4/N][warmup][DSv4] Migrate common attention kernels ( #50176 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-08 10:25:48 -04:00
weihan
9ea8f3ffc3
[Bugfix] Honor STEP token pooling in DispatchPooler.for_seq_cls ( #55307 )
...
Signed-off-by: weihan <130119942+krsish@users.noreply.github.com >
2026-09-08 04:13:09 +00:00
7fa2c63796
[Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture ( #55377 )
...
Signed-off-by: ming <ming@inferact.ai >
Co-authored-by: Kimi Code <kimi-code@moonshot.cn >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-08 04:02:23 +00:00
1713b9866a
[2/N][warmup][DSv4] Migrate sequence and DCP kernels ( #53564 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 09:35:54 -04:00
vhagor and Isotr0py
3dc7a68ce4
[Perf] Extend Qwen Triton warmup to avoid first-request latency spikes ( #54797 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-07 11:46:00 +08:00
yulun and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
1f778486fc
[Bugfix][Offloader] Preserve prefetch static-buffer slot ownership ( #54975 )
...
Signed-off-by: Big2Wheel <1404207590@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 10:58:44 +08:00
de69e821b7
[ROCm][Perf][DeepSeek V4] Fuse native FP8 shared expert with MXFP4 routed experts ( #53161 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Cursor Agent <agent@cursor.com >
2026-09-06 19:42:21 -07:00
d87a440f88
[Core][MRV2] Support eagle3 spec decode with pipeline parallel ( #50514 )
...
Signed-off-by: Yongqin Wang <yongqinwang@roblox.com >
Co-authored-by: Cursor <noreply@cursor.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-05 11:45:34 +08:00
3f41d102c5
[1/2][Model Runner V2] DBO support, eager mode ( #50945 )
...
Signed-off-by: specture724 <specture724@gmail.com >
Signed-off-by: Anjie Hou <149605198+specture724@users.noreply.github.com >
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-09-04 14:17:39 -07:00
5690b02c03
[Quantization] Support online quantization with partially pre-quantized checkpoints ( #51392 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-04 11:46:06 -07:00
a69e75b9b6
Fast Start ( #54921 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: liusy58 <liusy58@smail.nju.edu.cn >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-04 06:32:06 -07:00
Luyi Xiao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8a728663c1
[Bugfix] Reject tokenizer-less Qwen VL processor init ( #54886 )
...
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-03 21:25:35 -07:00
Nick Hill
e55b93f296
[Core] Deprecate "all" mamba cache mode ( #55041 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-03 08:56:59 -07:00
Luke Alonso
1945a94575
[Bugfix][Tests] Stabilize B12X linear kernel checks ( #54996 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
2026-09-02 18:46:08 +00:00
2fe5cef35e
[Fix] Fix FSE compatibility detection for Quark-produced models ( #54573 )
...
Signed-off-by: xuebwang-amd <xuebwang@amd.com >
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: xuebwang-amd <xuebwang@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-01 12:12:20 -05:00
Andrii Skliar and Andrii Skliar
adebc41b7e
[Mamba] Add FlashInfer ReplaySSM backend ( #52506 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Andrii Skliar <andreyws96@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-09-01 10:26:43 -04:00
Subhash Polisetti
4707679cd2
[Bugfix][MiniCPM-V] Route video_embeds to the shared vision parser ( #54633 )
...
Signed-off-by: subhash polisetti <subhashr161347@gmail.com >
2026-09-01 09:16:07 +00:00
Roberto L. Castro and Codex
e16b5e518d
[1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels ( #50175 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-08-31 23:26:05 -04:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
