41 Commits
Author SHA1 Message Date
Mikko TukiainenCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f9b65c30c7 [ROCm][Perf] Make AITER FlyDSL GDN prefill default on gfx942/gfx950 (#60645)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 10:11:57 +00:00
Jiangyun ZhuandClaude Opus 5.5 badfcdbf91 [Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend (#60050)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:49:46 +08:00
liuzhenweiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ff440c7805 [BreakableCG] Add eager break for Mamba-family ops (#57881)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-10 14:06:57 +08:00
Harry MellorandClaude Opus 5.5 2c99ee9333 [Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:29:03 +00:00
Mikko TukiainenandCursor 548597367e [ROCm][Perf] Add AITER FlyDSL GDN prefill backend (#57560)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-07 09:44:22 -05:00
e4294144a8 [Perf] Keep non-speculative GDN decode on the standard path (#59735)
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
2026-10-02 13:44:53 +00:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 14:14:10 +00:00
44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-09-23 22:45:23 -07:00
Wentao Ye fae5cbfc8b [Refactor] Remove dead code multiple places (#58002)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-22 16:36:03 -04:00
Mikko TukiainenandCursor 68363d64ec [ROCm][Perf] Avoid extra reshape kernel in Qwen GDN output norm (#47842)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-22 14:38:16 +08:00
382970ee6c [Kimi-K3][AMD] Return KDA and MLA projection outputs directly (#50592)
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-21 15:44:41 -05:00
Yan Ma d05da62e9c [XPU] fix incorrect gdn kernel log for XPU path (#54871)
Signed-off-by: Yan Ma <yan.ma@intel.com>
2026-09-20 14:00:31 +08:00
fbe8a157fb [ROCm][Perf] Enable the AITER GDN decode fast path for flat qkvz layouts (#53623)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
2026-09-19 06:05:04 -07:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
975dca5bb5 [Perf][GDN] Scatter mixed speculative outputs into the caller buffer (#57140)
Signed-off-by: Arpan Tripathi <tripathiarpan20@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Xavier Lyu <9938360+xavierlyu@users.noreply.github.com>
Co-authored-by: Bohdan Podziubanchuk <114604629+Danbog32@users.noreply.github.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-16 13:03:57 +00:00
Stefano Castagnetta f6326f53bd [Perf][GDN] Enable the FlashInfer GDN prefill kernel on SM12x (#55715)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
2026-09-08 12:26:03 +00:00
Thien Tran d9105ea800 [Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation (#55272)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-09-06 21:57:56 -07:00
Jee Jee Li 199cb9b964 [Kernel] Remove unused fake implementation (#55535)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-09-06 17:54:32 -07:00
Li, Jiang fdbf2ddbd2 [Bugfix][CPU] Fix several bugs (#54042)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-08-31 15:33:06 +08:00
+5 e126687a9a [Model] Support Qwen3.8-Flash-Next (#53896)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-08-31 05:57:56 +00:00
7ab2923489 [Flashinfer] Upgrade Flashinfer version to 0.6.18 (#54313)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
2026-08-30 16:01:05 -07:00
a4d70bef37 [Model Runner V2] Reserve CUDA graph memory (#53306)
Signed-off-by: Anh Tran <anh.tran.3889@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Anh Tran <anh.tran.3889@gmail.com>
Co-authored-by: anhtra3889 <anhtra@nvidia.com>
2026-08-23 23:11:19 -07:00
kiroxu cdb8545a91 [Kernel][Perf] Support Qwen head ratios in fused GDN MTP (#52539)
Signed-off-by: kiroxu <148877251+BabyDrangoner@users.noreply.github.com>
2026-08-17 18:17:40 -07:00
1be3628367 [Kernel][Perf] Add fused CUDA post-conv MTP decode kernel for Qwen3.5 GDN (#51674)
Signed-off-by: Jie Fang <jief@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 11:44:45 +08:00
kliuae f96261637c [ROCm][Perf] Kimi-K3 Remove prefill pipeline stall in chunk KDA (#51862)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
2026-08-13 09:11:45 +00:00
Jiangyun ZhuandOpenAI Codex 5af7c8dad7 [Bugfix] Align Qwen GDN gates with speculative tokens (#51812)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-08-11 23:35:30 +08:00
skysnow2001andDouglas Lehr f43e1d26e3 [ROCm] Enable AITER and FP8 inference on GFX120x (#43615)
Signed-off-by: skysnow2001 <skysnow9285@gmail.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
2026-08-03 16:45:37 -05:00
+13 aeeb36b1f1 [New model] Kimi K3 (#50000)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
2026-07-30 03:49:58 -07:00
Wentao Ye 626c90b2d5 [Refactor] Move fla to third party (#48500)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-07-16 19:22:36 +01:00
Artem PerevedentsevandMatthew Bonanni 7cd1d57b74 [CI/Build][Docker] Bump nvidia-cutlass-dsl to 4.6.0 and drop packaging workarounds (#47442)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-07-16 13:51:41 +00:00
Canlin Guoandyewentao256 a02984ed47 [Perf][Qwen] Replace MOE all-reduce with reduce-scatter (#47006)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
2026-07-12 06:14:49 +00:00
Jiangyun Zhu 300e33797f [Perf] fuse more rmsnorm and all-reduce in qwen3.5 (#46998)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
2026-07-10 15:37:51 +08:00
AlexHuangClaudeKunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8d4f54966c fix(quantization): Fix AWQ dequantize on Intel XPU and refactor AutoAWQ config (#42727)
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: AlexHuang <jihuihuang@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-18 20:12:28 +08:00
Vadim Gimpelsonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> fa27d4e9cf [PERF] [Qwen3.5] Split mixed prefill+decode batches: route decodes to the recurrent kernel (#44700)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-06 22:13:50 +08:00
Jiangyun Zhu a55fccfc7c [mamba] unify KDA conv states into one cache to match 2-state SSM layout (#44539) 2026-06-04 20:38:05 +02:00
IdoAtadTD c91a87f01a [BugFix] [GDN] Read linear_key_head_dim from hf_text_config for multimodal models (#43978)
Signed-off-by: IdoAtadTD <ido.atad@twodelta.com>
2026-06-02 17:17:55 +03:00
zexplorerhjandhaojiangzheng f3b2a819f7 [Perf][KDA] Fuse gate softplus, chunk-local cumsum, and RCP_LN2 scaling (#43667)
Signed-off-by: haojiangzheng <justineric096@gmail.com>
Co-authored-by: haojiangzheng <justineric096@gmail.com>
2026-05-28 13:47:08 +00:00
Thien Tran d56612c621 [GDN] GDN Prefill kernel for SM100 (#43273)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-05-26 14:02:11 +08:00
Artem Perevedentsev 23f7b11bf4 [Bugfix] Detect wrong libcute_dsl_runtime.so variant in FlashInfer GDN (#43427)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-05-22 19:33:33 +00:00
wangxiyuan 7e1b45a092 [Attention] Mamba attention module refactor (#41126)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2026-05-22 17:13:12 +08:00