Harry Huang and Thien Tran
494663940b
[Qwen4Exp] Clean up HC and PLE implementations for Qwen3.8-Flash-Next ( #61454 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-18 19:38:21 +08:00
Andreas Karatzas and Claude
29345f42a6
[ROCm][Qwen4Exp] Support HC and MoE sequence parallelism on ROCm ( #61471 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-15 20:11:17 -05:00
77154ca611
[Perf][Model][Qwen4Exp] QSA: decode four FP8 e4m3 bytes per register below sm_89 ( #60992 )
...
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-14 13:06:16 +08:00
Stefano Castagnetta and Kartikey Vyas
4e0681ed56
[Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings ( #56273 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai >
2026-10-14 08:24:55 +08:00
Tim002 and Thien Tran
96fc2da315
[Perf][Qwen4Exp] Add SM121 TP=2 merged QSA projection plans ( #61437 )
...
Signed-off-by: TimRen01 <HolaTim@hotmail.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 22:58:44 +00:00
bc3ddb198a
[Model][Qwen4Exp] FP8 QSA main KV cache on Ampere (sm_80, sm_86) ( #60602 )
...
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Signed-off-by: kimseunghyun <incongnito12@gmail.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 15:30:19 +08:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c75cb63cd2
[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next ( #56322 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 20:56:44 +08:00
SeongJun Lee and lesj0610
a98247ab4d
[Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 ( #59853 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-10-10 06:51:52 +00:00
82e89e9944
[Perf][Qwen4Exp] Retune H200 M=4 merged QSA LL-GEMM plan ( #60776 )
...
Signed-off-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-09 07:18:50 +00:00
Mikko Tukiainen and Cursor
e2414ec1db
[ROCm][Qwen4Exp] Use fused PLE Triton kernels on the AMD backend ( #60021 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-08 15:47:56 +00:00
d9503dc503
[Bugfix][Qwen4Exp] Derive the QSA head_dim from the unsharded head count ( #59945 )
...
Signed-off-by: lesj0610 <lesj0610@godoiksan.org >
Co-authored-by: lesj0610 <lesj0610@godoiksan.org >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-07 12:24:40 +00:00
Stefano Castagnetta and Thien Tran
4ff028d77e
[Bugfix][Qwen4Exp] Honor --kv-cache-dtype-skip-layers in QSA attention ( #60023 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-05 12:13:30 +00:00
Stefano Castagnetta and Thien Tran
55b80221ed
[Perf][Qwen4Exp] Keep the HC up projection on the skinny GEMM path ( #60027 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-05 20:09:36 +08:00
c04c79e50d
[Performance] Add SM121 TP=2 skinny-GEMM plans ( #59632 )
...
Signed-off-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com >
Co-authored-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-05 18:27:51 +08:00
042ab0305c
[Perf][Qwen4Exp] Merge QSA QKVG and indexer QK projections ( #59533 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: namgyu-youn <namgyu.dev@gmail.com >
2026-10-05 07:03:29 +00:00
Harry Mellor
9e0b6fcf77
[CI] Bump Transformers version to 5.18.0 ( #59621 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-04 00:42:35 +00:00
Stefano Castagnetta and Benjamin Chislett
f590eb2448
[Perf][Qwen4Exp] Add SM121 TP=1 skinny-GEMM plans ( #59753 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-10-02 17:17:45 +00:00
Harry Mellor and Claude Opus 5
e3cae8d2ac
[Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor ( #57387 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-02 02:25:18 +00:00
cb6458c5ce
[Core] Remove deprecated mamba_cache_mode "all" ( #58997 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
2026-10-01 02:59:37 -07:00
Shuolei Wang and Thien
c9578f10ab
[Qwen3.8-Flash-Next] Fuse main QK-norm/RoPE/gate and KV-cache write into the QSA pre-indexer launch ( #57097 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Thien <thien@inferact.ai >
Co-authored-by: Thien <thien@inferact.ai >
2026-10-01 14:21:06 +08:00
namgyu-youn and Thien Tran
4ed438b79c
[Perf][Qwen4Exp] Add SM100 low-latency decode GEMM plans ( #59214 )
...
Signed-off-by: namgyu-youn <namgyu.dev@gmail.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-01 03:40:55 +00:00
cjackal and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
491f44adfa
[Model] Extend device-side mm normalization to Qwen3VL/Qwen3.5/Qwen4Next ( #51289 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 11:44:54 +08:00
Jiangyun Zhu and OpenAI Codex
20b52e9f5b
[Perf][Qwen3.8] Reduce PLE metadata construction overhead ( #58114 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 20:46:22 +08:00
Thien Tran
d6cce94fd4
[Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA ( #58957 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-28 18:51:38 +08:00
Thien Tran
7a877ae34d
[Qwen3.8-Flash-Next] Avoid memory fragmentation in QSA indexer logits workspace ( #57105 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-27 10:11:52 +08:00
267eee54bf
[Qwen4Exp][ROCm] PLE n-gram table CPU offload ( #57497 )
...
Signed-off-by: Mathew Odden <modden@redhat.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-24 23:28:20 -05:00
48d8880d09
[Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool ( #58489 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 10:51:50 +08:00
Jhao-Ting Chen
4fb767ff0a
[Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection ( #53175 )
...
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com >
2026-09-23 20:29:06 -07:00
4f9a41ac94
[CI/Build][Hardware][NVIDIA] Add Rubin CUDA 13.4 nightly images ( #55953 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-17 08:05:31 -07:00
Shuolei Wang and Roger Wang
7bbce752b8
[Perf][Model] Qwen4Exp QSA: sm_90 tuning table for _select_config ( #57273 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-09-17 07:02:24 +00:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
dff1bde84d
[Model] Qwen4Exp: fp8_e4m3 main KV cache on the QSA path ( #55557 )
...
Signed-off-by: Andrea Semeraro <andrea.semeraro@sezione1.it >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-09-16 15:07:52 +00:00
ec4a3a5370
[CI] Bump Transformers version to 5.17.0 ( #56108 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-16 06:34:16 -07:00
3f55ad2f07
[Qwen3.8-Flash-Next] Fuse PLE residual and QSA output gate ( #55309 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-14 12:16:25 +00:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
3116c5d06b
[Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism ( #54371 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 07:32:34 -07:00
60ad959b6f
Fix block FP8 MTP in ModelOpt mixed checkpoints ( #55513 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-08 22:47:21 +00:00
Thien Tran
94e26dd3dd
[Qwen3.8-Flash-Next] Support FP8 indexer cache for QSA ( #54890 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-07 12:15:45 +00:00
Thien Tran
d9105ea800
[Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation ( #55272 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-06 21:57:56 -07:00
Jee Jee Li
199cb9b964
[Kernel] Remove unused fake implementation ( #55535 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-09-06 17:54:32 -07:00
Harry Huang
28e605fb33
[Bugfix][Qwen4Exp] fix state index strides in fused PLE conv ( #55375 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
2026-09-05 14:02:20 +00:00
Thien Tran and Kimi
31a8a26662
[Qwen3.8-Flash-Next] Improve QSA sparse GQA for prefill and short-ctx decode ( #54873 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-04 06:08:02 -07:00
Canlin Guo
fd4a151262
[Kernel] Reuse Qwen4Exp HC combine-norm for MTP input ( #54687 )
...
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com >
Signed-off-by: Canlin <canlinguosdu@gmail.com >
2026-09-04 02:57:56 -07:00
Thien Tran
a5c9179e73
[Qwen3.8-Flash-Next] Compact indexer logits workspace to improve prefill efficiency ( #54915 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-04 00:43:20 -07:00
sychen52 and Vadim Gimpelson
d4d703caf9
[Bugfix][Model] Fix FP8 PLE loading in mixed ModelOpt checkpoints ( #54882 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-09-03 21:09:39 +04:00
Thien Tran and Codex
f870b92976
[Qwen3.8-Flash-Next] Fuse Qwen4Exp PLE kernels ( #54517 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Codex <codex@openai.com >
2026-09-02 12:32:16 +08:00
Harry Huang
1e300895ac
[Qwen4] validate FP8 PLE weight scale after loading ( #54722 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
2026-09-02 04:30:58 +00:00
Thien Tran
003e34341a
[Qwen3.8-Flash-Next] Separate prefill and decode paths for QSA indexer ( #54513 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-02 09:08:10 +08:00
Zheng Cai
191cecd51e
[Kernel][Qwen] Add Hopper LL-GEMM tuning table for Qwen4Exp ( #54560 )
...
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com >
2026-09-01 11:21:41 +00:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00