Simon Danielsson
3f729dafcc
[Perf][ROCm][GLM-5.3-Flash] AITER prefill indexer topk ( #60753 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-10-14 13:48:56 -05:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
Nick Riasanovsky
a686c8e1b7
[Triton] Migrate Triton kernels off tl.make_block_ptr for triton-main compat ( #60502 )
...
Signed-off-by: Nick Riasanovsky <njriasan@meta.com >
2026-10-12 11:38:43 -04:00
f26d9f432c
[Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state ( #59147 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-11 17:55:00 +00:00
3e8e1364ff
[Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP ( #54951 )
...
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com >
Signed-off-by: Zheng Cai <caizheng1993@gmail.com >
Signed-off-by: Codex <noreply@openai.com >
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-11 10:43:37 -04:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
Chen Cheng and Isotr0py
d095e61c0b
[Bugfix][Spec Decode] Fix GLM-5.3-Flash DFlash2/EAGLE3 aux hidden-state capture with the upstream config ( #60763 )
...
Signed-off-by: Chen Cheng <85005591+ischencheng@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-09 11:35:48 +00:00
Shijin Zhang
2274e86842
[Spec Decode][Model] Support DFlash2 draft models with GLM-5.3-Flash ( #56983 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
2026-10-08 14:41:10 -04:00
9f3d70e081
[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) ( #55686 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: fxmarty-amd <felmarty@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:04:46 -04:00
Nicolò Lucchesi
70bdce6133
[Model][GLM-5.3-Flash] Warm up the DCP top-k merge kernels for the kpool indexer ( #60032 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 11:05:52 +02:00
2a54f6b625
[ROCm][Perf][GLM-5.3-Flash] Enable AITER fused shared experts on gfx950 ( #59221 )
...
Signed-off-by: Jin Tao <jin.tao@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-07 20:53:40 +08:00
Nicolò Lucchesi
30d4032363
[Model][GLM-5.3-Flash] Support DCP for the kpool sparse indexer ( #59211 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-05 16:44:14 +02:00
ae53b06898
[Bugfix][GLM-5.3-Flash] Keep batch x heads out of gridDim.z in the GLM-5.3-Flash fused recurrent KDA kernel ( #56974 )
...
Signed-off-by: NolenLiang <nliang@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com >
2026-10-05 09:31:45 +00:00
0979892992
[Bugfix][Multimodal] Fix GLM-5.3-Flash vision tower crashes on image input ( #59126 )
...
Signed-off-by: Juan Calderon-Perez <835733+gaby@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-02 15:25:37 +00:00
Simon Danielsson and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
98b29aa99a
[ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes ( #58167 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 11:47:41 +01:00
5688a4dd4a
[Bugfix][GLM-5.3] Size the image encoder cache from the exact token ceiling ( #59565 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Co-authored-by: Kimi Code <kimi-code@moonshot.cn >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-02 10:43:29 +00:00
Harry Mellor and Claude Opus 5
e3cae8d2ac
[Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor ( #57387 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-02 02:25:18 +00:00
df5668e5c9
[XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp ( #57172 )
...
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-30 12:19:33 +00:00
Simon Danielsson
3627a6a124
[ROCm][Perf][GLM-5.3-Flash] Stride-aware decode KDA ( #57979 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-30 08:16:29 +00:00
cjackal
0a30bc3f9a
[Model] Extend device-side mm normalization to GLM4V/GLM5Next ( #55389 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-09-30 15:44:46 +08:00
c37f86e572
[Bugfix] GLM-5.3-Flash: fp8 plan dtype on SM90 sparse MLA, and right-size the indexer prefill workspace ( #55222 )
...
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net >
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Mikhail Kostryukov <drakosha81@proton.me >
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-09-29 22:36:39 +00:00
Chauncey
3e2a7e74a5
[Feat][Model] Enable KDA prefill checkpoints for GLM-5.3-Flash ( #56960 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-09-29 09:14:30 +00:00
65ed0a2642
[Bugfix][GLM-5.3-Flash] Take video placeholder timestamps from the pixel path's frame sampler ( #55647 )
...
Signed-off-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-29 06:45:29 +00:00
2617fe9383
[Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding ( #58454 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 12:51:34 -04:00
Yifan Qiao and Claude Fable 5.1
39724758e7
[Bugfix] GLM-5.3-Flash: launch the kpool paged MQA logits in the varlen mode its schedule was built with ( #55270 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-25 00:13:55 +00:00
lucamotz
276db94fb2
[Bugfix][Model][Spec Decode] Defer disposable GLM MTP head ( #55442 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
2026-09-22 20:08:27 +00:00
c9b34fdb2d
[Bugfix][GLM-5.3-Flash] Run the dense MLP layers on the sequence-parallel shard ( #58061 )
...
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-22 21:58:21 +08:00
Chauncey
bf01fc4a31
[GLM-5.3-Flash] Route kpool indexer top-k through the shared SparseIndexerTopk dispatcher ( #57546 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-09-20 11:17:59 +00:00
db1bfdd4fb
[Bugfix][GLM-5.3-Flash] Address kpool tail blocks by the padded indexer stride in the NVIDIA prefill seed kernel ( #57477 )
...
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-20 18:15:04 +08:00
Wentao Ye
36fa72d2d0
[GLM5.3 Perf] Size the GLM-5 sparse indexer decode workspace, 3072 MiB GPU memory saved ( #57701 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-19 16:57:01 -04:00
Julien Debache and Harry Mellor
2bdae2a5a8
[fix] Mistral-Large-3 accuracy regression on main ( #57563 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-18 14:19:16 +00:00
d12c276853
[Bugfix][ROCm] Alias SparseAttnIndexerKpool.forward_cuda to forward_native (GLM-5.3-Flash boot crash) ( #57425 )
...
Signed-off-by: Mustafa YILDIRIM <mustafa@character.ai >
Co-authored-by: Hermes Agent <hermes@nousresearch.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-09-18 03:36:55 +00:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2bbdfcfcfe
[Perf][GLM5.3-Flash] Use cooperative top-k for small GLM decode batches ( #57327 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-18 10:31:35 +08:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Jiangyun Zhu and TJian
4fe9e6f6e5
[Refactor][GLM-5.3-Flash] Move sparse_attn_indexer_kpool into the model folder and split AMD/NVIDIA ( #55358 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-09-16 05:50:35 +00:00
Colin Z and Andreas Karatzas
c8d1cf077a
[ROCm] [Bugfix] Enable Load and Inference of GLM-5.3-Flash Quark MXFP4 Checkpoint ( #56176 )
...
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-16 01:25:29 +00:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
18f8aa0465
[Refactor] Remove dead kernel code ( #56845 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-15 13:17:53 -07:00
b443c1cc4e
[Perf][GLM-5.3-Flash] Use FlashKDA for KDA chunked prefill (1.7-3.8x faster than the Triton chunk path) ( #55737 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-14 08:36:19 +00:00
Roberto L. Castro and Codex
e52be1a62d
[9/N][warmup][DSv4] Migrate MHC TileLang kernels ( #50178 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-12 17:25:23 -07:00
aed894c190
[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels ( #56323 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-12 17:24:24 -07:00
Jared Wen and Claude Fable 5.1
1768273c13
[Perf][GLM-5.3-Flash] Decode hot-path cleanups: strided KDA recurrent inputs, NoPE MQA query without concat, no duplicate router GEMM ( #55736 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-10 16:28:33 +08:00
8369affa54
[Feat] Add EPLB support for GLM-5.3-Flash ( #55119 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-05 05:28:33 +00:00
Harry Mellor and Claude Opus 5
78300cdabf
[Bugfix][Docs] Package glm5next nvidia subtree and fix its docstrings ( #55214 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-03 21:25:26 -07:00
Jiangyun Zhu
98ed0856f3
[Model] add GLM-5.3-Flash support ( #53906 )
2026-09-03 09:40:35 -07:00