45 Commits
Author SHA1 Message Date
Simon Danielsson 3f729dafcc [Perf][ROCm][GLM-5.3-Flash] AITER prefill indexer topk (#60753)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
2026-10-14 13:48:56 -05:00
Yongye ZhuandClaude Opus 5.5 a6009ea85b [Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer (#61149)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 23:24:43 -07:00
Nick Riasanovsky a686c8e1b7 [Triton] Migrate Triton kernels off tl.make_block_ptr for triton-main compat (#60502)
Signed-off-by: Nick Riasanovsky <njriasan@meta.com>
2026-10-12 11:38:43 -04:00
Jared WenClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f26d9f432c [Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state (#59147)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-11 17:55:00 +00:00
3e8e1364ff [Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP (#54951)
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com>
Signed-off-by: Zheng Cai <caizheng1993@gmail.com>
Signed-off-by: Codex <noreply@openai.com>
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-10-11 10:43:37 -04:00
02e55d2af3 [KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-10-09 15:48:37 -04:00
Chen ChengandIsotr0py d095e61c0b [Bugfix][Spec Decode] Fix GLM-5.3-Flash DFlash2/EAGLE3 aux hidden-state capture with the upstream config (#60763)
Signed-off-by: Chen Cheng <85005591+ischencheng@users.noreply.github.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-10-09 11:35:48 +00:00
Shijin Zhang 2274e86842 [Spec Decode][Model] Support DFlash2 draft models with GLM-5.3-Flash (#56983)
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
2026-10-08 14:41:10 -04:00
9f3d70e081 [Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-10-08 12:04:46 -04:00
Nicolò Lucchesi 70bdce6133 [Model][GLM-5.3-Flash] Warm up the DCP top-k merge kernels for the kpool indexer (#60032)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-10-08 11:05:52 +02:00
2a54f6b625 [ROCm][Perf][GLM-5.3-Flash] Enable AITER fused shared experts on gfx950 (#59221)
Signed-off-by: Jin Tao <jin.tao@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
2026-10-07 20:53:40 +08:00
Nicolò Lucchesi 30d4032363 [Model][GLM-5.3-Flash] Support DCP for the kpool sparse indexer (#59211)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-10-05 16:44:14 +02:00
bcsdhjewmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Codex
ae53b06898 [Bugfix][GLM-5.3-Flash] Keep batch x heads out of gridDim.z in the GLM-5.3-Flash fused recurrent KDA kernel (#56974)
Signed-off-by: NolenLiang <nliang@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-05 09:31:45 +00:00
0979892992 [Bugfix][Multimodal] Fix GLM-5.3-Flash vision tower crashes on image input (#59126)
Signed-off-by: Juan Calderon-Perez <835733+gaby@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-10-02 15:25:37 +00:00
Simon Danielssonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 98b29aa99a [ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes (#58167)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 11:47:41 +01:00
5688a4dd4a [Bugfix][GLM-5.3] Size the image encoder cache from the exact token ceiling (#59565)
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-10-02 10:43:29 +00:00
Harry MellorandClaude Opus 5 e3cae8d2ac [Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor (#57387)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-02 02:25:18 +00:00
df5668e5c9 [XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp (#57172)
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-09-30 12:19:33 +00:00
Simon Danielsson 3627a6a124 [ROCm][Perf][GLM-5.3-Flash] Stride-aware decode KDA (#57979)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
2026-09-30 08:16:29 +00:00
cjackal 0a30bc3f9a [Model] Extend device-side mm normalization to GLM4V/GLM5Next (#55389)
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
2026-09-30 15:44:46 +08:00
c37f86e572 [Bugfix] GLM-5.3-Flash: fp8 plan dtype on SM90 sparse MLA, and right-size the indexer prefill workspace (#55222)
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Mikhail Kostryukov <drakosha81@proton.me>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-09-29 22:36:39 +00:00
Chauncey 3e2a7e74a5 [Feat][Model] Enable KDA prefill checkpoints for GLM-5.3-Flash (#56960)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-09-29 09:14:30 +00:00
65ed0a2642 [Bugfix][GLM-5.3-Flash] Take video placeholder timestamps from the pixel path's frame sampler (#55647)
Signed-off-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-29 06:45:29 +00:00
Matt MastracciYifan QiaoClaude Opus 5.5Robert Shawmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2617fe9383 [Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding (#58454)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 12:51:34 -04:00
Yifan QiaoandClaude Fable 5.1 39724758e7 [Bugfix] GLM-5.3-Flash: launch the kpool paged MQA logits in the varlen mode its schedule was built with (#55270)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-25 00:13:55 +00:00
lucamotz 276db94fb2 [Bugfix][Model][Spec Decode] Defer disposable GLM MTP head (#55442)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
2026-09-22 20:08:27 +00:00
Jared WenClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c9b34fdb2d [Bugfix][GLM-5.3-Flash] Run the dense MLP layers on the sequence-parallel shard (#58061)
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-22 21:58:21 +08:00
Chauncey bf01fc4a31 [GLM-5.3-Flash] Route kpool indexer top-k through the shared SparseIndexerTopk dispatcher (#57546)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-09-20 11:17:59 +00:00
Jared WenClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
db1bfdd4fb [Bugfix][GLM-5.3-Flash] Address kpool tail blocks by the padded indexer stride in the NVIDIA prefill seed kernel (#57477)
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-20 18:15:04 +08:00
Wentao Ye 36fa72d2d0 [GLM5.3 Perf] Size the GLM-5 sparse indexer decode workspace, 3072 MiB GPU memory saved (#57701)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-19 16:57:01 -04:00
Julien DebacheandHarry Mellor 2bdae2a5a8 [fix] Mistral-Large-3 accuracy regression on main (#57563)
Signed-off-by: jdebache <jdebache@nvidia.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-09-18 14:19:16 +00:00
d12c276853 [Bugfix][ROCm] Alias SparseAttnIndexerKpool.forward_cuda to forward_native (GLM-5.3-Flash boot crash) (#57425)
Signed-off-by: Mustafa YILDIRIM <mustafa@character.ai>
Co-authored-by: Hermes Agent <hermes@nousresearch.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
2026-09-18 03:36:55 +00:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2bbdfcfcfe [Perf][GLM5.3-Flash] Use cooperative top-k for small GLM decode batches (#57327)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-18 10:31:35 +08:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
Jiangyun ZhuandTJian 4fe9e6f6e5 [Refactor][GLM-5.3-Flash] Move sparse_attn_indexer_kpool into the model folder and split AMD/NVIDIA (#55358)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
2026-09-16 05:50:35 +00:00
Colin ZandAndreas Karatzas c8d1cf077a [ROCm] [Bugfix] Enable Load and Inference of GLM-5.3-Flash Quark MXFP4 Checkpoint (#56176)
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-16 01:25:29 +00:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 18f8aa0465 [Refactor] Remove dead kernel code (#56845)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-15 13:17:53 -07:00
Jared WenClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
b443c1cc4e [Perf][GLM-5.3-Flash] Use FlashKDA for KDA chunked prefill (1.7-3.8x faster than the Triton chunk path) (#55737)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-14 08:36:19 +00:00
Roberto L. CastroandCodex e52be1a62d [9/N][warmup][DSv4] Migrate MHC TileLang kernels (#50178)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
2026-09-12 17:25:23 -07:00
aed894c190 [6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-12 17:24:24 -07:00
Jared WenandClaude Fable 5.1 1768273c13 [Perf][GLM-5.3-Flash] Decode hot-path cleanups: strided KDA recurrent inputs, NoPE MQA query without concat, no duplicate router GEMM (#55736)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 16:28:33 +08:00
8369affa54 [Feat] Add EPLB support for GLM-5.3-Flash (#55119)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 05:28:33 +00:00
Harry MellorandClaude Opus 5 78300cdabf [Bugfix][Docs] Package glm5next nvidia subtree and fix its docstrings (#55214)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 21:25:26 -07:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00