e83222396f
[ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches ( #57199 )
...
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-13 17:21:30 +00:00
31e90ac370
[Attention] Enable FA4 sparse MQA decode on Blackwell ( #55866 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-13 09:31:37 -04:00
Woosuk Kwon and Claude Opus 5.5
e1a5086256
[Attention] Remove cascade attention ( #61126 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:41:07 -07:00
e0317b28ce
[Feature] Support batch invariance on ROCm ( #52231 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-12 10:53:26 -05:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
Praneeth Reddy A and Li, Jiang <jiang1.li@intel.com>
1c47ebfc57
[Docs] Fix VLLM_CPU_KVCACHE_SPACE default and units ( #60053 )
...
Signed-off-by: Praneeth <praneethreddyarikatla@gmail.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-10-09 19:43:55 +08:00
Yan Ma
4e318ac563
[XPU] fix online fp8_per_channel quantization ( #56027 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-09 18:38:01 +08:00
Woosuk Kwon
194bd72be7
[Compile] Remove torch.compile-based sequence parallelism and AsyncTP ( #60517 )
2026-10-08 08:28:23 -07:00
wangxiyuan and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
072d49bb6e
[Platform] Support platform-specific MLA prefill backend selection ( #56959 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 15:25:21 +00:00
aditi-amd
385d86a6cf
[Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) ( #57057 )
...
Signed-off-by: Aditi Rana <aditi.rana@amd.com >
2026-10-06 20:42:52 +00:00
Jee Jee Li
e3af5bf3ce
[LoRA] Code cleanup ( #60017 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-05 09:15:53 +00:00
bd42276e1e
[Misc][Platform] Check aligned KV block sizes against every attention backend ( #58457 )
...
Signed-off-by: Divy <divy@coralbricks.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-04 11:13:20 -04:00
cb6458c5ce
[Core] Remove deprecated mamba_cache_mode "all" ( #58997 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
2026-10-01 02:59:37 -07:00
Fanli Lin and Kunshang Ji
6a6c2952fa
[XPU][CI] Make test_mamba_prefix_cache block-size agnostic ( #58269 )
...
Signed-off-by: Lin, Fanli <fanli.lin@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-29 07:12:43 +08:00
Shanshan Shen
7ba3df63cb
[ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform ( #58983 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-28 05:46:28 -07:00
Mark McLoughlin and AI Assistant
45d56c0baf
[Core] Model console logging as CLI configuration ( #57205 )
...
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: AI Assistant <noreply@openai.com >
2026-09-25 11:36:59 +01:00
wangxiyuan
44257c2e35
[Feature] Triton kernel dispatcher ( #43048 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-09-25 04:07:43 +00:00
liuzhenwei
dcfc17e0b1
[XPU] enable XPU GRAPH by default ( #51600 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-09-24 19:17:25 +08:00
Li, Jiang
b44895cf93
[CPU] Use pre-built triton ( #58140 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-24 16:12:34 +08:00
67ddac3ea8
[CPU][GDN] Support NIXL DS convolution-state layout ( #53300 )
...
Signed-off-by: Li, Tianmu <tianmu.li@intel.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-24 14:55:20 +08:00
e78e367c6e
[ROCm] Give turboquant boundary layers a layout-compatible backend ( #54988 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-23 22:00:07 -05:00
e6dc16cebd
[PCP] Support prefill context parallelism with data parallelism ( #57075 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
Divy and Claude Opus 5
e26547adfb
[Bugfix] Pick a KV block size supported by every attention backend ( #49845 )
...
Signed-off-by: Divy <divy@coralbricks.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-23 16:36:35 -04:00
c4cfd7007b
[CI][ROCm] Add an MI355 Kimi-K3 unit test group ( #58012 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 11:25:51 -05:00
Tony Lin and Flora Feng
e4340e41c9
[Feat][XPU] VLLM_BATCH_INVARIANT support for Dense/MoE models ( #55881 )
...
Signed-off-by: Tony Lin <tony.lin@intel.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-22 11:06:54 -04:00
23cd0346ce
[CPU] Align scheduler and NIXL CPU affinity per local rank ( #53636 )
...
Signed-off-by: Li, Tianmu <tianmu.li@intel.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-17 16:43:08 +08:00
528fa835fd
[ROCm] Stage large pageable H2D copies instead of registering them ( #56343 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-17 07:14:43 +00:00
Matt
2d8a5c5741
[ROCm][Bugfix] Reduce CUDA graph divergences ( #57229 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-09-17 05:12:13 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
wangxiyuan and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
118e17f5ff
[Platform] Move env check function to platform interface ( #48599 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-16 10:08:36 +00:00
Ganesh R and Cursor
82731931ae
[Bugfix][CPU] Fall back to CpuPlatform when zentorch fails to import ( #54923 )
...
Signed-off-by: R <Ganesh.R@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-16 17:15:12 +08:00
b6ce714354
[Bugfix] Format kernel-import errors eagerly so warning_once does not retain them ( #54098 )
...
Signed-off-by: Zeyu Wang <zwang86@users.noreply.github.com >
Co-authored-by: Zeyu Wang <zwang86@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-15 16:35:54 -04:00
78e84261ab
[Performance] Add Triton kernel for Gemma3n sparse GELU ( #48498 )
...
Signed-off-by: BWAAEEEK <jooho414@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-14 07:42:15 +00:00
82a85dc1d2
[Attention] Add Triton/FlashInfer composite for multimodal prefix attention ( #56305 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-13 14:29:41 +00:00
Andreas Karatzas and OpenAI Codex
9dd969da09
[Model][ROCm] Enable DeepSeek V4 Vision ( #55107 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-11 09:17:30 -07:00
Wentao Ye
5fe77aecfc
[Deprecation] Deprecate items scheduled for 0.29 ( #55353 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-11 08:17:02 -07:00
d0dfe587d5
[Bugfix] Fall back to full decode graphs for noncompiled models ( #55095 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-09-11 01:56:05 -07:00
Giuseppe Grossi
6ff479e1f7
[ROCm] Add better kv dtype error discoverability ( #55236 )
...
Signed-off-by: Giuseppe Grossi <ggrossi@amd.com >
2026-09-10 09:17:41 +00:00
Li, Jiang
c3ccc0e957
[Model] Add DeepSeek-V4 CPU backend ( #55355 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-10 14:14:48 +08:00
Qiu Chunshuo
7c2f1ff495
[Core] Scope PCP-DP validation to GPU manager ( #54523 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
2026-09-08 08:18:01 -07:00
Artur Fierka
5e6f6a8ed4
[Bugfix][Quantization][XPU] Fix moe_wna16 linear weight loading ( #52651 )
...
Signed-off-by: Artur Fierka <artur.fierka@intel.com >
2026-09-07 20:20:17 +08:00
Jiangyun Zhu
98ed0856f3
[Model] add GLM-5.3-Flash support ( #53906 )
2026-09-03 09:40:35 -07:00
Luyi Xiao and OpenAI Codex
aa71f9bcc4
[Bugfix] Log platform plugin detection failures ( #52285 )
...
Signed-off-by: Luyi Xiao <xiao.luy@northeastern.edu >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-01 19:11:52 +00:00
dbb7fffddb
[ROCm][MLA][DCP] Support causal multi-token verification ( #51705 )
...
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com >
Signed-off-by: billishyahao <bill.he@amd.com >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: billishyahao <bill.he@amd.com >
2026-08-31 10:42:28 -07:00
Tianmu Li and OpenAI Codex
c6c33f2b1f
[CPU] Support FP16/BF16 persisted GDN state on AMX ( #52191 )
...
Signed-off-by: Li, Tianmu <tianmu.li@intel.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-31 16:32:20 +08:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
Yizhou
1ebff996a1
[Fix] Improve ROCm detection in WSL environments ( #38434 )
...
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com >
2026-08-30 04:23:10 +00:00
Eoin-Houstoun and Claude Fable 5
30e2394c83
[Bugfix] Record non-ImportError attention backend probe failures instead of crashing engine init ( #51703 )
...
Signed-off-by: Eoin <eoin@turintech.ai >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-20 02:36:00 -07:00
Roger Wang
755492e37d
Revert "[Kernel] Gemma-4 FA4 FP8 Kernel" ( #52987 )
2026-08-19 10:48:46 -07:00
