702 Commits
Author SHA1 Message Date
b2456c622b [ROCm][CI] Make ROCm 10.0 the default image; keep ROCm 7.2 as -rocm72 (#58761)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-10-17 21:43:59 -05:00
Gabriel WuandClaude Opus 5.5 dcd07967ca [CPU] Keep the hybrid block size across the engine core's config rerun (#61805)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 12:06:03 +08:00
b544587408 [Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
2026-10-16 23:16:47 +00:00
91ed2859c7 [CPU][Zen] Update zentorch to 2.14.0.1 & enable float16 on Venice and later (#61411)
Signed-off-by: Ganesh R <ganesh.r@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-10-15 00:23:50 -07:00
81e1c3f183 [XPU][MLA] Integrate XPU MLA support into the existing FLASH_ATTN_MLA backend (#58297)
Signed-off-by: baodii <di.bao@intel.com>
Signed-off-by: Zhouxiaoya25 <xiaoya.zhou@intel.com>
Signed-off-by: ZhouXiaoya25 <xiaoya.zhou@intel.com>
Co-authored-by: baodii <di.bao@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-10-15 12:47:37 +08:00
rasmithandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2622ee21aa [ROCm][BugFix] Don't use AITER rms_norm as an IR OP on gfx90a when AITER always on. (#59290)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 12:48:37 -07:00
amd-xavierwangandJartX 06265beb25 [ROCm] Enable fp8 query for Triton attention (#60296)
Signed-off-by: xawang <xawang@amd.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
2026-10-14 15:03:20 -04:00
hclandHarry Mellor 2631afd3fe [Bugfix][Platform] Fall back to torch when device-level NVML memory is unsupported (GB10) (#57378)
Signed-off-by: hclsys <chenglunhu@gmail.com>
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-10-14 12:45:29 +00:00
Jiangyun ZhuandClaude Opus 5.5 badfcdbf91 [Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend (#60050)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:49:46 +08:00
Ryan RockandMicah Williamson f43d7e1a0f [CI][AMD] Gate Aiter MHA FA on dtype (#60679)
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
2026-10-13 18:55:19 -05:00
e83222396f [ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches (#57199)
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
2026-10-13 17:21:30 +00:00
31e90ac370 [Attention] Enable FA4 sparse MQA decode on Blackwell (#55866)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-10-13 09:31:37 -04:00
Woosuk KwonandClaude Opus 5.5 e1a5086256 [Attention] Remove cascade attention (#61126)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:41:07 -07:00
e0317b28ce [Feature] Support batch invariance on ROCm (#52231)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-10-12 10:53:26 -05:00
02e55d2af3 [KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-10-09 15:48:37 -04:00
Praneeth Reddy AandLi, Jiang <jiang1.li@intel.com> 1c47ebfc57 [Docs] Fix VLLM_CPU_KVCACHE_SPACE default and units (#60053)
Signed-off-by: Praneeth <praneethreddyarikatla@gmail.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-10-09 19:43:55 +08:00
Yan Ma 4e318ac563 [XPU] fix online fp8_per_channel quantization (#56027)
Signed-off-by: Yan Ma <yan.ma@intel.com>
2026-10-09 18:38:01 +08:00
Woosuk Kwon 194bd72be7 [Compile] Remove torch.compile-based sequence parallelism and AsyncTP (#60517) 2026-10-08 08:28:23 -07:00
wangxiyuanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 072d49bb6e [Platform] Support platform-specific MLA prefill backend selection (#56959)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 15:25:21 +00:00
aditi-amd 385d86a6cf [Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) (#57057)
Signed-off-by: Aditi Rana <aditi.rana@amd.com>
2026-10-06 20:42:52 +00:00
Jee Jee Li e3af5bf3ce [LoRA] Code cleanup (#60017)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-10-05 09:15:53 +00:00
bd42276e1e [Misc][Platform] Check aligned KV block sizes against every attention backend (#58457)
Signed-off-by: Divy <divy@coralbricks.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-10-04 11:13:20 -04:00
cb6458c5ce [Core] Remove deprecated mamba_cache_mode "all" (#58997)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
2026-10-01 02:59:37 -07:00
Fanli LinandKunshang Ji 6a6c2952fa [XPU][CI] Make test_mamba_prefix_cache block-size agnostic (#58269)
Signed-off-by: Lin, Fanli <fanli.lin@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-09-29 07:12:43 +08:00
Shanshan Shen 7ba3df63cb [ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform (#58983)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-09-28 05:46:28 -07:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205)
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
2026-09-25 11:36:59 +01:00
wangxiyuan 44257c2e35 [Feature] Triton kernel dispatcher (#43048)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2026-09-25 04:07:43 +00:00
liuzhenwei dcfc17e0b1 [XPU] enable XPU GRAPH by default (#51600)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
2026-09-24 19:17:25 +08:00
Li, Jiang b44895cf93 [CPU] Use pre-built triton (#58140)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-09-24 16:12:34 +08:00
Tianmu LiOpenAI CodexLi, Jiang <jiang1.li@intel.com>
67ddac3ea8 [CPU][GDN] Support NIXL DS convolution-state layout (#53300)
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-09-24 14:55:20 +08:00
e78e367c6e [ROCm] Give turboquant boundary layers a layout-compatible backend (#54988)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-23 22:00:07 -05:00
Lucas WilkinsonQiuChunshuoClaude Fable 5.1CodexRobert ShawNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e6dc16cebd [PCP] Support prefill context parallelism with data parallelism (#57075)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
DivyandClaude Opus 5 e26547adfb [Bugfix] Pick a KV block size supported by every attention backend (#49845)
Signed-off-by: Divy <divy@coralbricks.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-23 16:36:35 -04:00
c4cfd7007b [CI][ROCm] Add an MI355 Kimi-K3 unit test group (#58012)
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-23 11:25:51 -05:00
Tony LinandFlora Feng e4340e41c9 [Feat][XPU] VLLM_BATCH_INVARIANT support for Dense/MoE models (#55881)
Signed-off-by: Tony Lin <tony.lin@intel.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
2026-09-22 11:06:54 -04:00
Tianmu LiOpenAI CodexLi, Jiang <jiang1.li@intel.com>
23cd0346ce [CPU] Align scheduler and NIXL CPU affinity per local rank (#53636)
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-09-17 16:43:08 +08:00
528fa835fd [ROCm] Stage large pageable H2D copies instead of registering them (#56343)
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-09-17 07:14:43 +00:00
Matt 2d8a5c5741 [ROCm][Bugfix] Reduce CUDA graph divergences (#57229)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
2026-09-17 05:12:13 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
wangxiyuanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 118e17f5ff [Platform] Move env check function to platform interface (#48599)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-16 10:08:36 +00:00
Ganesh RandCursor 82731931ae [Bugfix][CPU] Fall back to CpuPlatform when zentorch fails to import (#54923)
Signed-off-by: R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-16 17:15:12 +08:00
zwang86Zeyu Wangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
b6ce714354 [Bugfix] Format kernel-import errors eagerly so warning_once does not retain them (#54098)
Signed-off-by: Zeyu Wang <zwang86@users.noreply.github.com>
Co-authored-by: Zeyu Wang <zwang86@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-15 16:35:54 -04:00
78e84261ab [Performance] Add Triton kernel for Gemma3n sparse GELU (#48498)
Signed-off-by: BWAAEEEK <jooho414@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-14 07:42:15 +00:00
82a85dc1d2 [Attention] Add Triton/FlashInfer composite for multimodal prefix attention (#56305)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-13 14:29:41 +00:00
Andreas KaratzasandOpenAI Codex 9dd969da09 [Model][ROCm] Enable DeepSeek V4 Vision (#55107)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-11 09:17:30 -07:00
Wentao Ye 5fe77aecfc [Deprecation] Deprecate items scheduled for 0.29 (#55353)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-11 08:17:02 -07:00
d0dfe587d5 [Bugfix] Fall back to full decode graphs for noncompiled models (#55095)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
2026-09-11 01:56:05 -07:00
Giuseppe Grossi 6ff479e1f7 [ROCm] Add better kv dtype error discoverability (#55236)
Signed-off-by: Giuseppe Grossi <ggrossi@amd.com>
2026-09-10 09:17:41 +00:00
Li, Jiang c3ccc0e957 [Model] Add DeepSeek-V4 CPU backend (#55355)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-09-10 14:14:48 +08:00