hcl and Harry Mellor
2631afd3fe
[Bugfix][Platform] Fall back to torch when device-level NVML memory is unsupported (GB10) ( #57378 )
...
Signed-off-by: hclsys <chenglunhu@gmail.com >
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-14 12:45:29 +00:00
31e90ac370
[Attention] Enable FA4 sparse MQA decode on Blackwell ( #55866 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-13 09:31:37 -04:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
Mark McLoughlin and AI Assistant
45d56c0baf
[Core] Model console logging as CLI configuration ( #57205 )
...
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: AI Assistant <noreply@openai.com >
2026-09-25 11:36:59 +01:00
e6dc16cebd
[PCP] Support prefill context parallelism with data parallelism ( #57075 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
b6ce714354
[Bugfix] Format kernel-import errors eagerly so warning_once does not retain them ( #54098 )
...
Signed-off-by: Zeyu Wang <zwang86@users.noreply.github.com >
Co-authored-by: Zeyu Wang <zwang86@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-15 16:35:54 -04:00
78e84261ab
[Performance] Add Triton kernel for Gemma3n sparse GELU ( #48498 )
...
Signed-off-by: BWAAEEEK <jooho414@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-14 07:42:15 +00:00
82a85dc1d2
[Attention] Add Triton/FlashInfer composite for multimodal prefix attention ( #56305 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-13 14:29:41 +00:00
Qiu Chunshuo
7c2f1ff495
[Core] Scope PCP-DP validation to GPU manager ( #54523 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
2026-09-08 08:18:01 -07:00
Jiangyun Zhu
98ed0856f3
[Model] add GLM-5.3-Flash support ( #53906 )
2026-09-03 09:40:35 -07:00
Eoin-Houstoun and Claude Fable 5
30e2394c83
[Bugfix] Record non-ImportError attention backend probe failures instead of crashing engine init ( #51703 )
...
Signed-off-by: Eoin <eoin@turintech.ai >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-20 02:36:00 -07:00
wangxiyuan
73b83949f7
[Platform] Add check_runner_kv_caches_multi_layer ( #51633 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-08-13 19:25:07 +00:00
Sebastian Woo and Claude
4f76c8ad9d
[Bugfix][Platform] Stop re-initializing NVML on every device-capability check ( fixes #50381 ) ( #50393 )
...
Signed-off-by: seewoo <seewoo@ucsc.edu >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-07 08:30:00 +00:00
Alejandro Paredes La Torre and Shengqi Chen
0a684ab0c0
[Bugfix] Fix WSL circular import from pin_memory warning_once ( #48444 )
...
Signed-off-by: AlejandroParedesLT <alejandroparedeslatorre@gmail.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-07-20 18:30:55 -07:00
Michael Goin and Claude Fable 5
ecf4aa5ce2
[Bugfix] Fix FlashInfer non-causal draft attention (DFlash/DSpark) on Blackwell ( #48167 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-15 12:44:01 -07:00
Tyler Michael Smith
68b4a1d582
Fix NVML capability lookup for visible devices ( #47892 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-07-08 09:07:44 -04:00
Chris Leonard
fbc9ba6d30
New stable abi cleanup ( #46656 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
2026-07-03 14:02:26 +08:00
Wentao Ye and TJian
007b5a52ed
[Log] Update to log once ( #46511 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-06-24 14:45:16 +00:00
Dakai An
dc0d318177
[Attention] Add FLASH_ATTN_MLA_SPARSE backend for Hopper sparse MLA ( #46189 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
2026-06-24 09:33:10 +00:00
44d95069e9
Enable DeepSeek V4 and GLM-5.1 on SM120 ( #43477 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-06-22 11:54:14 -07:00
Nick Hill
7df3d7dada
[Core] Ensure memory is pinned prior to async h2d copy ( #45424 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-20 20:02:24 -07:00
ebfbcfe46a
Stop setting CUDA_VISIBLE_DEVICES internally in vLLM, add device_ids arg ( #45026 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: kourosh hakhamaneshi <kouroshHakha@users.noreply.github.com >
2026-06-20 13:38:10 -07:00
Chris Leonard
b9a7cd464c
[12/n] final _C library kernel migration ( #45415 )
2026-06-19 06:57:26 -07:00
Jimmy Lee
8bf374955f
[Bug Fix] Allow pinned memory for WSL2 ( #41496 )
...
Signed-off-by: Jimmy Lee <hirejimmylee@gmail.com >
2026-06-16 05:56:26 +00:00
Raj Joshi and Cursor
9090368b65
[Feat] Add support for per GPU worker RDMA NIC selection ( #42083 )
...
Signed-off-by: Raj Joshi <rajjoshi@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-05-28 12:45:23 -07:00
TJian
0ba46d4b11
[ROCm][DSV4] Enable Tilelang MHC replacing torch/triton mhc ( #43679 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-05-28 07:05:28 +00:00
7e33081cee
[Attention] Make FlexAttention and FlashAttention use num-blocks first layouts ( #42095 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-05-26 19:55:56 -07:00
Artem Perevedentsev
1cb224430b
[GDN] Enable FI Blackwell GDN prefill kernel ( #40717 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-05-20 01:46:55 -07:00
Yongye Zhu and Roger Wang
0d2732dd91
[MLA Attention Backend] Add TOKENSPEED_MLA backend for DSR1/Kimi K25 prefill + decode on Blackwell ( #41778 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-05-13 23:48:02 -07:00
Luka Govedič
d58c42e19c
[vLLM IR] 2/N fused_add_rms_norm and maybe_inplace overload ( #36823 )
...
Signed-off-by: Luka Govedič <lgovedic@redhat.com >
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2026-05-01 23:41:15 -04:00
Wentao Ye and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
8f87eb4622
[Refactor] Clean up log once scope="local" ( #40540 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-22 16:42:43 -04:00
Michael Goin
1174723eba
Fix TURBOQUANT backend selection in cuda.py ( #40060 )
...
Signed-off-by: Michael Goin <mgoin64@gmail.com >
2026-04-17 07:31:41 -07:00
f4b42df048
[Attention Backend] TurboQuant: 2-bit KV cache compression with 4x capacity ( #38479 )
...
Signed-off-by: vibhavagarwal5 <vibhavagarwal5@gmail.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-04-14 19:57:13 -07:00
haosdent and DarkLight1337
995e9a209e
[Bugfix] Use is_integrated to detect UMA GPUs for memory reporting ( #35356 )
...
Signed-off-by: haosdent <haosdent@gmail.com >
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-04-13 11:07:40 -07:00
Yan Ma
ec68d53b2b
Add platform manual_seed_all API ( #38468 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-04-10 13:43:50 +08:00
Qidong Su and Claude Opus 4.6
ed733802f0
Fix NUMA binding on non-CDMM Grace-Blackwell systems ( #39361 )
...
Signed-off-by: Qidong Su <soodoshll@gmail.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-09 07:36:51 +00:00
75e01a39a1
[Feature] NUMA binding support for GPU workers ( #38635 )
...
Signed-off-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: Jason Li <jasonlizhengjian@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-04-08 09:55:24 -07:00
Matthew Bonanni
116f4be405
[1/N][Cleanup] Standardize on use of is_quantized_kv_cache ( #38659 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-04-01 04:08:01 +00:00
40bb175027
[vLLM IR] 1/N Implement IR skeleton and rms_norm op ( #33825 )
...
Signed-off-by: Luka Govedič <lgovedic@redhat.com >
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com >
Signed-off-by: chzhang <chaojun.zhang@intel.com >
Signed-off-by: Luka Govedic <luka.govedic@gmail.com >
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com >
Co-authored-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Luka Govedič <ProExpertProg@h100-01.nemg-001.lab.rdu2.dc.redhat.com >
2026-03-31 22:15:05 -04:00
4dfad17ed1
replace cuda_device_count_stateless() to current_platform.device_count() ( #37841 )
...
Signed-off-by: Liao, Wei <wei.liao@intel.com >
Signed-off-by: wliao2 <wei.liao@intel.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-03-31 22:32:54 +08:00
SherryC41 and sherryC41
6ec5e9fd37
refactor: abstract deepgemm support into platform ( #37519 )
...
Co-authored-by: sherryC41 <sherry.c.c41@gmail.com >
2026-03-20 17:54:08 +00:00
mikaylagawarecki
8b10e4fb31
[1/n] Migrate permute_cols to libtorch stable ABI ( #31509 )
...
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
2026-03-19 11:27:26 -04:00
Wentao Ye
c373b5c00d
[Log] Reduce duplicate log ( #37313 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-03-18 10:57:44 -04:00
Wei Zhao
b36adfa349
[Perf] Set Flashinfer sparse MLA as default backend for FP8 kv cache ( #37252 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-03-17 20:09:20 +00:00
Matthew Bonanni
77a73458e3
Reapply [Attention] Refactor check_and_update_config ( #35122 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-03-09 07:17:14 -07:00
Isotr0py
b0906d8b02
[MM Encoder] Default to use TORCH_SDPA backend for ViT on Volta/Turing GPU ( #36472 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-03-09 03:43:44 -07:00
nkm-meta
792cbd64ca
Add platform method to enable custom collective ops registration ( #34760 )
...
Signed-off-by: Naina Kuruballi Mahesh <nainakm@meta.com >
2026-03-05 00:50:32 +00:00
dea268336f
[1/N] Elastic EP Milestone 2 ( #34861 )
...
Signed-off-by: Yongji Wu <wuyongji317@gmail.com >
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Signed-off-by: Ron Tourgeman <rtourgeman@nvidia.com >
Co-authored-by: Yongji Wu <wuyongji317@gmail.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com >
2026-02-28 04:46:42 +00:00