MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
MOVZX and Hinata
5f3c7bd8ab
[Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
...
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
the live server process maps none of them
- Delete tests that only build architectures this fork no longer
registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
import at module level
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 19:31:07 +07:00
MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
b2456c622b
[ROCm][CI] Make ROCm 10.0 the default image; keep ROCm 7.2 as -rocm72 ( #58761 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-17 21:43:59 -05:00
rasmith and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2622ee21aa
[ROCm][BugFix] Don't use AITER rms_norm as an IR OP on gfx90a when AITER always on. ( #59290 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 12:48:37 -07:00
amd-xavierwang and JartX
06265beb25
[ROCm] Enable fp8 query for Triton attention ( #60296 )
...
Signed-off-by: xawang <xawang@amd.com >
Co-authored-by: JartX <sagformas@epdcenter.es >
2026-10-14 15:03:20 -04:00
Ryan Rock and Micah Williamson
f43d7e1a0f
[CI][AMD] Gate Aiter MHA FA on dtype ( #60679 )
...
Signed-off-by: Ryan Rock <ryan.rock@amd.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
2026-10-13 18:55:19 -05:00
e83222396f
[ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches ( #57199 )
...
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-13 17:21:30 +00:00
e0317b28ce
[Feature] Support batch invariance on ROCm ( #52231 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-12 10:53:26 -05:00
wangxiyuan and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
072d49bb6e
[Platform] Support platform-specific MLA prefill backend selection ( #56959 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 15:25:21 +00:00
aditi-amd
385d86a6cf
[Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) ( #57057 )
...
Signed-off-by: Aditi Rana <aditi.rana@amd.com >
2026-10-06 20:42:52 +00:00
Shanshan Shen
7ba3df63cb
[ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform ( #58983 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-28 05:46:28 -07:00
e78e367c6e
[ROCm] Give turboquant boundary layers a layout-compatible backend ( #54988 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-23 22:00:07 -05:00
e6dc16cebd
[PCP] Support prefill context parallelism with data parallelism ( #57075 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
c4cfd7007b
[CI][ROCm] Add an MI355 Kimi-K3 unit test group ( #58012 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 11:25:51 -05:00
528fa835fd
[ROCm] Stage large pageable H2D copies instead of registering them ( #56343 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-17 07:14:43 +00:00
Matt
2d8a5c5741
[ROCm][Bugfix] Reduce CUDA graph divergences ( #57229 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-09-17 05:12:13 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
78e84261ab
[Performance] Add Triton kernel for Gemma3n sparse GELU ( #48498 )
...
Signed-off-by: BWAAEEEK <jooho414@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-14 07:42:15 +00:00
Andreas Karatzas and OpenAI Codex
9dd969da09
[Model][ROCm] Enable DeepSeek V4 Vision ( #55107 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-11 09:17:30 -07:00
Wentao Ye
5fe77aecfc
[Deprecation] Deprecate items scheduled for 0.29 ( #55353 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-11 08:17:02 -07:00
d0dfe587d5
[Bugfix] Fall back to full decode graphs for noncompiled models ( #55095 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-09-11 01:56:05 -07:00
Giuseppe Grossi
6ff479e1f7
[ROCm] Add better kv dtype error discoverability ( #55236 )
...
Signed-off-by: Giuseppe Grossi <ggrossi@amd.com >
2026-09-10 09:17:41 +00:00
Qiu Chunshuo
7c2f1ff495
[Core] Scope PCP-DP validation to GPU manager ( #54523 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
2026-09-08 08:18:01 -07:00
dbb7fffddb
[ROCm][MLA][DCP] Support causal multi-token verification ( #51705 )
...
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com >
Signed-off-by: billishyahao <bill.he@amd.com >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: billishyahao <bill.he@amd.com >
2026-08-31 10:42:28 -07:00
wangxiyuan
73b83949f7
[Platform] Add check_runner_kv_caches_multi_layer ( #51633 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-08-13 19:25:07 +00:00
PikaPikachu
466855a2bf
[ROCm] Enable DeepSeek-V4 on gfx11 ( #47017 )
...
This PR enables DeepSeek-V4 checkpoints on ROCm gfx11/RDNA devices.
Signed-off-by: Letian Kang <Letian.Kang@amd.com >
2026-08-12 08:55:10 +08:00
5426311d91
[Kernel][ROCm][Perf] FlyDSL decode-attention kernel for 4-bit TurboQuant KV cache ( #47896 )
...
Signed-off-by: root <root@smci355-ccs-aus-m02-09.cs-aus.dcgpu >
Co-authored-by: root <root@smci355-ccs-aus-m02-09.cs-aus.dcgpu >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 08:45:39 -07:00
Isotr0py
073c510c91
[Migration] Migrate bitsandbytes support to OOT plugin ( #43529 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-08-09 10:27:31 +08:00
Flora Cui
7e85d3a42c
[ROCm] Enable pinned memory on supported WSL2 kernels ( #50126 )
...
Signed-off-by: Flora Cui <flora.cui@amd.com >
2026-08-07 10:43:42 -05:00
skysnow2001 and Douglas Lehr
f43e1d26e3
[ROCm] Enable AITER and FP8 inference on GFX120x ( #43615 )
...
Signed-off-by: skysnow2001 <skysnow9285@gmail.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-08-03 16:45:37 -05:00
Adrian and Claude
4fdd3e0b74
[ROCm][ViT] Detect Triton-AMD kernels at their new aiter location ( #40289 )
...
Signed-off-by: Adrian Kurt <info@zzit.ch >
Signed-off-by: Adrian <info@zipfuzapfu.ch >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-31 15:30:09 -04:00
+1
4f1da84eb5
Enable gfx1250 ROCm architecture ( #46516 )
...
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com >
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: dllehr <douglehr@amd.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-31 01:41:12 +00:00
yinfengLiu
f1e8fd27ae
[ROCm] Add AITER FP8 ViT encoder attention ( #49937 )
...
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
2026-07-30 06:22:10 +00:00
51878e5b6e
[2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends ( #44455 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-07-11 11:11:16 -04:00
Micah Williamson and Andreas Karatzas
6e3a983cf3
[ROCm] Remove erroneous inclusion of gptq_marlin as supported quant scheme on ROCm ( #46655 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-06-24 21:27:19 -05:00
Micah Williamson
d6696e2385
[ROCm] Begin Deprecation Window for CUDA_VISIBLE_DEVICES on ROCm ( #46636 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-06-24 20:40:28 +00:00
stefankoncarevic and Andreas Karatzas
2b4a7491ec
[ROCm][CI] Query total device memory via amdsmi to avoid HIP init ( #46141 )
...
Signed-off-by: stefankoncarevic <stefan.koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-06-22 15:12:24 -05:00
8d4f54966c
fix(quantization): Fix AWQ dequantize on Intel XPU and refactor AutoAWQ config ( #42727 )
...
Signed-off-by: Alex <alex.tech.lab@outlook.com >
Signed-off-by: AlexHuang <jihuihuang@tencent.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-18 20:12:28 +08:00
Hongxia Yang and tjtanaa
e28e8c8782
[ROCm][Quant] Minimax-M3: Enable fp8_per_channel for bf16 weights on mi300x ( #45854 )
...
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-06-17 12:02:40 +00:00
Isotr0py
6635279d8a
[Migration] Migrate GGUF quantization support to plugin ( #39612 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-06-12 12:02:21 -07:00
165b7864d0
[ROCM] [FEAT] Integrate Aiter hipBLASLt GEMM online tuning ( #40426 )
...
Signed-off-by: hanlin12 <hanlin12@amd.com >
Signed-off-by: Han Lin <hanlin12@amd.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-06-04 23:45:36 -07:00
JartX
0cff0741ff
[Kernel][ROCm] Native W4A16 kernel for AMD RDNA3 (gfx1100) — fp16 + bf16 ( #41394 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
2026-05-29 11:04:40 +00:00
Nicolò Lucchesi
5b115bb8a3
[Attention][AMD] Standardize kv layout to blocks first for AMD ( #43660 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-05-28 12:28:50 -05:00
7e33081cee
[Attention] Make FlexAttention and FlashAttention use num-blocks first layouts ( #42095 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-05-26 19:55:56 -07:00
07aeaf9d4d
[6/n] Migrate activation kernels, gptq, gguf, non cutlass w8a8 to libtorch stable ABI (continued) ( #42663 )
...
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-05-20 00:18:12 -07:00
fa2a33b893
[Quant] Consolidate GPTQ: rename gptq_marlin.py to auto_gptq.py ( #38288 )
...
Signed-off-by: Chengyi Nie <cnie@roblox.com >
Co-authored-by: Chengyi Nie <cnie@roblox.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-05-15 08:25:52 +08:00
628c436301
[New Model][ROCm] Add AMD support for DeepSeek V4 ( #40871 )
...
Signed-off-by: ganyi <ygan@amd.com >
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Signed-off-by: tjtanaavllm <tunjian.tan@amd.com >
Co-authored-by: ganyi <ygan@amd.com >
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: tjtanaavllm <tunjian.tan@amd.com >
2026-05-05 08:55:37 -07:00
Luka Govedič
d58c42e19c
[vLLM IR] 2/N fused_add_rms_norm and maybe_inplace overload ( #36823 )
...
Signed-off-by: Luka Govedič <lgovedic@redhat.com >
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2026-05-01 23:41:15 -04:00