MOVZX and Hinata
5f3c7bd8ab
[Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
...
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
the live server process maps none of them
- Delete tests that only build architectures this fork no longer
registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
import at module level
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 19:31:07 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
e711723352
[ROCm] gfx1250 optimizations and model enablement ( #58800 )
...
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: Juan Villamizar <jpvillam@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
2026-10-15 14:51:57 -07:00
Kai and Kunshang Ji
de2b869ee9
[Bugfix][Kernel] Register int8 scaled_mm as a custom op ( #57331 )
...
Signed-off-by: Kai <limkc.reply@gmail.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-15 09:27:49 +08:00
fe98d90a9d
[Dependency] Update Humming and remove NVFP4 scale workaround ( #61556 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 13:32:36 -07:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
e0317b28ce
[Feature] Support batch invariance on ROCm ( #52231 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-12 10:53:26 -05:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
7d99c2c4fd
[Feature][Humming] Humming feature integration ( #56685 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-20 08:11:01 -07:00
Misha Goin and ErinYin
e5fce7b56b
[Core][Kernel] Share persistent workspaces for Marlin and Humming ( #57421 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: ErinYin <liang.yin@daocloud.io >
2026-09-20 17:28:33 +08:00
Yuwen Zhou and Li, Jiang <jiang1.li@intel.com>
b346479065
[CPU] Add CPU FP8 W8A8 linear/MoE support ( #49942 )
...
Signed-off-by: yuwenzho <yuwen.zhou@intel.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-18 17:59:42 +08:00
danisereb
2bdbbc8080
[BugFix] Fix is_supported of cutlass FP8 linear (selected and fails on A100) ( #55884 )
...
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com >
2026-09-16 12:31:43 -07:00
Shanshan Shen
bc0f47cd03
[ROCm][Bugfix] Revert #56433 + #51692 to fix accuracy breakdown for DeepSeek-V4 ( #57132 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-16 18:58:23 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Rebecca Lee and Cursor
6ca2b23e22
[ROCm] Expose kFp8DynamicTokenSym on AITER PTPC linears ( #54248 )
...
Signed-off-by: Rebecca Lee <rebecca.lee@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-16 11:48:30 -05:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Li, Jiang
b68408043d
[Bugfix][CPU] Add per-tensor FP8 W8A16 kernel to fix Ministral crash on CPU ( #56985 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-16 15:49:17 +08:00
Li, Jiang
79f0be21ff
[Bugfix][CPU] Fix DeepSeek-R1 (FP8 MLA + MoE) correctness on CPU backend ( #56773 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-14 17:30:28 +08:00
Ayushman Singh and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7bcca16729
[Bugfix] Fix DeepGEMM FP8 warmup coverage ( #56452 )
...
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-12 14:20:38 -04:00
Matt
b4da4d17ae
[ROCm][Bugfix] Fix AITER preshuffled FP8 block-scale kernel ( #56433 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-09-11 11:44:57 +00:00
Simon Danielsson
7470082f57
[ROCm][Perf] Add bpreshuffled blockscaled fp8 GEMM ( #51692 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-10 16:18:37 +00:00
Li, Jiang
c3ccc0e957
[Model] Add DeepSeek-V4 CPU backend ( #55355 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-10 14:14:48 +08:00
Jee Jee Li
199cb9b964
[Kernel] Remove unused fake implementation ( #55535 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-09-06 17:54:32 -07:00
b7624069ff
[Fusion] Manual ActivationQuantFusionPass initial application ( #51415 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: ElizaWszola <ewszola@redhat.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: ElizaWszola <ewszola@redhat.com >
2026-09-03 13:37:31 -04:00
Misha Goin and OpenAI Codex
541c6d64c1
[Bugfix][Quantization] Support CT block FP8 with Marlin ( #52966 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-19 12:41:26 -07:00
Strahinja Stamenkovic and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ddbf826bee
[ROCm] Gate Torch FP8 scaled-MM on architecture support ( #51021 )
...
Signed-off-by: sstamenk <strahinja.stamenkovic@amd.com >
Signed-off-by: Strahinja Stamenkovic <strahinja.stamenkovic@amd.com >
Signed-off-by: Strahinja Stamenkovic <sstamenk@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-18 17:54:51 +00:00
Misha Goin and OpenAI Codex
bca7bea240
Remove VLLM_TEST_FORCE_FP8_MARLIN to replace with linear_backend/moe_backend ( #52182 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-18 12:58:04 -04:00
Misha Goin
4ab5e5012a
[Refactor] Simplify B12X linear kernels and warmup ( #52368 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-17 09:45:45 -07:00
Luke Alonso
3c79b1a8bf
[Kernel] Add B12X dense linear backends ( #52016 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
2026-08-13 19:48:33 -07:00
zofia
fe4c5dcd4c
[XPU] [Bugfix] process ragged weights in xpu linear backend ( #52118 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
2026-08-14 09:11:43 +08:00
Lena Onyshchenko and Kunshang Ji
751f2ccdd3
[Kernel][XPU] Tensor-descriptor operand loads for Triton W8A8 scaled_mm ( #47205 )
...
Signed-off-by: oonyshch <xonyshch@gmail.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-10 00:56:11 +00:00
Jinzhen Lin and OpenAI Codex
fcde8e1460
[Refactor] refactor humming linear and moe backends to use explicit layer configs ( #49610 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-07 10:03:19 -07:00
zofia and Kunshang Ji
8060260e1d
[Linear] [Kernel] add block-wise scaled_mm ( #49932 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 14:40:26 +08:00
skysnow2001 and Douglas Lehr
f43e1d26e3
[ROCm] Enable AITER and FP8 inference on GFX120x ( #43615 )
...
Signed-off-by: skysnow2001 <skysnow9285@gmail.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-08-03 16:45:37 -05:00
Li, Jiang
c8602c7906
[CPU] Refine CPU kernel dispatch ( #50801 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-08-03 17:42:53 +08:00
zofia and Kunshang Ji
e481da9508
[XPU] [Linear] add torch as xpu linear backend ( #49664 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-03 14:30:21 +08:00
Ma Jian and Kunshang Ji
ef0d084b4b
[XPU] Fix FP8 block scale layout for MLA compatibility ( #50349 )
...
Signed-off-by: Ma Jian <jian1.ma@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-31 11:05:35 +08:00
+1
4f1da84eb5
Enable gfx1250 ROCm architecture ( #46516 )
...
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com >
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: dllehr <douglehr@amd.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-31 01:41:12 +00:00
jiacao-amd and TJian
4f56321d7e
[ROCm] Cache fp32 upcast of static e8m0 weight scale in AITER scaled_mm ( #47773 )
...
Signed-off-by: jiacao-amd <jiahui.cao@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-07-28 11:34:16 -05:00
823eaf667d
[XPU] FP8 o_proj with fp8_bmm and load-time scale transpose ( #48334 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-20 16:32:03 +08:00
Jinzhen Lin and mgoin
d891b9bd51
[Quantization] add humming moe backend to all dense/moe oracles ( #41652 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-07-06 13:36:07 -07:00
Ma Jian and Kunshang Ji
26c754d847
[XPU][Bugfix] Do not transpose weight_scale_inv at load time ( #47116 )
...
Signed-off-by: Ma Jian <jian1.ma@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-06 17:15:26 +08:00
Chaojun Zhang and Copilot
1a308c449c
[XPU] Add W8A8 FP8 linear kernel with multi-granularity quant support ( #43645 )
...
Signed-off-by: Chaojun,Zhang <chaojun.zhang@intel.com >
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-07-04 18:10:01 +08:00
2785a5e0e6
[Bugfix][ROCm] Fix FP8 per-tensor scale rank mismatch causing Inductor assertion failure ( #44912 )
...
Signed-off-by: nehmathe2 <nehmathe2@gmail.com >
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Signed-off-by: nehmathe <nehmathe@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-06-16 20:17:42 -07:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
71b961dd35
[Perf] SM90 cutlass fp8 mm supports odd M by swap_ab, 180~290% kernel performance improvement ( #44572 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-13 12:05:45 -07:00
Michael Goin and Claude
c90650088d
Add the QuantizedActivation linear-kernel contract ( #44260 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-06-12 13:48:15 -07:00
Michael Goin
6afa25000c
[Bugfix] Canonicalize FP8 weight layout to (K, N) at the source ( #44735 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-06-08 14:37:36 -06:00
Ma Jian
eebce65756
[XPU]feat: add DeepSeek-V4 XPU attention decode path ( #42953 )
...
Signed-off-by: Ma Jian <jian1.ma@intel.com >
2026-06-08 13:27:12 +08:00
165b7864d0
[ROCM] [FEAT] Integrate Aiter hipBLASLt GEMM online tuning ( #40426 )
...
Signed-off-by: hanlin12 <hanlin12@amd.com >
Signed-off-by: Han Lin <hanlin12@amd.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-06-04 23:45:36 -07:00
e5232679a3
[XPU] Add XPU block-scaled W8A8 fp8 path ( #39968 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com >
Co-authored-by: Yuxiang <yuxiang.liang@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-06-03 20:16:19 +08:00