MOVZX and Hinata
5f3c7bd8ab
[Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
...
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
the live server process maps none of them
- Delete tests that only build architectures this fork no longer
registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
import at module level
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 19:31:07 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
fe98d90a9d
[Dependency] Update Humming and remove NVFP4 scale workaround ( #61556 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 13:32:36 -07:00
zofia
4dc5add6ea
[XPU] [Linear] [MXFP8] enable torch as mxfp8 linear backend ( #51933 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com >
2026-10-14 00:48:38 -07:00
e0317b28ce
[Feature] Support batch invariance on ROCm ( #52231 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-12 10:53:26 -05:00
Mehmet Cagri
08e03df940
[Bugfix][ROCm] Use a zero default for masked scales in the MXFP8 GEMM ( #59454 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
2026-10-01 11:32:25 -05:00
e440b75bb6
[ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision ( #50605 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: jpvillam <juan.villamizar@amd.com >
2026-09-25 14:55:52 -05:00
Lingpeng Jin and Andreas Karatzas
35f92d2abc
[ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-launch split-K ( #58659 )
...
Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-24 21:37:41 -07:00
Samuel Kim and mgoin
f55419c43c
[Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits ( #54223 )
...
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-25 02:54:11 +00:00
Fangzhou Ai and Claude Opus 5.5
3059155b47
[ROCm][Perf] MXFP8 GEMM on native 32x32 block scales for gfx950 ( #58510 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-24 21:26:30 -05:00
7d99c2c4fd
[Feature][Humming] Humming feature integration ( #56685 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-20 08:11:01 -07:00
Misha Goin and ErinYin
e5fce7b56b
[Core][Kernel] Share persistent workspaces for Marlin and Humming ( #57421 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: ErinYin <liang.yin@daocloud.io >
2026-09-20 17:28:33 +08:00
siyu and Isotr0py
4cc15f2121
[Quantization] Let ModelOpt MXFP8 layers load pre-processed weights Purpose ( #57316 )
...
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn >
Signed-off-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
2026-09-19 09:16:39 +00:00
rasmith and Shanshan Shen
acc2ed2a5f
[ROCm][CI][The Rock 10] Fix (MI355) Quantized Models failure on The Rock 10 with Triton 3.8.x ( #54849 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-17 15:20:35 -05:00
JohnQinAMD and Claude Opus 5
ef5f7cd119
[ROCm][DSv4.1][Perf] Dequantize the MXFP8 weight once when dot_scaled cannot be used ( #56560 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-15 09:07:26 +08:00
9b959b8657
[Model] DeepSeek-V4.1-Flash Model Definitions ( #56228 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-10 20:16:26 +08:00
7a977c0699
[ModelOpt] Redesign the LinearMethod classes using the generic QuantKey-driven method ( #49381 )
...
Signed-off-by: Juhi Mittal <juhim@nvidia.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-01 16:28:56 -04:00
Seonjin and Misha Goin
bd8865a299
[Kernel] Add FlashInfer TRTLLM MXFP8 linear backend ( #52204 )
...
Signed-off-by: seonjinn <sna@nvidia.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-08-20 10:07:22 -04:00
69d3335066
[Bugfix][Quantization] Guard the MXFP8 FlashInfer path on FlashInfer availability ( #52648 )
...
Signed-off-by: Han Li <han.li124@imperial.ac.uk >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-08-17 18:38:42 -07:00
Misha Goin
4ab5e5012a
[Refactor] Simplify B12X linear kernels and warmup ( #52368 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-17 09:45:45 -07:00
Luke Alonso
3c79b1a8bf
[Kernel] Add B12X dense linear backends ( #52016 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
2026-08-13 19:48:33 -07:00
Jinzhen Lin and OpenAI Codex
fcde8e1460
[Refactor] refactor humming linear and moe backends to use explicit layer configs ( #49610 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-07 10:03:19 -07:00
d779835a11
[XPU] Support MXFP8 linear weights for INC DeepSeek V4 model ( #48476 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 13:17:48 +08:00
Ethan Yang and amd-ethany
d9e57ea82e
[ROCm][Perf] MXFP8 dense-linear + grouped-MoE GEMM optimizations for MiniMax-M3 ( #46117 )
...
Signed-off-by: amd-ethany <amd-ethany@users.noreply.github.com >
Co-authored-by: amd-ethany <amd-ethany@users.noreply.github.com >
2026-07-08 04:03:34 +00:00
Jinzhen Lin and mgoin
d891b9bd51
[Quantization] add humming moe backend to all dense/moe oracles ( #41652 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-07-06 13:36:07 -07:00
Yongye Zhu
11b56b2ff2
[Kernel] Add FlashInferCutedslMxfp8LinearKernel (cute-dsl mm_mxfp8) ( #46393 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-06-23 12:45:49 -07:00
Hongxia Yang
f2beaa80c8
[ROCm][Quant] mxfp8 moe/linear gfx950 tuning for MiniMax-M3 ( #45725 )
...
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com >
2026-06-16 18:50:40 +00:00
0a1c5034f5
[Model] Add MiniMax M3 support ( #45381 )
...
Signed-off-by: youkaichao <youkaichao@gmail.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-06-16 01:01:25 +08:00
zofia
1b19bd7589
[MXFP8] [XPU] add a new compressed tensor schema and add a xpu mxfp8 gemm kernel ( #38707 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
2026-04-13 16:59:20 +08:00
Michael Goin
11e2375fe2
[Refactor] Move MXFP8 GEMM management into MxFp8LinearKernel ( #39205 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-04-10 14:02:03 -07:00