MOVZX and Hinata
5f3c7bd8ab
[Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
...
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
the live server process maps none of them
- Delete tests that only build architectures this fork no longer
registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
import at module level
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 19:31:07 +07:00
MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
9eb7fb9157
[Build] Remove unused tools/install_deepgemm.sh ( #61265 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 15:57:49 -04:00
Jee Jee Li
bee95149e2
Update DeepGEMM commit ( #61744 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-15 17:54:40 -07:00
f8460ef236
[Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache ( #61371 )
...
Signed-off-by: juntianl <juntianl@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-15 13:58:25 -07:00
Micah Williamson
a4d553a46a
Revert "[Build] Update deep_gemm git tag for locality_domain update" ( #61703 )
2026-10-14 14:57:44 -07:00
Chris Leonard and Shengqi Chen
ded7ae35e5
[Build] Update deep_gemm git tag for locality_domain update ( #61491 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Signed-off-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-14 15:41:16 -04:00
Nick Hill and Claude Opus 5.5
78b32b3fdc
Revert "[Build] DeepGEMM migration to torch Stable ABI ( #59482 )" ( #61510 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-13 15:33:50 -07:00
Chris Leonard and Shengqi Chen
f5fefbffe5
[Build] DeepGEMM migration to torch Stable ABI ( #59482 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Signed-off-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-13 08:01:44 -07:00
31e90ac370
[Attention] Enable FA4 sparse MQA decode on Blackwell ( #55866 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-13 09:31:37 -04:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
Yongye Zhu and Claude Opus 5.5
2eb491e18b
[Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a ( #60684 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 13:40:45 +08:00
Chris Leonard and Shengqi Chen
cc73cca9c8
[Build] Migrate vendored DeepGEMM from pybind to TORCH_LIBRARY (abi3) ( #48962 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-06 08:39:04 -07:00
Yongye Zhu and Claude Opus 5.5
7566d83bd3
[Attention][MiniMax-M3] NVFP4 KV cache on the MSA sparse attention path ( #59300 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 14:39:55 -07:00
almayne and Li, Jiang <jiang1.li@intel.com>
93dd80f636
[CPU] Conv1d optimised kernel for aarch64 ( #54093 )
...
Signed-off-by: Anna Mayne <anna.mayne@arm.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-30 20:23:42 +08:00
Rukhaiya2004 and Li, Jiang <jiang1.li@intel.com>
72e7874fa6
[Hardware][Power] Enable W4A16 (AWQ & GPTQ) quantization on POWER10 using VSX ( #59149 )
...
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-30 19:44:48 +08:00
Rehan Khan and Li, Jiang <jiang1.li@intel.com>
583637f126
[CPU][Perf] Add vectorized Sampler Kernel ( #53913 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-30 15:16:28 +08:00
4be061c5af
[Kernel] Bump FlashKDA to keep the recurrent state in fp32 ( #58846 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-09-26 22:20:44 +00:00
e440b75bb6
[ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision ( #50605 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: jpvillam <juan.villamizar@amd.com >
2026-09-25 14:55:52 -05:00
f34a0e0797
[CPU] Gate the AVX10.2 paths on compiler support ( #58133 )
...
Signed-off-by: R <Ganesh.R@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-24 18:07:53 +08:00
Wentao Ye
355d789f0d
[Compilation] Fix QuTLASS compilation with PyTorch 2.13 ( #58173 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-22 20:43:28 -07:00
91d7324cb1
[perf] wire FA and FlashMLA for sm90 GLM5Next NoPE SparseMLA ( #55385 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Leoyzen <leoyzen@gmail.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-22 02:03:59 -07:00
eb87980585
[Build] Fix DeepGEMM CUDA 12.9 release builds ( #57554 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-09-20 20:14:00 -07:00
Yuwen Zhou and Li, Jiang <jiang1.li@intel.com>
b346479065
[CPU] Add CPU FP8 W8A8 linear/MoE support ( #49942 )
...
Signed-off-by: yuwenzho <yuwen.zhou@intel.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-18 17:59:42 +08:00
117cf43fb6
[HARDWARE][POWER] Enable W8A8 INT8 MoE on POWER ( #55316 )
...
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com >
Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com >
Co-authored-by: Akash kaothalkar <akash.kaothalkar@ibm.com >
Co-authored-by: Antigravity <antigravity@google.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-17 21:45:13 +08:00
Yongye Zhu
929aa2a7ed
[Build] Bump DeepGEMM pin to a6bbb80 ( #57218 )
2026-09-16 14:49:35 -07:00
Shang Wang and Codex
031f5810c1
[Build][NVIDIA] Update public Rubin dependencies and MSA compatibility ( #56545 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Codex <noreply@openai.com >
2026-09-15 16:29:13 -07:00
Yongye Zhu and Claude Opus 5
8263ea12bd
[Build] Move DeepGEMM pin to the vLLM fork and update the Mega MoE call convention ( #56876 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-15 00:57:18 -07:00
e6eb0d120c
[Model][DSv4.1] Store the whole KV in MXFP8 (FlashMLA V4.1 record) ( #56893 )
...
Signed-off-by: Yongye Zhu <yongye@inferact.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-14 20:12:55 -07:00
fa008bdccf
[Perf][Kernel] Integrate DeepSelect TopK for the DSA sparse indexer ( #56464 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-09-13 00:44:37 -07:00
Srijan Keshri and Shengqi Chen
6b153463a8
[Build] Define _USE_MATH_DEFINES for FlashMLA targets ( #54007 )
...
Signed-off-by: arcusbuilds <srijankeshri007@gmail.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-09-12 02:45:32 -07:00
Jiangyun Zhu and Codex
ec6b0494fe
[CI] Bump CUTLASS DSL to 4.7 ( #54927 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-11 20:11:29 -07:00
Wei Zhao
9d88ceb026
[KDA] Update flashKDA to support bf16 checkpoint state ( #56485 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-09-11 23:53:24 +00:00
Li, Jiang
c3ccc0e957
[Model] Add DeepSeek-V4 CPU backend ( #55355 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-10 14:14:48 +08:00
zhao, zhenhui
1c344ed41e
[CPU] [Feat] Add native AMX-FP8 attention impl for Diamond Rapids ( #49410 )
...
Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com >
2026-09-06 11:06:56 +08:00
GroupieSteven
4ee2595512
[Attention] Sync FA with upstream ( #54819 )
...
Signed-off-by: Chuyue Wang <stevenwang0805@outlook.com >
2026-09-04 23:48:32 -04:00
Thien Tran
f4e6136146
[Kimi-K3] Bump FlashKDA to fix unstable inverse ( #54859 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-02 00:16:29 -07:00
sychen52
7c5dc571cb
[Attention][DSA] Enable W4A16 DSA ( #51724 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
2026-09-01 12:37:47 +08:00
Matthew Bonanni and OpenAI Codex
de9250ac9e
[Attention] Enable masked MHA for GLM-5 head dimensions ( #53785 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-27 21:17:12 +00:00
Matthew Bonanni and OpenAI Codex
75dea9b4ae
[Bugfix] Update FlashMLA for sparse decode workspace fix ( #53755 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-27 13:37:29 +00:00
simon-veitner-redhat
7156c63bef
[SM100] Hdim 256 optimized ( #52980 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
2026-08-25 20:28:40 -07:00
9eb9d9d395
[Perf] Support internal prefill checkpoints for Mamba prefix caching, 9%~25% TTFT improvement ( #52789 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-22 10:32:40 -07:00
Roger Wang
755492e37d
Revert "[Kernel] Gemma-4 FA4 FP8 Kernel" ( #52987 )
2026-08-19 10:48:46 -07:00
Li, Jiang
e4d61d0d22
[CPU] Add AMX-only high-performance MLA backend for DeepSeek V2/V3/R1 ( #52616 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-08-19 10:28:52 +08:00
kiroxu
a18c9b56ff
[Kimi-K3][Perf] Update FlashKDA for automatic K2 V-split ( #52458 )
2026-08-16 18:04:38 -07:00
Shane Widanagama
d4801990a4
[CI/Build] Add warning for unsupported global PTX architecture requests in... ( #51901 )
2026-08-15 07:01:52 -07:00
827a2af806
[Kernel] Gemma-4 FA4 FP8 Kernel ( #48666 )
...
Signed-off-by: Qixiang Lin <qixiangl@nvidia.com >
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com >
Co-authored-by: Qixiang Lin <qixiangl@nvidia.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-08-13 17:42:00 -07:00
Yongye Zhu and Kimi Code
025d56a11e
[Build] Update DeepGEMM pin to deepseek-ai nv_dev tip ( #52035 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
2026-08-12 13:49:04 -07:00
Rehan Khan and Li, Jiang <jiang1.li@intel.com>
b22afe45ac
[CPU] Enable GPTQ and AWQ quantization for s390x ( #51148 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-08-10 11:41:43 +08:00