MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
Mikko Tukiainen
abad32a718
[ROCm][Perf] Fix Gemma AR+RMS prefill fallback ( #58547 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Signed-off-by: Mikko Tukiainen <mikko.tukiainen@amd.com >
2026-10-12 19:53:08 +00:00
Misha Goin
58aa2f1ef1
[Refactor] Move auxiliary files out of the repository root ( #58572 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-24 12:38:56 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
ab35354c21
[ROCm][AITER] Skip AITER norm kernels when flattening to 2D would copy ( #55991 )
...
Signed-off-by: Zheng Gong <zgong@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-09-16 07:26:56 +00:00
78e84261ab
[Performance] Add Triton kernel for Gemma3n sparse GELU ( #48498 )
...
Signed-off-by: BWAAEEEK <jooho414@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-14 07:42:15 +00:00
Jee Jee Li
199cb9b964
[Kernel] Remove unused fake implementation ( #55535 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-09-06 17:54:32 -07:00
Max Hu and Max Hu
d462dee37d
[Bugfix][Triton] Make fp8_min/fp8_max constexpr in _quantize_pad_fp8_kernel ( #51872 )
...
Signed-off-by: Max Hu <maxhu@nvidia.com >
Co-authored-by: Max Hu <maxhu@nvidia.com >
2026-08-12 03:09:54 +00:00
Chaojun Zhang and Kunshang Ji
482cfc20f0
[XPU] Unify XPU RMSNorm kernels with vllm_c and drop redundant XPU-specific implementation ( #46981 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-31 17:28:18 +08:00
Shangdi Yu and OpenAI Codex
45b60e3914
[Kernel][Helion] Disable unsafe B200 RMS reduction warp specialization ( #50345 )
...
Signed-off-by: Shangdi Yu <shangdiy@meta.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-30 20:37:52 +00:00
BadrBasowid
59e831c09a
[Compilation]Fuse Transformers Residual Add + RMSNorm ( #48757 )
...
Signed-off-by: BadrBasowid <badr.basowid@gmail.com >
2026-07-30 13:46:36 +00:00
72297d859b
[XPU] Route weightless RMSNorm to _C dispatch ( #47121 )
...
Signed-off-by: Yintong Lu <yintong.lu@intel.com >
Co-authored-by: Yongqi Wang <yongqi.wang@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-29 18:59:58 +08:00
Andreas Karatzas
49f31d7cee
[ROCm] Make vllm_c RMSNorm output contiguous ( #49913 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-26 23:59:14 -05:00
Shangdi Yu and Claude Opus 4.8
c9be3a8aa1
[Kernel][Helion] Disable warp specialization in rms_norm_per_block_quant B200 configs ( #48797 )
...
Signed-off-by: Shangdi Yu <shangdiy@meta.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-17 21:39:09 +00:00
Shangdi Yu
efa2e424f6
[Helion] Fix degenerate scale_ub in kernel input generators ( #48868 )
...
Signed-off-by: Shangdi Yu <shangdiy@meta.com >
2026-07-16 19:57:45 +00:00
Xiaohong (Sean) Chen
6036bf110a
[Kernel][Helion] Add Helion kernel benchmark script ( #48512 )
...
Signed-off-by: Sean Chen <seachen@redhat.com >
2026-07-15 15:43:06 +00:00
Xiaohong (Sean) Chen
2fa63e0fff
[Kernel][Helion] Helion kernel lazy registration ( #48264 )
...
Signed-off-by: Sean Chen <seachen@redhat.com >
2026-07-15 15:42:46 +00:00
9fde043f54
[Kernel][Helion][1/N] Add Helion kernel for silu_and_mul_per_block_quant ( #43994 )
...
Signed-off-by: Sean Chen <seachen@redhat.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 00:19:01 +08:00
Wentao Ye
b1c6dba558
[Refactor] Remove multiple dead code ( #47329 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-06 07:54:08 -07:00
zofia and Kunshang Ji
72f639927f
[XPU] [RMSNorm] revert weightless change on xpu ( #46987 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-06-29 19:03:06 +00:00
Xiaohong (Sean) Chen
49e28e8e91
[Kernel][Helion][1/N] Add Helion kernel for fused_qk_norm_rope ( #44010 )
...
Signed-off-by: Sean Chen <seachen@redhat.com >
2026-06-29 22:54:15 +08:00
cyq
e06a83445c
[Bugfix] Normalize slashes in Helion GPU names ( #46101 )
...
Signed-off-by: cyq <15000851237@163.com >
2026-06-24 18:22:49 -05:00
Xiaohong (Sean) Chen and Yanan Cao
46f74e144b
[Kernel][Helion][1/N] Add Helion kernel for rms_norm_dynamic_per_token_quant ( #34432 )
...
Signed-off-by: Sean Chen <seachen@redhat.com >
Co-authored-by: Yanan Cao <gmagogsfm@gmail.com >
2026-06-17 23:03:54 +08:00
arghyadeep sarkar
93bbe94d3a
[Kernel] Add weightless RMSNorm CUDA kernels for has_weight=False ( #41430 ) ( #44109 )
...
Signed-off-by: hello-args <args.sarkar@gmail.com >
2026-06-16 23:45:55 -07:00
Xiaohong (Sean) Chen and Yanan Cao
ce3ef17bec
[Kernel][Helion][1/N] Add Helion kernel for rms_norm_per_block_quant ( #36895 )
...
Signed-off-by: Sean Chen <seachen@redhat.com >
Co-authored-by: Yanan Cao <gmagogsfm@gmail.com >
2026-06-16 22:09:52 +08:00
Xiaohong (Sean) Chen and Yanan Cao
272c16953e
[Kernel][Helion][1/N] Add Helion kernel for dynamic_per_token_scaled_fp8_quant ( #33790 )
...
Signed-off-by: Sean Chen <seachen@redhat.com >
Co-authored-by: Yanan Cao <gmagogsfm@gmail.com >
2026-06-12 12:50:06 -04:00
2ec6594db9
[Kernel][Helion][1/N] Add Helion kernel for per_token_group_fp8_quant ( #36902 )
...
Signed-off-by: Sean Chen <seachen@redhat.com >
Co-authored-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-11 08:59:08 -07:00
Andreas Karatzas
d28bdf9344
[ROCm][CI] Fix ROCm LoRA Transformers fallback with full CUDA graphs ( #41577 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-23 04:31:32 +00:00
Yanan Cao and Claude Sonnet 4
0b99971352
[Kernel][Helion] Optimize Helion config parsing latency ( #40850 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com >
2026-05-07 20:27:34 -07:00
Luka Govedič
d58c42e19c
[vLLM IR] 2/N fused_add_rms_norm and maybe_inplace overload ( #36823 )
...
Signed-off-by: Luka Govedič <lgovedic@redhat.com >
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2026-05-01 23:41:15 -04:00
Zhanda Zhu and Yubo Gao
5d5c776444
[Perf] FP8 FlashInfer Attn for ViT ( #38065 )
...
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com >
Co-authored-by: Yubo Gao <ybgao-nvidia@users.noreply.github.com >
2026-04-27 13:44:15 +08:00
Yanan Cao and Claude Sonnet 4
5cdddddd4a
[Kernel] [Helion] Force disable HOP path due to performance regression ( #40171 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com >
2026-04-17 17:36:49 -04:00
Yanan Cao and Claude Opus 4.6
edc3648966
[Kernel][Helion] Fix inductor fusion of Helion HOP ( #39944 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-16 04:41:26 -07:00
Jiangyun Zhu and Luka Govedič
8060bb0333
[vLLM IR] rework gemma_rms_norm ( #39014 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2026-04-07 01:37:00 -07:00
Robert Shaw
7d266abb22
Revert "[vLLM IR] gemma_rms_norm" ( #38998 )
2026-04-04 17:48:08 -04:00
Xiaoshuang Wang
156405d243
[vLLM IR] gemma_rms_norm ( #38780 )
...
Signed-off-by: Icey <1790571317@qq.com >
2026-04-04 13:55:52 -04:00
Yanan Cao and Claude Sonnet 4
73f48ce559
[Kernel] [Helion] Use warning_once in get_gpu_name to prevent log spam ( #38743 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com >
2026-04-01 21:30:31 -07:00
40bb175027
[vLLM IR] 1/N Implement IR skeleton and rms_norm op ( #33825 )
...
Signed-off-by: Luka Govedič <lgovedic@redhat.com >
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com >
Signed-off-by: chzhang <chaojun.zhang@intel.com >
Signed-off-by: Luka Govedic <luka.govedic@gmail.com >
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com >
Co-authored-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Luka Govedič <ProExpertProg@h100-01.nemg-001.lab.rdu2.dc.redhat.com >
2026-03-31 22:15:05 -04:00
Yanan Cao and Claude Sonnet 4
cc671cb110
[Kernel] [Helion] [17/N] Add Helion kernel torch.compile support ( #38592 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com >
2026-03-31 17:06:42 -04:00
Yanan Cao and Claude Opus 4.6
ff9fbc9aff
[Kernel][Helion] [16/N] Refactor register_kernel API to be more Dynamo-friendly ( #36705 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-18 01:23:35 +00:00
Yanan Cao and Claude Opus 4.6
584a3f56de
[Kernel][Helion][13/N] Force static_shapes=False in helion register ( #36677 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-12 05:35:29 +00:00
Yanan Cao and Claude Opus 4.6
cf632499ee
[Kernel] [Helion] [15/N] Split config files into per-platform files ( #36698 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-11 17:25:29 -04:00
Yanan Cao and Claude Opus 4.6
0ce21c46a0
[Kernel] [Helion] [14/N] Set autotune_ignore_errors=True during autotuning ( #36683 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-11 17:25:04 -04:00
Yanan Cao
b0651021e5
[Kernel] [Helion] [11/N] Retune configs for silu_mul_fp8 ( #36062 )
2026-03-04 21:25:59 -08:00
Yanan Cao
9098ce690c
[Kernel] [Helion] [7/N] Use HOP to represent Helion Kernel call to enable fx tracing and pattern matching ( #34390 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
2026-02-27 09:21:35 -08:00
Yanan Cao and Claude Opus 4.6
9d7577b2bd
[Kernel] [Helion] [9/N] Canonicalize GPU variant names to base model names ( #34928 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-20 19:55:51 -08:00
Yanan Cao
a6d0299c75
[Kernel] [Helion] [6/N] Add num_tokens dimension to silu_mul autotuning and dispatching ( #34185 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
2026-02-20 08:36:51 -08:00
Andreas Karatzas
f13e86d8dd
[Kernels] Fix Helion GPU utils to use platform-agnostic device name API ( #34537 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-02-14 20:29:23 -08:00
Yanan Cao
de13dd781f
[Kernel] [Helion] [5/N] Add Helion Autotuning infrastructure ( #34025 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
2026-02-12 18:21:05 -08:00