MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
SeongJun Lee and lesj0610
a98247ab4d
[Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 ( #59853 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-10-10 06:51:52 +00:00
stefankoncarevic and Andreas Karatzas
6c300ddb9e
[CI][Bugfix] Relax packed_qk_rope_ correctness test to one ULP ( #59008 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-28 17:29:59 +00:00
Thien Tran
39d49539de
[KimiViT][Perf] Fuse per-layer QK RoPE into one in-place kernel ( #58651 )
2026-09-27 18:18:06 -07:00
16070ed7e9
[Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope ( #58526 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Kimi <noreply@moonshot.cn >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-24 18:30:32 -07:00
b715699df4
[ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE ( #50212 )
...
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com >
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 11:11:57 -05:00
Michał Ganczarenko and Claude
73c59d365b
[XPU] Wire up SYCL apply_rotary_emb kernel in ApplyRotaryEmb ( #55721 )
...
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com >
Signed-off-by: Michał Ganczarenko <michal.ganczarenko@intel.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-22 20:43:24 -07:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Harry Mellor and Claude Opus 5
c191787a68
[Bugfix] Align vLLM YaRN with Transformers and stop re-scaling max_model_len ( #56446 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-11 13:01:38 +00:00
Jakub Byczkowski and Copilot
c9355e25e8
[XPU][Bugfix] Add forward_xpu to Ernie4_5_VLRotaryEmbedding ( #55942 )
...
Signed-off-by: Jakub Byczkowski <jakub.byczkowski@intel.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-09-10 17:48:00 +08:00
fe26c705f7
[Model] Add Bailing V3 VL support ( #55921 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 02:46:56 -07:00
Harry Mellor and Claude Opus 5
719284fe15
[Core][Model] Unify XD-RoPE into M-RoPE and derive the channel count ( #56078 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-09 09:23:30 -07:00
Jee Jee Li
199cb9b964
[Kernel] Remove unused fake implementation ( #55535 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-09-06 17:54:32 -07:00
Yan Ma
6feafb8b7d
[XPU] follow cuda path for mrope on XPU ( #53201 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-08-21 10:13:27 +00:00
Jakub Byczkowski and Copilot
c296cf8259
[Bugfix] Add forward_xpu to XDRotaryEmbedding for HunyuanOCR on XPU ( #52174 )
...
Signed-off-by: Jakub Byczkowski <jakub.byczkowski@intel.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-08-18 07:54:05 +08:00
bastefaniak
71b0da7c4e
[Bugfix] Fix .../mrope.py::apply_interleaved_rope() when torch.compile is used in torch==2.13 ( #52005 )
...
Signed-off-by: Bartosz Stefaniak <bstefaniak@nvidia.com >
2026-08-13 22:08:40 +00:00
Andreas Karatzas
05a0814863
[ROCm] Fix and optimize GPT-J-style MRoPE ( #49906 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-28 10:50:10 -06:00
Yufeng He and Wentao Ye
1f60771c74
fix: guard flash-attn rotary import ( #42679 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-06-11 08:43:31 -04:00
00d1fb7747
[Bugfix][ROCm] ApplyRotaryEmb: fall back to native when flash_attn rotary grid would exceed the HIP per-dim limit ( #43684 )
...
Signed-off-by: vLLM ROCm fix <noreply@example.com >
Signed-off-by: amd-fuweiy <fuweiy@amd.com >
Co-authored-by: vLLM ROCm fix <noreply@example.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-06-06 02:29:16 -07:00
Schwinn Saereesitthipitak
b7c5baf63d
fix: keep DeepSeek V4 RoPE cache on inv_freq device ( #43926 )
...
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com >
Signed-off-by: Schwinn Saereesitthipitak <17022745+galletas1712@users.noreply.github.com >
2026-06-05 02:30:29 +04:00
akii96
de12f5ca0b
[ROCm][GPT-OSS] Avoid repeated compile-time cos_sin_cache.to(bf16) casts in rotary path ( #42833 )
...
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com >
2026-05-27 16:22:27 +08:00
117afeea46
Fix error in Dynamic NTK scaling ( #41277 )
...
Signed-off-by: Max de Bayser <mbayser@br.ibm.com >
Signed-off-by: Max de Bayser <maxdebayser@gmail.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-05-19 17:27:54 -04:00
Jonathan Mamou and Wentao Ye
c8a6e272e0
[CPU] Fix rotary embedding for CPU without flash-attn ops ( #42225 )
...
Signed-off-by: jmamou <jonathan.mamou@intel.com >
Signed-off-by: Jonathan Mamou <jonathan.mamou@intel.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-05-12 15:05:35 +00:00
Rohan Potdar and ElizaWszola
a51376b3f0
[Performance][DSR1]: Fused RoPE+KVCache+q_concat for MLA ( #40392 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com >
Co-authored-by: ElizaWszola <ewszola@redhat.com >
2026-05-11 14:10:50 +00:00
Jee Jee Li and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
9d8ad5b408
[Bugfix] Fix repeated DSv4 RoPE cache initialization ( #41148 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-29 20:29:55 +08:00
+7
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
1096125073 and xiayongqiang
71a9125c67
[New Model]: add support for telechat3 ( #38510 )
...
Signed-off-by: xiayongqiang <xiayq1@chinatelecom.cn >
Co-authored-by: xiayongqiang <xiayq1@chinatelecom.cn >
2026-04-03 08:26:22 +08:00
08ed2b9688
feat(models): implement Google Gemma 4 architecture support (MoE, Multimodal, Reasoning, Tool-Use) ( #38826 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Signed-off-by: Luciano Martins <lucianomartins@google.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
2026-04-02 11:13:28 -07:00
Wang, Yiting
68e1b711f1
[XPU] Add deepseek_scaling_rope fused kernel ( #36612 )
...
Signed-off-by: yitingw1 <yiting.wang@intel.com >
2026-03-16 12:35:08 +08:00
Kunshang Ji
53ec16a705
[Hardware] Replace torch.cuda.device_count/current_device/set_device API ( #36145 )
...
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-03-12 07:57:47 -07:00
Jiayi Yan and Harry Mellor
6a895197fa
[Bugfix][CI] fix typos ( #34934 )
...
Signed-off-by: 1195343015 <1195343015@qq.com >
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-05 17:05:46 +00:00
Rohan Potdar
f38f8c9742
[ROCm]: Enable customop and rope+kvcache fusion for AITER RoPE ( #35180 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
2026-02-25 04:36:40 +00:00
haosdent
a7f341c323
[Bugfix] Fix MRotaryEmbedding missing truncate attr with YaRN scaling ( #35080 )
...
Signed-off-by: haosdent <haosdent@gmail.com >
2026-02-23 16:05:52 +00:00
Simon Danielsson
4292c90a2a
[Bugfix] Support RotaryEmbedding CustomOp for gpt-oss ( #33800 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-02-04 20:17:41 +00:00
zxy and Isotr0py
a3acfa1071
[Models] Intern-S1-Pro ( #33636 )
...
Signed-off-by: zxy <zhou0493@e.ntu.edu.sg >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-02-03 05:49:45 -08:00
Kunshang Ji
e10604480b
[XPU][1/N] Deprecate ipex and switch to vllm-xpu-kernels for xpu platform ( #33379 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-02-02 22:46:10 -08:00
will b. and Eduardo Salinas
46b4a02794
Fix DeepSeek V2 RoPE initialization error ( #33501 )
...
Signed-off-by: Eduardo Salinas <edus@microsoft.com >
Signed-off-by: catswe <212922539+catswe@users.noreply.github.com >
Co-authored-by: Eduardo Salinas <edus@microsoft.com >
2026-02-01 21:00:56 +00:00
hujiaxin0 and Emilie1001
ba45bedfd1
[model] Add support for openPangu7B-VL ( #32449 )
...
Signed-off-by: hujiaxin <524446785@qq.com >
Signed-off-by: Emilie1001 <79921183+Emilie1001@users.noreply.github.com >
Co-authored-by: Emilie1001 <79921183+Emilie1001@users.noreply.github.com >
2026-01-30 15:54:27 +08:00
Harry Mellor
f1acbd68c5
[CI] Enable mypy import following for vllm/compilation ( #33199 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-01-28 08:59:54 +00:00
Luka Govedič and Michael Goin
5e4e0e51f4
[torch.compile] Compile CustomOp.forward_native for SiluAndMul and QuantFP8 to avoid raw torch ops inside opaque custom ops ( #32806 )
...
Signed-off-by: Luka Govedič <lgovedic@redhat.com >
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-01-22 19:52:26 -08:00
Lucas Kabela
ad8818bb5e
[Misc][BE] Type coverage for vllm/compilation [3/3] ( #31748 )
...
Signed-off-by: Lucas Kabela <lucaskabela@meta.com >
2026-01-12 19:24:38 +00:00
Lucas Kabela
ea6d067a2a
[Misc][LLaMa4] Compile LLaMa Vision Encoder ( #30709 )
...
Signed-off-by: Lucas Kabela <lucaskabela@meta.com >
2026-01-09 22:01:38 -05:00
Michael Goin
d5ec6c056f
[UX] Add vLLM model inspection view ( #29450 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-01-09 10:12:35 -07:00
Shanshan Shen
08d954f036
[Doc] Add developer guide for CustomOp ( #30886 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-01-09 16:21:11 +00:00
RickyChen / 陳昭儒
c455b771fd
[Bugfix][CPU] Fix RotaryEmbedding fallback causing gibberish with --enforce-eager ( #31643 )
...
Signed-off-by: rickychen-infinirc <ricky.chen@infinirc.com >
2026-01-06 01:25:38 +08:00
Shanshan Shen
23a1946e3b
[CustomOp][Refactor] Extract common methods for ApplyRotaryEmb CustomOp ( #31021 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2025-12-19 22:16:09 +08:00
jiahanc
53ad423f26
[Perf] enable flashinfer rotary_embedding custom ops in DeepSeek rotary ( #30729 )
...
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
2025-12-18 14:31:18 -05:00
Li, Jiang and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
0cd5353644
[Bugfix][CPU] Fix CPU backend ROPE dispatch for VL models ( #30829 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
Signed-off-by: Li, Jiang <bigpyj64@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-12-16 23:25:12 -08:00