108 Commits
Author SHA1 Message Date
MOVZXandHinata 5aa2ac64f8 [Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.

- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
  scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
  CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
  CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
  render routes, and vllm/config/speech_to_text.py. GenerationTask is
  generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
  Multi, Offloading and Example connectors stay; production uses the
  CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
  arch gate, declarations and op registrations. The CUDA build never
  defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
  fail with "The model ... does not exist. Valid aliases: ...".

Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 21:06:19 +07:00
MOVZXandHinata 5f3c7bd8ab [Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
  target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
  and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
  code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
  server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
  the live server process maps none of them
- Delete tests that only build architectures this fork no longer
  registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
  import at module level

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 19:31:07 +07:00
MOVZXandHinata 2066d3143f [Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 17:10:56 +07:00
MOVZXandHinata 2b544b7ba0 [Mini-VLLM] Cut dead subsystems across serving, connectors, parsers
- Remove P/D transfer connectors: nixl, nixl_chat, mooncake x2, hf3fs,
  moriio, hisparse, example and hidden-states connectors
- Reduce weight_transfer to the base API used by model loading
- Remove kv_offload tiering, simple_kv_offload, ec cpu/example,
  snapshot command, engine snapshot server
- Remove transformers model fallback, dsv4 tokenizers, dead platforms
  cpu/tpu/xpu, flashmla third_party, dsa, megamoe/flydsl, dead attn
  ops, dead layers (short_conv, fused_embed_norm, indexer_topk,
  relu2 gemma_ar, lightning_attn, minimax_rms_norm), gemm_rs_ar
- Keep MoE LoRA op (imported eagerly by punica_gpu); drop fp8 variant
- Rebuild parser hubs around Qwen3 adapters only (36 dead adapters)
- Remove dead tests with the code they covered

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 02:19:03 +07:00
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 00:20:48 +07:00
dskwe 49cb2bbf71 [Bugfix] Honor code_revision in config-based tokenizer loading (#61611)
Signed-off-by: Yuan Li <dskwelmcy@163.com>
2026-10-16 05:21:12 +00:00
Artem Perevedentsev bb1438a93f [CI] Fix Unlimited-OCR loading after upstream model_type rename (#61590)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-14 01:24:24 -07:00
dskweandCyrus Leung 28d679e762 [Bugfix] Preserve independent tokenizer code_revision in snapshot resolution (#61272)
Signed-off-by: Yuan Li <dskwelmcy@163.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-10-13 04:23:45 +00:00
3facf822b7 [Performance] Load HF tokenizers and processors from a local snapshot (#60825)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-12 13:45:54 +00:00
Wentao Ye 3709632ff2 [Refactor] Remove dead code multiple places (#60120)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-10-10 15:41:10 +00:00
wang.yuqi c1692c85f1 [Misc] mv chat_utils.py to renderers. (#60522)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-10-10 06:50:34 +00:00
KamranandCodex 8d10c38b73 [Perf] Reduce fallback decode CPU by 69% at 32K (#60497)
Signed-off-by: spa5k <79936503+spa5k@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
2026-10-08 21:42:56 +00:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 02:30:52 -07:00
Harry MellorandClaude Opus 5.5 5d20979fc9 [Misc] Remove code paths for Transformers < 5.16.1 (#59762)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 11:17:24 +00:00
Martin Hickey fcfacc3499 [Bugfix][Frontend] Seed derender detokenization from the prompt (#59046)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-10-02 03:17:56 +00:00
Shijie Lyu 37cecc8141 [Bugfix][Tokenizer] Fix off-by-one max_token_id from vocab_size (#59491)
Signed-off-by: Shijie Lyu <lshjhf@gmail.com>
2026-10-01 05:43:30 +00:00
ddd6fbca14 [Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: zhec <chengyunfei@ruc.edu.cn>
2026-09-26 02:18:58 +00:00
Harry MellorandClaude Opus 5.5 d636456b1f [Frontend] Remove the slow tokenizer mode (#58545)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 08:48:28 -07:00
e562003adb [Tokenizer] Drop dead Mistral tokenizer shims for transformers#41962 (#56325)
Signed-off-by: Aditya Ganesh Kumar <adty910@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-09-18 12:46:41 +01:00
adenzhou1350OpenAI CodexIsotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
cc09352e5c [Bugfix][Multimodal] Preserve DeepSeek V4 image block spacing (#56882)
Signed-off-by: Xucheng Zhou <aden1350@outlook.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-18 18:44:39 +08:00
JasonandChauncey 2909ad8fa4 [Bugfix] Attach request-level tools to existing system message in DeepSeek V4 Python renderer (#51856)
Signed-off-by: Jason Yao <wsyjh8@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
2026-09-18 11:07:25 +08:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
CedricHuangandOpenAI Codex eb42686a30 [Bugfix][Frontend] Support Responses text types in DeepSeek V4.1 (#56299)
Signed-off-by: CedricHuang <38417461+CedricHwong@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-14 10:43:09 +08:00
Julien Denize 983b7e28c9 [Bugfix] Avoid MistralCommonBackend for HF tokenizers (#54192)
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
2026-09-10 17:57:44 +00:00
Bugen ZhaoandIsotr0py b47b01cf33 [Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-10 01:15:06 -07:00
Adababyandshaolila 5133e1d285 [Frontend] Migrate input-validation errors to VLLMValidationError (harmony/mistral/chat_utils/params) (#55735)
Signed-off-by: shaolila <shaolila@buaa.edu.cn>
Co-authored-by: shaolila <shaolila@buaa.edu.cn>
2026-09-08 06:45:40 +00:00
Isotr0py 1356635d83 [New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support (#54566)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-02 11:09:09 -07:00
Moe Huzaifa 19406fae28 [Bugfix][Tokenizer] Replace bare asserts in the DeepSeek V4 encoder (#53747)
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com>
2026-08-25 16:30:23 +00:00
Jiahao LiangandChauncey 59d7fc92ea [Bugfix][DeepSeek V4] Handle trailing system messages in prompt rendering (#51262)
Signed-off-by: jiahaoliang <gzliangjiahao@gmail.com>
Signed-off-by: Jiahao Liang <gzliangjiahao@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
2026-08-25 07:49:44 +00:00
JC-ut0 38e9cefdef [Bugfix] Return HTTP 400 instead of 501 for unknown chat roles in DeepSeek encoders (#53071)
Signed-off-by: JC-ut0 <809602657@qq.com>
2026-08-20 09:23:41 +00:00
Clinton ThomasLucas Bourtoulemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
b64a2708b0 Bound generation inputs before expensive work (#51447)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 22:03:50 +08:00
Flora FengandWentao Ye 8bcc916a98 [Bugfix] Fix DeepSeek V4/3.2 tokenizer vocab size overcount crashing guided decoding (#51727)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-08-10 23:00:08 +00:00
Bugen Zhao 7743486190 [Frontend] DeepSeek V4 0731 reasoning effort prompts & mappings (#50580) 2026-08-04 08:51:32 -07:00
andrewbcohereandCyrus Leung 9c110fa522 [Frontend] Cohere chat v2 api support (#47189)
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-07-31 22:53:54 -07:00
Julien Denize 1a20d23dab [PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947) 2026-07-30 15:38:00 +02:00
Bugen Zhao f5a7cce9b6 [Model] Add Kimi K3 support: Python frontend [2/2] (#50093)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-07-29 01:06:01 -07:00
Wentao Ye b5bcb3ce88 [Refactor] Remove dead code in multiple files (#49745)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-07-27 15:58:26 -04:00
Andreas Karatzas cbc3a87200 [Tokenizer] Use HF config for HF tokenizers (#49907)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-07-27 07:49:11 +00:00
Andreas Karatzas 9863102ed9 [CI] Reuse loaded config for cached tokenizer (#49509)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
2026-07-24 12:32:40 -07:00
aoshen02mvanhornClaude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
dcfebf93f4 [Bugfix] Fix logprobs token-string collision from SentencePiece space… (#48674)
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: mvanhorn <mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-20 12:17:18 +08:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 877dae9c68 [Refactor] Remove deepseek dead code (#48780)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-17 14:57:13 +00:00
6570c9800c [Model] Add Inkling model support [1/N] (#48799)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-15 23:40:07 -07:00
Harry Mellor dd94484577 Bump Transformers version to 5.10.4 (#41359)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-07-07 05:13:28 -07:00
c6741b2ad4 [Model] Support Unlimited OCR (#46564)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Roger Wang <hey@rogerw.io>
2026-06-27 23:09:18 -07:00
Tiezhen WANG c7645bce04 Remove grok model arch from vllm (#46706)
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
2026-06-25 23:02:10 -07:00
Andrew BarnesandOpenAI Codex a9a8a32dcd Register parsed config classes before tokenizer init (#40299)
Signed-off-by: Bortlesboat <bortstheboat@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-06-16 05:33:08 +00:00
WEI CHENG CHIU 5b2943f5a6 [Bugfix] Return the tokenizer from maybe_make_thread_pool so it survives pickling (#45460)
Signed-off-by: Wayne Chiu <waynehacking8@gmail.com>
2026-06-13 06:01:35 +00:00
Isotr0py 6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-06-12 12:02:21 -07:00
Harry Mellor 18d87a87dc Deprecate Transformers v4 support (#45161)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-06-11 11:04:01 +08:00
Harry Mellor 3cc9fecd58 Deprecated 1st generation Qwen and QwenVL models (#45131)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-06-10 14:55:33 +00:00