23 Commits
Author SHA1 Message Date
MOVZXandHinata 5aa2ac64f8 [Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.

- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
  scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
  CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
  CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
  render routes, and vllm/config/speech_to_text.py. GenerationTask is
  generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
  Multi, Offloading and Example connectors stay; production uses the
  CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
  arch gate, declarations and op registrations. The CUDA build never
  defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
  fail with "The model ... does not exist. Valid aliases: ...".

Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 21:06:19 +07:00
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 00:20:48 +07:00
Talor Abramovich 81e42c9987 [Feature][Frontend] Control the log level of request JSON body using env var (#60630)
Signed-off-by: talora <talora@nvidia.com>
2026-10-15 11:07:19 +00:00
GuanLuoandsfeng33 6d1daa9fe5 [Bugfix][Frontend] Expose completion response schemas in OpenAPI (#60926)
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
2026-10-13 18:54:06 +00:00
SrirajandFlora Feng db6e3cd8c4 [Bugfix][Frontend] Log the full request body when max_log_len is unset (#60310)
Signed-off-by: sricursion <sricursion@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
2026-10-07 00:42:17 +00:00
5dd693122c [MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:27:45 +01:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
wang.yuqi fe4d4109c2 [Frontend][last/N] Move all non-OpenAI content out of the OpenAI folder. (#56369)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-09-11 05:33:26 +00:00
Raphaël RiallandOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ea40bb9e90 [feature] Watermarked generation and detection (Gumbel-max algorithm) (#54053)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 21:00:15 -07:00
wang.yuqiandCyrus Leung eeb549a74d [Frontend] Move engine/protocol.py out openai folder (#54492)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
2026-08-31 10:13:50 +00:00
Moe Huzaifa 3e833933dc [Bugfix][Frontend] Keep credentials out of the Rust frontend launch log (#53738)
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com>
2026-08-25 16:32:12 +00:00
Moe Huzaifa d3e2888c75 [Bugfix][Frontend] Redact hf_token in the non-default args log (#53625)
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com>
2026-08-25 03:06:46 +00:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-08-19 09:43:46 +00:00
LuckyStep 9a9aa2b017 [Bugfix] Redact api_key in non-default args log (#52523)
Signed-off-by: LuckyStep <67696304+Andy365-365@users.noreply.github.com>
2026-08-19 15:52:53 +08:00
wang.yuqi b8165e5e58 [Frontend] Consolidate entrypoint exception handler (#52261)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-08-14 09:27:24 +00:00
yang rui 152c913a63 [Frontend] Log output token IDs at DEBUG level (#52098)
Signed-off-by: ruirui6946 <142162413+ruirui6946@users.noreply.github.com>
2026-08-13 13:11:51 +00:00
Muhammad FawazandMahad Durrani e87521626f Sanitize server file paths from validation error responses (#46415)
Signed-off-by: muhammadfawaz1 <135441198+muhammadfawaz1@users.noreply.github.com>
Co-authored-by: Mahad Durrani <114791389+mahadrehmann@users.noreply.github.com>
2026-07-09 17:46:29 +08:00
Muhammad FawazandMahad Rehmann 1bf149f334 Filter Pydantic-internal markers from validation error param (#46457)
Signed-off-by: muhammadfawaz1 <135441198+muhammadfawaz1@users.noreply.github.com>
Co-authored-by: Mahad Rehmann <114791389+mahadrehmann@users.noreply.github.com>
2026-06-23 13:20:50 +00:00
Muhammad FawazandMahad Durrani 69bdd34542 [Bugfix] Fall back to Pydantic loc for param in validation errors (#46038)
Signed-off-by: professorsab <135441198+professorsab@users.noreply.github.com>
Co-authored-by: Mahad Durrani <114791389+mahadrehmann@users.noreply.github.com>
2026-06-19 19:11:11 +08:00
Andreas Karatzas 1033ffac2e [CI] Wait for SSL cert refresher events in the test (#45489)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-06-13 04:57:18 +00:00
Juan Pérez de Algabaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 9492362972 [Security] Apply sanitize_message to Anthropic and STT error paths (#45119)
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-11 10:05:34 +00:00
wang.yuqi d01d0b4646 [Frontend] Consolidate online serving utils. (#44479)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-06-04 06:49:31 +00:00