[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces

Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.

- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
  scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
  CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
  CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
  render routes, and vllm/config/speech_to_text.py. GenerationTask is
  generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
  Multi, Offloading and Example connectors stay; production uses the
  CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
  arch gate, declarations and op registrations. The CUDA build never
  defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
  fail with "The model ... does not exist. Valid aliases: ...".

Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-19 21:06:19 +07:00
co-authored by Hinata
parent 5f3c7bd8ab
commit 5aa2ac64f8
269 changed files with 417 additions and 29932 deletions
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -8,7 +8,6 @@ from transformers import AutoModelForSeq2SeqLM
from vllm.assets.audio import AudioAsset
from vllm.envs import disable_envs_cache
from vllm.lora.request import LoRARequest
from vllm.multimodal.audio import AudioResampler
from ....conftest import HfRunner, VllmRunner
@@ -26,7 +25,6 @@ AUDIO_MODEL_SETTINGS: dict[str, dict[str, Any]] = {
"<|end_of_text|>\n"
"<|start_of_role|>assistant<|end_of_role|>"
),
"audio_lora_path": "ibm-granite/granite-speech-3.3-2b",
},
"nvidia/audio-flamingo-3-hf": {
"prompt": (
@@ -90,7 +88,6 @@ def test_transformers_audio_generation(
model_id: str,
):
settings = AUDIO_MODEL_SETTINGS[model_id]
audio_lora_path = settings.get("audio_lora_path")
audio, orig_sr = AUDIO_ASSET.audio_and_sample_rate
target_sr = settings.get("sampling_rate", orig_sr)
@@ -105,19 +102,13 @@ def test_transformers_audio_generation(
max_model_len=2048,
enforce_eager=True,
limit_mm_per_prompt={"audio": 1},
enable_lora=audio_lora_path is not None,
max_lora_rank=64,
**settings.get("vllm_runner_kwargs", {}),
) as vllm_model:
lora_request = (
LoRARequest("audio", 1, audio_lora_path) if audio_lora_path else None
)
vllm_outputs = vllm_model.generate_greedy_logprobs(
[settings["prompt"]],
128,
num_logprobs=10,
audios=[audio],
lora_request=lora_request,
)
with hf_runner(