[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
Production runs one static model behind the OpenAI-compatible API. Every adapter path and non-OpenAI HTTP surface is unreachable there. - Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora CLI group, and the LoRA tests. - Drop the plugin loader in vllm/plugins. Platform resolution is now CUDA-only and the general/stat-logger plugin hooks are gone. - Delete the speech-to-text and realtime entrypoints, the scale-out render routes, and vllm/config/speech_to_text.py. GenerationTask is generate-only. - Cut DecodeBenchConnector and the disagg-prefill doc and image. The Multi, Offloading and Example connectors stay; production uses the CPU KV offload tier. - Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake arch gate, declarations and op registrations. The CUDA build never defined the guard and no Python caller existed. - Delete the orphan Mistral and Pixtral chat fixtures and examples/. - Fix _check_model: removing the LoRA early return made every request fail with "The model ... does not exist. Valid aliases: ...". Verified on an RTX 3090: import sweep reports 1242 modules with 7 known optional-dependency failures; a live server answers chat, completions, responses, tokenize and detokenize with zero log errors and 404 for the removed surfaces; collection yields 8558 tests with the 3 pre-existing errors; ruff adds no new finding; the incremental _C build succeeds. Co-authored-by: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -8,7 +8,6 @@ from transformers import AutoModelForSeq2SeqLM
|
||||
|
||||
from vllm.assets.audio import AudioAsset
|
||||
from vllm.envs import disable_envs_cache
|
||||
from vllm.lora.request import LoRARequest
|
||||
from vllm.multimodal.audio import AudioResampler
|
||||
|
||||
from ....conftest import HfRunner, VllmRunner
|
||||
@@ -26,7 +25,6 @@ AUDIO_MODEL_SETTINGS: dict[str, dict[str, Any]] = {
|
||||
"<|end_of_text|>\n"
|
||||
"<|start_of_role|>assistant<|end_of_role|>"
|
||||
),
|
||||
"audio_lora_path": "ibm-granite/granite-speech-3.3-2b",
|
||||
},
|
||||
"nvidia/audio-flamingo-3-hf": {
|
||||
"prompt": (
|
||||
@@ -90,7 +88,6 @@ def test_transformers_audio_generation(
|
||||
model_id: str,
|
||||
):
|
||||
settings = AUDIO_MODEL_SETTINGS[model_id]
|
||||
audio_lora_path = settings.get("audio_lora_path")
|
||||
|
||||
audio, orig_sr = AUDIO_ASSET.audio_and_sample_rate
|
||||
target_sr = settings.get("sampling_rate", orig_sr)
|
||||
@@ -105,19 +102,13 @@ def test_transformers_audio_generation(
|
||||
max_model_len=2048,
|
||||
enforce_eager=True,
|
||||
limit_mm_per_prompt={"audio": 1},
|
||||
enable_lora=audio_lora_path is not None,
|
||||
max_lora_rank=64,
|
||||
**settings.get("vllm_runner_kwargs", {}),
|
||||
) as vllm_model:
|
||||
lora_request = (
|
||||
LoRARequest("audio", 1, audio_lora_path) if audio_lora_path else None
|
||||
)
|
||||
vllm_outputs = vllm_model.generate_greedy_logprobs(
|
||||
[settings["prompt"]],
|
||||
128,
|
||||
num_logprobs=10,
|
||||
audios=[audio],
|
||||
lora_request=lora_request,
|
||||
)
|
||||
|
||||
with hf_runner(
|
||||
|
||||
Reference in New Issue
Block a user