[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
Production runs one static model behind the OpenAI-compatible API. Every adapter path and non-OpenAI HTTP surface is unreachable there. - Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora CLI group, and the LoRA tests. - Drop the plugin loader in vllm/plugins. Platform resolution is now CUDA-only and the general/stat-logger plugin hooks are gone. - Delete the speech-to-text and realtime entrypoints, the scale-out render routes, and vllm/config/speech_to_text.py. GenerationTask is generate-only. - Cut DecodeBenchConnector and the disagg-prefill doc and image. The Multi, Offloading and Example connectors stay; production uses the CPU KV offload tier. - Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake arch gate, declarations and op registrations. The CUDA build never defined the guard and no Python caller existed. - Delete the orphan Mistral and Pixtral chat fixtures and examples/. - Fix _check_model: removing the LoRA early return made every request fail with "The model ... does not exist. Valid aliases: ...". Verified on an RTX 3090: import sweep reports 1242 modules with 7 known optional-dependency failures; a live server answers chat, completions, responses, tokenize and detokenize with zero log errors and 404 for the removed surfaces; collection yields 8558 tests with the 3 pre-existing errors; ruff adds no new finding; the incremental _C build succeeds. Co-authored-by: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
@@ -245,12 +245,6 @@ class ServingGenerativeScoring(BaseServing):
|
||||
# Note: Mixed item types (string and token list) are validated by
|
||||
# Pydantic at request parsing time, so we don't need to check here.
|
||||
|
||||
try:
|
||||
lora_request = self._maybe_get_adapters(request) # type: ignore[arg-type]
|
||||
except (ValueError, TypeError, RuntimeError) as e:
|
||||
logger.exception("Error preparing request components")
|
||||
return self.create_error_response(e)
|
||||
|
||||
base_id = self._base_request_id(raw_request, default=request.request_id)
|
||||
request_id = f"generative-scoring-{base_id}"
|
||||
created_time = int(time.time())
|
||||
@@ -290,7 +284,6 @@ class ServingGenerativeScoring(BaseServing):
|
||||
f"{request_id}-{i}",
|
||||
engine_input,
|
||||
params=sampling_params,
|
||||
lora_request=lora_request,
|
||||
)
|
||||
|
||||
try:
|
||||
@@ -299,7 +292,6 @@ class ServingGenerativeScoring(BaseServing):
|
||||
engine_inputs,
|
||||
[sampling_params] * len(engine_inputs),
|
||||
request_id,
|
||||
lora_request=lora_request,
|
||||
trace_headers=trace_headers,
|
||||
priority=request.priority,
|
||||
)
|
||||
@@ -343,7 +335,7 @@ class ServingGenerativeScoring(BaseServing):
|
||||
total_completion_tokens += len(read.result.outputs[0].token_ids)
|
||||
|
||||
# Build response
|
||||
model_name = self.models.model_name(lora_request)
|
||||
model_name = self.models.model_name()
|
||||
response = GenerativeScoringResponse(
|
||||
id=request_id,
|
||||
created=created_time,
|
||||
|
||||
Reference in New Issue
Block a user