MOVZXandHinata
5aa2ac64f8
Production runs one static model behind the OpenAI-compatible API. Every adapter path and non-OpenAI HTTP surface is unreachable there. - Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora CLI group, and the LoRA tests. - Drop the plugin loader in vllm/plugins. Platform resolution is now CUDA-only and the general/stat-logger plugin hooks are gone. - Delete the speech-to-text and realtime entrypoints, the scale-out render routes, and vllm/config/speech_to_text.py. GenerationTask is generate-only. - Cut DecodeBenchConnector and the disagg-prefill doc and image. The Multi, Offloading and Example connectors stay; production uses the CPU KV offload tier. - Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake arch gate, declarations and op registrations. The CUDA build never defined the guard and no Python caller existed. - Delete the orphan Mistral and Pixtral chat fixtures and examples/. - Fix _check_model: removing the LoRA early return made every request fail with "The model ... does not exist. Valid aliases: ...". Verified on an RTX 3090: import sweep reports 1242 modules with 7 known optional-dependency failures; a live server answers chat, completions, responses, tokenize and detokenize with zero log errors and 404 for the removed surfaces; collection yields 8558 tests with the 3 pre-existing errors; ruff adds no new finding; the incremental _C build succeeds. Co-authored-by: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
45 lines
1.0 KiB
Python
45 lines
1.0 KiB
Python
# SPDX-License-Identifier: Apache-2.0
|
|
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
|
|
|
from .interfaces import (
|
|
HasInnerState,
|
|
SupportsHCSP,
|
|
SupportsMRoPE,
|
|
SupportsMultiModal,
|
|
SupportsMultiModalEmbeddings,
|
|
SupportsPP,
|
|
has_inner_state,
|
|
supports_hc_sp,
|
|
supports_mrope,
|
|
supports_multimodal,
|
|
supports_multimodal_embeddings,
|
|
supports_pp,
|
|
)
|
|
from .interfaces_base import (
|
|
VllmModelForPooling,
|
|
VllmModelForTextGeneration,
|
|
is_pooling_model,
|
|
is_text_generation_model,
|
|
)
|
|
from .registry import ModelRegistry
|
|
|
|
__all__ = [
|
|
"ModelRegistry",
|
|
"VllmModelForPooling",
|
|
"is_pooling_model",
|
|
"VllmModelForTextGeneration",
|
|
"is_text_generation_model",
|
|
"HasInnerState",
|
|
"has_inner_state",
|
|
"SupportsHCSP",
|
|
"supports_hc_sp",
|
|
"SupportsMultiModal",
|
|
"supports_multimodal",
|
|
"SupportsMultiModalEmbeddings",
|
|
"supports_multimodal_embeddings",
|
|
"SupportsMRoPE",
|
|
"supports_mrope",
|
|
"SupportsPP",
|
|
"supports_pp",
|
|
]
|