MOVZXandHinata
5aa2ac64f8
Production runs one static model behind the OpenAI-compatible API. Every adapter path and non-OpenAI HTTP surface is unreachable there. - Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora CLI group, and the LoRA tests. - Drop the plugin loader in vllm/plugins. Platform resolution is now CUDA-only and the general/stat-logger plugin hooks are gone. - Delete the speech-to-text and realtime entrypoints, the scale-out render routes, and vllm/config/speech_to_text.py. GenerationTask is generate-only. - Cut DecodeBenchConnector and the disagg-prefill doc and image. The Multi, Offloading and Example connectors stay; production uses the CPU KV offload tier. - Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake arch gate, declarations and op registrations. The CUDA build never defined the guard and no Python caller existed. - Delete the orphan Mistral and Pixtral chat fixtures and examples/. - Fix _check_model: removing the LoRA early return made every request fail with "The model ... does not exist. Valid aliases: ...". Verified on an RTX 3090: import sweep reports 1242 modules with 7 known optional-dependency failures; a live server answers chat, completions, responses, tokenize and detokenize with zero log errors and 404 for the removed surfaces; collection yields 8558 tests with the 3 pre-existing errors; ruff adds no new finding; the incremental _C build succeeds. Co-authored-by: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
68 lines
1.6 KiB
Python
68 lines
1.6 KiB
Python
# SPDX-License-Identifier: Apache-2.0
|
|
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
|
|
|
import pytest
|
|
|
|
|
|
@pytest.fixture
|
|
def sample_prompts():
|
|
return [
|
|
"Hello, my name is",
|
|
"The president of the United States is",
|
|
"The capital of France is",
|
|
"The future of AI is",
|
|
]
|
|
|
|
|
|
@pytest.fixture
|
|
def sample_regex():
|
|
return (
|
|
r"((25[0-5]|(2[0-4]|1\d|[1-9]|)\d)\.){3}"
|
|
r"(25[0-5]|(2[0-4]|1\d|[1-9]|)\d)"
|
|
)
|
|
|
|
|
|
@pytest.fixture
|
|
def sample_json_schema():
|
|
return {
|
|
"type": "object",
|
|
"properties": {
|
|
"name": {"type": "string"},
|
|
"age": {"type": "integer"},
|
|
"skills": {
|
|
"type": "array",
|
|
"items": {"type": "string", "maxLength": 10},
|
|
"minItems": 3,
|
|
},
|
|
"work_history": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "object",
|
|
"properties": {
|
|
"company": {"type": "string"},
|
|
"duration": {"type": "number"},
|
|
"position": {"type": "string"},
|
|
},
|
|
"required": ["company", "position"],
|
|
},
|
|
},
|
|
},
|
|
"required": ["name", "age", "skills", "work_history"],
|
|
}
|
|
|
|
|
|
@pytest.fixture
|
|
def sample_structured_outputs_choices():
|
|
return [
|
|
"Python",
|
|
"Java",
|
|
"JavaScript",
|
|
"C++",
|
|
"C#",
|
|
"PHP",
|
|
"TypeScript",
|
|
"Ruby",
|
|
"Swift",
|
|
"Kotlin",
|
|
]
|