[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions, rl, render-launcher and batch entrypoints; keep scale-out token routes - Drop the Swagger static bundle and the /docs, /redoc routes - Delete device-keyed tuning configs for fused_moe, quant utils, mamba select-state and helion; no sm86 entry ever existed for any of them - Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant quantizers and their config, metrics and loader plumbing - Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu requirements and the setup.py cpu target path - Delete 29 environment variables nothing reads and 14 chat-template fallbacks for models this fork no longer registers - Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark, dsv3 gemm) with their kernels, bindings, wrappers and CMake entries - Delete the connector test suites that targeted removed connectors - Cut the Mistral tokenizer and parser stack, its tests, and the mistral_common and ijson dependency pins - Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier cut deleted while keeping the model tree Co-authored-by: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
+3
-42
@@ -86,16 +86,7 @@ _REGISTERED_MODEL_CLASS_OVERRIDES: set[tuple[str, str]] = set()
|
||||
RunnerOption = Literal["auto", RunnerType]
|
||||
ConvertType = Literal["none", "embed", "classify"]
|
||||
ConvertOption = Literal["auto", ConvertType]
|
||||
TokenizerMode = Literal[
|
||||
"auto",
|
||||
"hf",
|
||||
"mistral",
|
||||
"deepseek_v32",
|
||||
"deepseek_v4",
|
||||
"inkling",
|
||||
"kimi_k3",
|
||||
"cohere",
|
||||
]
|
||||
TokenizerMode = Literal["auto", "hf"]
|
||||
ModelDType = Literal["auto", "half", "float16", "bfloat16", "float", "float32"]
|
||||
LogprobsMode = Literal[
|
||||
"raw_logits", "raw_logprobs", "processed_logits", "processed_logprobs"
|
||||
@@ -172,24 +163,14 @@ class ModelConfig:
|
||||
tokenizer_mode: TokenizerMode | str = "auto"
|
||||
"""Tokenizer mode:
|
||||
|
||||
- "auto" will use the tokenizer from `mistral_common` for Mistral models
|
||||
if available, otherwise it will use the "hf" tokenizer.
|
||||
- "auto" will use the "hf" tokenizer.
|
||||
- "hf" will use the fast tokenizer if available.
|
||||
- "mistral" will always use the tokenizer from `mistral_common`.
|
||||
- "deepseek_v32" will always use the tokenizer from `deepseek_v32`.
|
||||
- "deepseek_v4" will always use the tokenizer from `deepseek_v4`.
|
||||
- "deepseek_v41" will use the DeepSeek V4.1 prompt encoder.
|
||||
- "kimi_k3" will always use the "hf" tokenizer but render chat prompts
|
||||
with Kimi K3's Python XTML encoding instead of a Jinja template.
|
||||
- "cohere" uses the standard HF tokenizer but renders the chat template
|
||||
via the `cohere_melody` library (cmd3 / cmd4 templates) instead of
|
||||
Jinja, and surfaces grounded-citation metadata on responses.
|
||||
- Other custom values can be supported via plugins.
|
||||
|
||||
To swap the Rust BPE backend that powers HF fast tokenizers for the
|
||||
[fastokens](https://github.com/crusoecloud/fastokens) implementation, set
|
||||
`VLLM_USE_FASTOKENS=1` instead — that override applies to any mode that
|
||||
loads an HF fast tokenizer (`hf`, `deepseek_v32`, `deepseek_v4`, …)."""
|
||||
loads an HF fast tokenizer."""
|
||||
trust_remote_code: bool = False
|
||||
"""Trust remote code (e.g., from HuggingFace) when downloading the model
|
||||
and tokenizer."""
|
||||
@@ -267,8 +248,6 @@ class ModelConfig:
|
||||
(linear, moe) and ignore patterns; see :class:`QuantizationConfigArgs`.
|
||||
Auto-populated from the matching online shorthand when `quantization` is
|
||||
one of the values in `ONLINE_QUANT_SHORTHAND_NAMES`."""
|
||||
allow_deprecated_quantization: bool = False
|
||||
"""Whether to allow deprecated quantization methods."""
|
||||
enforce_eager: bool = False
|
||||
"""Whether to always use eager-mode PyTorch. If True, we will disable CUDA
|
||||
graph and always execute the model in eager mode. If False, we will use
|
||||
@@ -403,8 +382,6 @@ class ModelConfig:
|
||||
logits_processors: list[str | type[LogitsProcessor]] | None = None
|
||||
"""One or more logits processors' fully-qualified class names or class
|
||||
definitions"""
|
||||
io_processor_plugin: str | None = None
|
||||
"""IOProcessor plugin name to load at model startup"""
|
||||
renderer_num_workers: int = 1
|
||||
"""Number of worker threads in the renderer thread pool. The pool is
|
||||
consumed by the async renderer path (e.g. the OpenAI-compatible API
|
||||
@@ -483,7 +460,6 @@ class ModelConfig:
|
||||
"hf_token",
|
||||
"hf_overrides",
|
||||
"logits_processors",
|
||||
"io_processor_plugin",
|
||||
"pooler_config",
|
||||
"multimodal_config",
|
||||
"limit_mm_per_prompt",
|
||||
@@ -1440,21 +1416,6 @@ class ModelConfig:
|
||||
)
|
||||
current_platform.verify_quantization(self.quantization)
|
||||
|
||||
if self.quantization in me_quant.DEPRECATED_QUANTIZATION_METHODS:
|
||||
if self.allow_deprecated_quantization:
|
||||
logger.warning(
|
||||
"The quantization method %s is deprecated "
|
||||
"and will be removed in future versions of vLLM.",
|
||||
self.quantization,
|
||||
)
|
||||
else:
|
||||
raise ValueError(
|
||||
"The quantization method %s is deprecated "
|
||||
"and will be removed in future versions of vLLM. To bypass, "
|
||||
"set `--allow-deprecated-quantization`.",
|
||||
self.quantization,
|
||||
)
|
||||
|
||||
def _verify_cuda_graph(self) -> None:
|
||||
# CUDAGraph capture not supported for encoder-decoder models on ROCm
|
||||
unsupported_rocm = self.is_encoder_decoder
|
||||
|
||||
Reference in New Issue
Block a user