[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack

- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-19 17:10:56 +07:00
co-authored by Hinata
parent bd70a61c20
commit 2066d3143f
1086 changed files with 122 additions and 291919 deletions
+3 -42
View File
@@ -86,16 +86,7 @@ _REGISTERED_MODEL_CLASS_OVERRIDES: set[tuple[str, str]] = set()
RunnerOption = Literal["auto", RunnerType]
ConvertType = Literal["none", "embed", "classify"]
ConvertOption = Literal["auto", ConvertType]
TokenizerMode = Literal[
"auto",
"hf",
"mistral",
"deepseek_v32",
"deepseek_v4",
"inkling",
"kimi_k3",
"cohere",
]
TokenizerMode = Literal["auto", "hf"]
ModelDType = Literal["auto", "half", "float16", "bfloat16", "float", "float32"]
LogprobsMode = Literal[
"raw_logits", "raw_logprobs", "processed_logits", "processed_logprobs"
@@ -172,24 +163,14 @@ class ModelConfig:
tokenizer_mode: TokenizerMode | str = "auto"
"""Tokenizer mode:
- "auto" will use the tokenizer from `mistral_common` for Mistral models
if available, otherwise it will use the "hf" tokenizer.
- "auto" will use the "hf" tokenizer.
- "hf" will use the fast tokenizer if available.
- "mistral" will always use the tokenizer from `mistral_common`.
- "deepseek_v32" will always use the tokenizer from `deepseek_v32`.
- "deepseek_v4" will always use the tokenizer from `deepseek_v4`.
- "deepseek_v41" will use the DeepSeek V4.1 prompt encoder.
- "kimi_k3" will always use the "hf" tokenizer but render chat prompts
with Kimi K3's Python XTML encoding instead of a Jinja template.
- "cohere" uses the standard HF tokenizer but renders the chat template
via the `cohere_melody` library (cmd3 / cmd4 templates) instead of
Jinja, and surfaces grounded-citation metadata on responses.
- Other custom values can be supported via plugins.
To swap the Rust BPE backend that powers HF fast tokenizers for the
[fastokens](https://github.com/crusoecloud/fastokens) implementation, set
`VLLM_USE_FASTOKENS=1` instead — that override applies to any mode that
loads an HF fast tokenizer (`hf`, `deepseek_v32`, `deepseek_v4`, …)."""
loads an HF fast tokenizer."""
trust_remote_code: bool = False
"""Trust remote code (e.g., from HuggingFace) when downloading the model
and tokenizer."""
@@ -267,8 +248,6 @@ class ModelConfig:
(linear, moe) and ignore patterns; see :class:`QuantizationConfigArgs`.
Auto-populated from the matching online shorthand when `quantization` is
one of the values in `ONLINE_QUANT_SHORTHAND_NAMES`."""
allow_deprecated_quantization: bool = False
"""Whether to allow deprecated quantization methods."""
enforce_eager: bool = False
"""Whether to always use eager-mode PyTorch. If True, we will disable CUDA
graph and always execute the model in eager mode. If False, we will use
@@ -403,8 +382,6 @@ class ModelConfig:
logits_processors: list[str | type[LogitsProcessor]] | None = None
"""One or more logits processors' fully-qualified class names or class
definitions"""
io_processor_plugin: str | None = None
"""IOProcessor plugin name to load at model startup"""
renderer_num_workers: int = 1
"""Number of worker threads in the renderer thread pool. The pool is
consumed by the async renderer path (e.g. the OpenAI-compatible API
@@ -483,7 +460,6 @@ class ModelConfig:
"hf_token",
"hf_overrides",
"logits_processors",
"io_processor_plugin",
"pooler_config",
"multimodal_config",
"limit_mm_per_prompt",
@@ -1440,21 +1416,6 @@ class ModelConfig:
)
current_platform.verify_quantization(self.quantization)
if self.quantization in me_quant.DEPRECATED_QUANTIZATION_METHODS:
if self.allow_deprecated_quantization:
logger.warning(
"The quantization method %s is deprecated "
"and will be removed in future versions of vLLM.",
self.quantization,
)
else:
raise ValueError(
"The quantization method %s is deprecated "
"and will be removed in future versions of vLLM. To bypass, "
"set `--allow-deprecated-quantization`.",
self.quantization,
)
def _verify_cuda_graph(self) -> None:
# CUDAGraph capture not supported for encoder-decoder models on ROCm
unsupported_rocm = self.is_encoder_decoder