[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
Production runs one static model behind the OpenAI-compatible API. Every adapter path and non-OpenAI HTTP surface is unreachable there. - Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora CLI group, and the LoRA tests. - Drop the plugin loader in vllm/plugins. Platform resolution is now CUDA-only and the general/stat-logger plugin hooks are gone. - Delete the speech-to-text and realtime entrypoints, the scale-out render routes, and vllm/config/speech_to_text.py. GenerationTask is generate-only. - Cut DecodeBenchConnector and the disagg-prefill doc and image. The Multi, Offloading and Example connectors stay; production uses the CPU KV offload tier. - Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake arch gate, declarations and op registrations. The CUDA build never defined the guard and no Python caller existed. - Delete the orphan Mistral and Pixtral chat fixtures and examples/. - Fix _check_model: removing the LoRA early return made every request fail with "The model ... does not exist. Valid aliases: ...". Verified on an RTX 3090: import sweep reports 1242 modules with 7 known optional-dependency failures; a live server answers chat, completions, responses, tokenize and detokenize with zero log errors and 404 for the removed surfaces; collection yields 8558 tests with the 3 pre-existing errors; ruff adds no new finding; the incremental _C build succeeds. Co-authored-by: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
@@ -20,12 +20,8 @@ __all__ = ["get_parameter_for_reload", "reload_weights"]
|
||||
|
||||
def get_parameter_for_reload(model: nn.Module, name: str) -> nn.Parameter:
|
||||
"""Resolve checkpoint names without changing the model's module tree."""
|
||||
from vllm.lora.layers import BaseLayerWithLoRA
|
||||
|
||||
module_name, _, parameter_name = name.rpartition(".")
|
||||
module = model.get_submodule(module_name)
|
||||
if isinstance(module, BaseLayerWithLoRA):
|
||||
module = module.base_layer
|
||||
return module.get_parameter(parameter_name)
|
||||
|
||||
|
||||
@@ -39,7 +35,7 @@ def reload_weights(
|
||||
"""Reload a model's weights in place from a weights iterator or from disk.
|
||||
|
||||
Callers are responsible for invalidating any state derived from the old
|
||||
weights (e.g. LoRA, encoder and multimodal caches).
|
||||
weights (e.g. encoder and multimodal caches).
|
||||
|
||||
Args:
|
||||
vllm_config: config of the model being reloaded. If `weights_path` is
|
||||
@@ -63,10 +59,7 @@ def reload_weights(
|
||||
|
||||
model_config = vllm_config.model_config
|
||||
load_config = vllm_config.load_config
|
||||
weights_to_load = {
|
||||
name.replace(".base_layer.", ".") if vllm_config.lora_config else name
|
||||
for name, _ in model.named_parameters()
|
||||
}
|
||||
weights_to_load = {name for name, _ in model.named_parameters()}
|
||||
counter_before_reloading = time.perf_counter()
|
||||
|
||||
# load weights from disk if none are provided
|
||||
|
||||
@@ -217,8 +217,8 @@ class ShardedStateLoader(BaseModelLoader):
|
||||
state_dict = self._filter_subtensors(model.state_dict())
|
||||
counter_before_loading_weights = time.perf_counter()
|
||||
for key, tensor in self.iterate_over_files(filepaths):
|
||||
# If loading with LoRA enabled, additional padding may
|
||||
# be added to certain parameters. We only load into a
|
||||
# Parameters padded beyond their checkpoint tensors
|
||||
# may carry extra trailing rows. We only load into a
|
||||
# narrowed view of the parameter data.
|
||||
param = state_dict[key]
|
||||
if param.dtype != tensor.dtype:
|
||||
|
||||
@@ -119,8 +119,6 @@ def initialize_model(
|
||||
kwargs["cache_config"] = vllm_config.cache_config
|
||||
if "quant_config" in all_params:
|
||||
kwargs["quant_config"] = vllm_config.quant_config
|
||||
if "lora_config" in all_params:
|
||||
kwargs["lora_config"] = vllm_config.lora_config
|
||||
if "scheduler_config" in all_params:
|
||||
kwargs["scheduler_config"] = vllm_config.scheduler_config
|
||||
with set_current_vllm_config(vllm_config, check_compile=True, prefix=prefix):
|
||||
|
||||
Reference in New Issue
Block a user