[LoRA] Support modules_to_save for sequence classification (#53555)

Signed-off-by: linitra24 <Joy25810@foxmail.com>
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
This commit is contained in:
linitra24
2026-09-16 07:15:42 -07:00
committed by GitHub
co-authored by Jee Jee Li
parent ec4a3a5370
commit c81eb3e69c
20 changed files with 681 additions and 21 deletions
+12
View File
@@ -422,6 +422,18 @@ vllm serve ibm-granite/granite-speech-3.3-2b \
Note: Default multimodal LoRAs are currently only available for `.generate` and chat completions.
## Sequence-Classification LoRA Adapters
vLLM supports PEFT sequence-classification adapters that save a complete, single-layer linear classification head through `modules_to_save`. The saved module must be named `score` or `classifier`.
See [classification_with_lora_offline.py](../../examples/pooling/classify/classification_with_lora_offline.py) for an offline classification example using a LoRA adapter.
This support has the following limitations:
- All adapters in one engine must have the same `num_labels` and hidden size as the base classification head. An incompatible adapter is rejected when it is loaded.
- A classification head stored as float32 is converted to the runtime head dtype when it is loaded.
- Token-classification adapters are not supported by this feature.
## Using Tips
### Configuring `max_lora_rank`