[Bugfix][LoRA] Validate incompatible MoE adapter formats (#59924)

Signed-off-by: Brian Weston <brianweston@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
This commit is contained in:
Brian Weston, PhD
2026-10-09 09:49:13 +08:00
committed by GitHub
co-authored by Codex Jee Jee Li linitra24
parent 581692cc0f
commit 201e537d98
4 changed files with 96 additions and 4 deletions
+9 -4
View File
@@ -280,10 +280,10 @@ curl -X POST http://localhost:8000/v1/load_lora_adapter \
!!! warning "You must know your adapter's layout"
Under `--enable-mixed-moe-lora-format`, vLLM trusts whatever
`is_3d_lora_weight` the caller declares — it does **not** inspect the
checkpoint to verify. A wrong declaration will load weights into the
wrong stacked buffers and silently produce garbage outputs, with no
error at load time. Confirm the layout before serving:
`is_3d_lora_weight` the caller declares. Loading checks reject fused 3D
expert weights routed to a 2D wrapper without the required flags, but
do not validate every layout mismatch. A wrong declaration can still
cause loading failures or incorrect outputs. Confirm the layout before serving:
- **2D (per-expert, megatron-style)** → set `is_3d_lora_weight: false`.
Adapter keys look like `...experts.{idx}.gate_proj.lora_A.weight`,
@@ -299,6 +299,11 @@ is ignored: vLLM picks the wrapper from the base model's
`is_3d_moe_weight` and the adapter is required to match. The field is
also ignored for non-MoE models.
For a model using a 2D MoE wrapper (such as Qwen3-MoE), loading fused 3D
expert adapter weights requires both `enable_mixed_moe_lora_format=True`
on the engine and `is_3d_lora_weight=True` on the request. Otherwise, adapter
loading raises a `ValueError` naming these settings before activation.
## LoRA model lineage in model card
The new format of `--lora-modules` is mainly to support the display of parent model information in the model card. Here's an explanation of how your current response supports this: