[LoRA] Initial EP support for LoRA (#40867)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
This commit is contained in:
Jee Jee Li
2026-05-09 00:31:23 -07:00
committed by GitHub
parent d6563d693c
commit ecd0b60aad
13 changed files with 209 additions and 73 deletions
+1
View File
@@ -129,6 +129,7 @@ def test_gpt_oss_lora_tp2(
tensor_parallel_size=2,
gpu_memory_utilization=0.8,
fully_sharded_loras=fully_sharded_loras,
enable_expert_parallel=not fully_sharded_loras,
compilation_config=vllm.config.CompilationConfig( # Avoid OOM
cudagraph_specialize_lora=False,
),