[Misc][LoRA] Support Rank Stabilized LoRA (RSLoRA) (#6909)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
This commit is contained in:
John Giorgi
2024-12-30 22:15:58 -08:00
committed by GitHub
co-authored by Jee Jee Li
parent 74fa1d123c
commit 82c49d3260
4 changed files with 30 additions and 22 deletions
+1 -1
View File
@@ -173,7 +173,7 @@ class LoRAModel(AdapterModel):
return cls(lora_model_id,
peft_helper.r,
loras,
scaling_factor=peft_helper.vllm_scaling_factor)
scaling_factor=peft_helper.vllm_long_context_scaling_factor)
@classmethod
def from_local_checkpoint(