[Misc][Spec Decode] support different load config for draft model (#34022)
Signed-off-by: zzhengkai <zzhengkai@devgpu049.ldc1.facebook.com> Co-authored-by: zzhengkai <zzhengkai@devgpu049.ldc1.facebook.com>
This commit is contained in:
co-authored by
zzhengkai
parent
bb2fc8b5e7
commit
6f2f59f2b3
@@ -1286,6 +1286,7 @@ class SpecDecodeBaseProposer:
|
||||
model = get_model(
|
||||
vllm_config=self.vllm_config,
|
||||
model_config=self.speculative_config.draft_model_config,
|
||||
load_config=self.speculative_config.draft_load_config,
|
||||
)
|
||||
return model
|
||||
|
||||
|
||||
Reference in New Issue
Block a user