[Bugfix] Fix Stream Sync for Shared Expert Overlap (#28430)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com> Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Co-authored-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
This commit is contained in:
co-authored by
Vadim Gimpelson
parent
bca74e32b7
commit
e605e8e323
@@ -3,6 +3,3 @@ accuracy_threshold: 0.45
|
||||
num_questions: 1319
|
||||
num_fewshot: 5
|
||||
max_model_len: 4096
|
||||
# Duo stream incompatabilbe with this model: https://github.com/vllm-project/vllm/issues/28220
|
||||
env:
|
||||
VLLM_DISABLE_SHARED_EXPERTS_STREAM: "1"
|
||||
|
||||
Reference in New Issue
Block a user