[Bugfix] Fix Stream Sync for Shared Expert Overlap (#28430)

Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
This commit is contained in:
Robert Shaw
2025-11-11 05:59:08 +00:00
committed by GitHub
co-authored by Vadim Gimpelson
parent bca74e32b7
commit e605e8e323
2 changed files with 15 additions and 33 deletions
@@ -3,6 +3,3 @@ accuracy_threshold: 0.45
num_questions: 1319
num_fewshot: 5
max_model_len: 4096
# Duo stream incompatabilbe with this model: https://github.com/vllm-project/vllm/issues/28220
env:
VLLM_DISABLE_SHARED_EXPERTS_STREAM: "1"