iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Bugfix] Enable inductor batch-invariant mode under VLLM_BATCH_INVARIANT (#61118)

Browse Source 
Signed-off-by: Yifan Feng <evanfengff@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
This commit is contained in:
yifanFengg
2026-10-17 14:53:06 -04:00
committed by GitHub
co-authored by Claude Opus 5.5 Wentao Ye Kevin H. Luu
parent 5c1c903501
commit 03e8300d2c
1 changed files with 3 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vllm/model_executor/determinism/batch_invariant.py
+3
View File
@@ -1424,6 +1424,9 @@ def override_envs_for_invariance():
# torch.compile settings
os.environ["VLLM_USE_AOT_COMPILE"] = "0"
import torch._inductor.config as inductor_config
inductor_config.batch_invariant = True
def init_batch_invariance():