Deepseek-v3 Batch Invariant on 8xH100 (#26609)

Signed-off-by: Bram Wasti <bwasti@meta.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
This commit is contained in:
Bram Wasti
2025-10-15 22:06:02 -07:00
committed by GitHub
co-authored by Wentao Ye
parent 785d8b6410
commit 7d8975de84
21 changed files with 1567 additions and 102 deletions
+1 -1
View File
@@ -1694,7 +1694,7 @@ class EngineArgs:
) -> None:
"""Set Default Arguments for V1 Engine."""
# V1 always uses chunked prefills and prefix caching
# V1 uses chunked prefills and prefix caching by default
# for non-pooling tasks.
# For pooling tasks the default is False
if model_config.runner_type != "pooling":