[Bugfix] Respect Inductor deterministic mode in combo-kernel defaults (#59074)
Signed-off-by: cheesecake <farzanaman99@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
OpenAI Codex
Claude Opus 5.5
parent
32fbfa15e8
commit
b9767299db
@@ -251,6 +251,18 @@ that may call 1+ triton kernels. On rare (but unfortunate) occasions, it may
|
||||
produce an incorrect triton kernel. This may manifest as silent incorrectness,
|
||||
CUDA illegal memory accesses, or loud errors.
|
||||
|
||||
### Deterministic compilation
|
||||
|
||||
When vLLM selects combo-kernel defaults, it disables combo-kernel benchmarking
|
||||
if Inductor's `deterministic` setting is enabled. A `deterministic` value in
|
||||
`inductor_compile_config` takes precedence over the global Inductor setting.
|
||||
On torch 2.13, `TORCHINDUCTOR_DETERMINISTIC=1` alone is reset by the first
|
||||
compiled frame ([pytorch#198563](https://github.com/pytorch/pytorch/issues/198563)),
|
||||
so set `deterministic` in `inductor_compile_config` instead.
|
||||
Explicit `combo_kernels` or `benchmark_combo_kernel` settings are preserved.
|
||||
This avoids conflicting compilation defaults; it does not guarantee reproducible
|
||||
model outputs.
|
||||
|
||||
### Inductor runtime assertions
|
||||
|
||||
By default (on torch < 2.12), vLLM disables Inductor's runtime assertions
|
||||
|
||||
Reference in New Issue
Block a user