[Guards] The bounds-guard telemetry code default is warn

VLLM_E2_GUARD_MODE now defaults to warn in code, matching what the shipped launchers set; v2 through v2.2.0
defaulted to off. The comment above it, the MODES docstring, the vllm/envs.py default and its annotation follow.
tests/v1/worker/gpu/test_guard_default_mode.py checks that code, comment and envs.py agree, from the sources, so
it runs without CUDA, and fails on the earlier source.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-09 20:04:39 +07:00
co-authored by Hinata
parent 036608b012
commit ef55e095c3
2 changed files with 14 additions and 12 deletions
+2 -2
View File
@@ -405,7 +405,7 @@ if TYPE_CHECKING:
# Guard telemetry: mode raise|warn|off and the reader poll interval in # Guard telemetry: mode raise|warn|off and the reader poll interval in
# seconds. counter() returns a real tensor in every mode, so the Triton # seconds. counter() returns a real tensor in every mode, so the Triton
# arguments and the captured graph do not depend on either. # arguments and the captured graph do not depend on either.
VLLM_E2_GUARD_MODE: str = "off" VLLM_E2_GUARD_MODE: str = "warn"
VLLM_E2_GUARD_INTERVAL: str = "0.05" VLLM_E2_GUARD_INTERVAL: str = "0.05"
# Debug allocator telemetry (poll seconds + output directory) and the # Debug allocator telemetry (poll seconds + output directory) and the
# scheduler allocation log path. # scheduler allocation log path.
@@ -2507,7 +2507,7 @@ environment_variables: dict[str, Callable[[], Any]] = {
).strip(), ).strip(),
"VLLM_QSA_KV_SCALES": lambda: os.getenv("VLLM_QSA_KV_SCALES", "").strip(), "VLLM_QSA_KV_SCALES": lambda: os.getenv("VLLM_QSA_KV_SCALES", "").strip(),
"VLLM_E2_GUARD_MODE": lambda: ( "VLLM_E2_GUARD_MODE": lambda: (
os.getenv("VLLM_E2_GUARD_MODE", "off").strip().lower() os.getenv("VLLM_E2_GUARD_MODE", "warn").strip().lower()
), ),
"VLLM_E2_GUARD_INTERVAL": lambda: os.getenv("VLLM_E2_GUARD_INTERVAL", "0.05"), "VLLM_E2_GUARD_INTERVAL": lambda: os.getenv("VLLM_E2_GUARD_INTERVAL", "0.05"),
"VLLM_E2_MEMSTATS": lambda: os.getenv("VLLM_E2_MEMSTATS", "0"), "VLLM_E2_MEMSTATS": lambda: os.getenv("VLLM_E2_MEMSTATS", "0"),
+12 -10
View File
@@ -40,13 +40,13 @@ MODES -- VLLM_E2_GUARD_MODE:
serving product than a loud stop. Read check_pending below before serving product than a loud stop. Read check_pending below before
using it -- the raise is late and it is local to one rank. using it -- the raise is late and it is local to one rank.
Targeted repro boots. Targeted repro boots.
warn log only -- EXCEPT for the sites in _ALWAYS_FATAL_SITES, which arm warn (DEFAULT since v2.5) log only -- EXCEPT for the sites in
the raise whatever the mode says, because their masked value has _ALWAYS_FATAL_SITES, which arm the raise whatever the mode says,
already been written somewhere durable. USE THIS FOR LONG STRESS because their masked value has already been written somewhere durable.
BOOTS. USE THIS FOR LONG STRESS BOOTS.
off (DEFAULT since v2: the reader thread off (the default in v2 through v2.2.0: the reader thread and the QSA clip
and the QSA clip reader cost ~0.3 ms per decode step on TP2xPP2; opt in reader cost ~0.3 ms per decode step on TP2xPP2; set
with VLLM_E2_GUARD_MODE=warn for diagnosis) VLLM_E2_GUARD_MODE=off to skip the reader)
no reader thread. The kernels still mask and still count; nothing is no reader thread. The kernels still mask and still count; nothing is
read -- INCLUDING the sites in _ALWAYS_FATAL_SITES. Use only when you read -- INCLUDING the sites in _ALWAYS_FATAL_SITES. Use only when you
have decided you do not want the evidence. have decided you do not want the evidence.
@@ -218,10 +218,12 @@ SITE_NAMES = {
} }
# Release defaults: off, 0.05 s. # Release defaults: warn, 0.05 s.
# Earlier revisions shipped raise / 0.5 while every qualification boot set both # Earlier revisions shipped raise / 0.5 while every qualification boot set both
# variables explicitly, so the defaults were never the thing being tested. # variables explicitly, so the defaults were never the thing being tested.
_MODE = os.environ.get("VLLM_E2_GUARD_MODE", "off").strip().lower() # v2 through v2.2.0 shipped the code default off; v2.5 puts
# it back to warn so the code default, this comment and the shipped launchers agree.
_MODE = os.environ.get("VLLM_E2_GUARD_MODE", "warn").strip().lower()
if _MODE not in ("raise", "warn", "off"): if _MODE not in ("raise", "warn", "off"):
# fail closed: a set-but-broken gate must not silently disarm the instrument # fail closed: a set-but-broken gate must not silently disarm the instrument
raise ValueError(f"VLLM_E2_GUARD_MODE must be raise|warn|off, got {_MODE!r}") raise ValueError(f"VLLM_E2_GUARD_MODE must be raise|warn|off, got {_MODE!r}")
@@ -491,7 +493,7 @@ def start_reader() -> None:
host, and a device-to-host copy from a second thread while a capture is host, and a device-to-host copy from a second thread while a capture is
active is exactly the call class cudaStreamCaptureModeGlobal -- the default active is exactly the call class cudaStreamCaptureModeGlobal -- the default
for torch.cuda.graph -- rejects. The QSA clip-counter reader survives it in for torch.cuda.graph -- rejects. The QSA clip-counter reader survives it in
practice, but that one is maintainer-gated and this one can be enabled by any user, so practice, but that one is opt-in and this one is on by default, so
the precedent is evidence, not proof. Starting after capture deletes the the precedent is evidence, not proof. Starting after capture deletes the
question rather than arguing it. question rather than arguing it.