[Guards] The bounds-guard telemetry code default is warn

VLLM_E2_GUARD_MODE now defaults to warn in code, matching what the shipped launchers set; v2 through v2.2.0
defaulted to off. The comment above it, the MODES docstring, the vllm/envs.py default and its annotation follow.
tests/v1/worker/gpu/test_guard_default_mode.py checks that code, comment and envs.py agree, from the sources, so
it runs without CUDA, and fails on the earlier source.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-09 20:04:39 +07:00
co-authored by Hinata
parent 036608b012
commit ef55e095c3
2 changed files with 14 additions and 12 deletions
+2 -2
View File
@@ -405,7 +405,7 @@ if TYPE_CHECKING:
# Guard telemetry: mode raise|warn|off and the reader poll interval in
# seconds. counter() returns a real tensor in every mode, so the Triton
# arguments and the captured graph do not depend on either.
VLLM_E2_GUARD_MODE: str = "off"
VLLM_E2_GUARD_MODE: str = "warn"
VLLM_E2_GUARD_INTERVAL: str = "0.05"
# Debug allocator telemetry (poll seconds + output directory) and the
# scheduler allocation log path.
@@ -2507,7 +2507,7 @@ environment_variables: dict[str, Callable[[], Any]] = {
).strip(),
"VLLM_QSA_KV_SCALES": lambda: os.getenv("VLLM_QSA_KV_SCALES", "").strip(),
"VLLM_E2_GUARD_MODE": lambda: (
os.getenv("VLLM_E2_GUARD_MODE", "off").strip().lower()
os.getenv("VLLM_E2_GUARD_MODE", "warn").strip().lower()
),
"VLLM_E2_GUARD_INTERVAL": lambda: os.getenv("VLLM_E2_GUARD_INTERVAL", "0.05"),
"VLLM_E2_MEMSTATS": lambda: os.getenv("VLLM_E2_MEMSTATS", "0"),
+12 -10
View File
@@ -40,13 +40,13 @@ MODES -- VLLM_E2_GUARD_MODE:
serving product than a loud stop. Read check_pending below before
using it -- the raise is late and it is local to one rank.
Targeted repro boots.
warn log only -- EXCEPT for the sites in _ALWAYS_FATAL_SITES, which arm
the raise whatever the mode says, because their masked value has
already been written somewhere durable. USE THIS FOR LONG STRESS
BOOTS.
off (DEFAULT since v2: the reader thread
and the QSA clip reader cost ~0.3 ms per decode step on TP2xPP2; opt in
with VLLM_E2_GUARD_MODE=warn for diagnosis)
warn (DEFAULT since v2.5) log only -- EXCEPT for the sites in
_ALWAYS_FATAL_SITES, which arm the raise whatever the mode says,
because their masked value has already been written somewhere durable.
USE THIS FOR LONG STRESS BOOTS.
off (the default in v2 through v2.2.0: the reader thread and the QSA clip
reader cost ~0.3 ms per decode step on TP2xPP2; set
VLLM_E2_GUARD_MODE=off to skip the reader)
no reader thread. The kernels still mask and still count; nothing is
read -- INCLUDING the sites in _ALWAYS_FATAL_SITES. Use only when you
have decided you do not want the evidence.
@@ -218,10 +218,12 @@ SITE_NAMES = {
}
# Release defaults: off, 0.05 s.
# Release defaults: warn, 0.05 s.
# Earlier revisions shipped raise / 0.5 while every qualification boot set both
# variables explicitly, so the defaults were never the thing being tested.
_MODE = os.environ.get("VLLM_E2_GUARD_MODE", "off").strip().lower()
# v2 through v2.2.0 shipped the code default off; v2.5 puts
# it back to warn so the code default, this comment and the shipped launchers agree.
_MODE = os.environ.get("VLLM_E2_GUARD_MODE", "warn").strip().lower()
if _MODE not in ("raise", "warn", "off"):
# fail closed: a set-but-broken gate must not silently disarm the instrument
raise ValueError(f"VLLM_E2_GUARD_MODE must be raise|warn|off, got {_MODE!r}")
@@ -491,7 +493,7 @@ def start_reader() -> None:
host, and a device-to-host copy from a second thread while a capture is
active is exactly the call class cudaStreamCaptureModeGlobal -- the default
for torch.cuda.graph -- rejects. The QSA clip-counter reader survives it in
practice, but that one is maintainer-gated and this one can be enabled by any user, so
practice, but that one is opt-in and this one is on by default, so
the precedent is evidence, not proof. Starting after capture deletes the
question rather than arguing it.