[Worker] Debug-gated diagnostics for Mamba slot ownership and guard encoding

- Host-side Mamba slot-ownership check (off|warn|raise, default off) with a
  failing control that proves it detects a wrong block-table row.
- Guard telemetry: int64 encoding limits and the extreme-value pair closed by
  test; a decoded-None counter field is now named in the warning; a
  captured-replay sentinel probe.
- Both knobs registered in envs.py.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-09 19:44:27 +07:00
co-authored by Hinata
parent 754fb3e29d
commit 31ad7e113f
+48
View File
@@ -615,6 +615,30 @@ def _resolve_rust_cli_path() -> str | None:
return raw
def _e2_diagnostic_mode(name: str) -> str:
"""Normalize an E2 diagnostic gate to off|warn|raise.
One normalizer for every knob in the family, so a registry entry and the
module that acts on it cannot disagree about what the environment said.
``1`` is accepted as ``warn`` for continuity with the earlier boolean
gates. A set-but-unrecognised value is a boot error, not a silent ``off``:
a broken gate that disarms its own instrument is the failure mode these
diagnostics exist to avoid.
"""
# An EMPTY value means off, not a boot error. Shell scripts and pytest
# fixtures both use `VAR=` to mean "unset" (see tests/v1/worker/
# test_e2_memstats.py), and rejecting it would turn a routine way of
# clearing the variable into a crash.
value = os.environ.get(name, "off").strip().lower() or "off"
if value == "1":
return "warn"
if value not in ("off", "warn", "raise"):
raise ValueError(
f"{name} must be off|warn|raise (or 1 for warn), got {value!r}"
)
return value
environment_variables: dict[str, Callable[[], Any]] = {
# ================== Installation Time Env Vars ==================
# Target device of vLLM, supporting [cuda (by default),
@@ -2193,6 +2217,23 @@ environment_variables: dict[str, Callable[[], Any]] = {
"VLLM_ELASTIC_EP_DRAIN_REQUESTS": lambda: bool(
int(os.getenv("VLLM_ELASTIC_EP_DRAIN_REQUESTS", "0"))
),
# PLE diagnostics. Both are off|warn|raise (1 is accepted as warn) and both
# default to off. They are registered here so the registry is the single
# source of truth for their normalization -- the modules that use them read
# these getters rather than calling os.getenv a second time, so the value
# the registry reports cannot drift from the value the check applies.
# Both are excluded from compile factors in `compile_factors` below: they
# gate HOST-side Python outside any capture and add no node to a compiled
# graph. If either ever grows a device-side component, remove it from
# `ignored_factors` in the same change.
#
# Read the pull-transport PLE semaphore back before a real launch, on the
# publishing rank only (vllm#53899 one-step-lag class).
"VLLM_E2_PLE_FLAG_CHECK": lambda: _e2_diagnostic_mode("VLLM_E2_PLE_FLAG_CHECK"),
# Host-side align row-rule and state-block collision check.
"VLLM_E2_MAMBA_SLOT_OWNERSHIP": lambda: _e2_diagnostic_mode(
"VLLM_E2_MAMBA_SLOT_OWNERSHIP"
),
# If set to 1, enable CUDA graph memory estimation during memory profiling.
# This profiles CUDA graph memory usage to provide more accurate KV cache
# memory allocation. Enabled by default as of v0.21.0
@@ -2340,6 +2381,13 @@ def compile_factors() -> dict[str, object]:
"VLLM_DEBUG_DUMP_PATH",
"VLLM_PORT",
"VLLM_CACHE_ROOT",
# PLE host-side diagnostics: both gate Python that runs outside any
# capture and neither adds a node to a compiled graph, so hashing them
# would invalidate every compile cache the moment a diagnostic boot is
# taken -- which is exactly when the cache is wanted. Remove from this
# set if either ever grows a device-side component.
"VLLM_E2_PLE_FLAG_CHECK",
"VLLM_E2_MAMBA_SLOT_OWNERSHIP",
# Runtime memory-plan persistence; does not affect compiled graphs.
"VLLM_ENABLE_STARTUP_PLAN",
# Location-only derived paths: where a cache/config directory lives