[PLE] VLLM_E2_PLE_PULL_TRANSPORT defaults to on when unset

With the switch unset, the PLE took upstream's UVA backend, whose Triton lookup kernel needs fp8e4nv, which sm_86
does not have, so a launcher without the switch failed at profile_run. Unset now means on, the shipped value;
0 still selects upstream's path.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-09 20:15:11 +07:00
co-authored by Hinata
parent 72485816d9
commit b5f9fdbbac
2 changed files with 8 additions and 4 deletions
+5 -2
View File
@@ -395,7 +395,10 @@ if TYPE_CHECKING:
# UVA backend is refused (vllm/v1/e2_guards.py). Off: the tree is upstream.
# AOT: yes (different module class and forward). CAPTURE: yes (the captured
# graph contains the wait/copy ops instead of a table lookup). Hashed.
VLLM_E2_PLE_PULL_TRANSPORT: bool = False
# UNSET = ON (since v2.5), the shipped value: upstream's UVA backend (the off
# path) compiles an fp8e4nv Triton kernel that sm_86 cannot build, so a launcher
# without this switch failed at profile_run. "0" still selects upstream's path.
VLLM_E2_PLE_PULL_TRANSPORT: bool = True
#
# RUNTIME-ONLY (registered for validate_environ + a single cached read):
# Static K/V scale sidecar: fill_()ed into existing fp32 device buffers
@@ -2518,7 +2521,7 @@ environment_variables: dict[str, Callable[[], Any]] = {
"MERLIN_TOPK_EXACT_MEDIUM": lambda: os.getenv("MERLIN_TOPK_EXACT_MEDIUM", None),
# PLE pull transport switch; strict "0"/"1" (see the TYPE_CHECKING block).
"VLLM_E2_PLE_PULL_TRANSPORT": lambda: _e2_strict_switch(
"VLLM_E2_PLE_PULL_TRANSPORT"
"VLLM_E2_PLE_PULL_TRANSPORT", default=True
),
# Runtime only; all of the following are in compile_factors.ignored_factors.
"VLLM_E2_ALLOW_BREAKABLE": lambda: os.getenv("VLLM_E2_ALLOW_BREAKABLE", "") == "1",
+3 -2
View File
@@ -15,8 +15,9 @@ class: ``Qwen4ExpNGramEmbeddingPull`` inherits it and ``PleOffloadLayer``.
table is upstream's ``Qwen4ExpPLEDeviceEmbedding`` allocated on CPU;
``forward_impl`` computes n-gram IDs and gathers raw (FP8) rows.
Selected only when ``VLLM_E2_PLE_PULL_TRANSPORT=1``; with the switch off this
module is never imported by the model and the tree is upstream.
Selected when ``VLLM_E2_PLE_PULL_TRANSPORT`` is ``1`` or unset (the fork default
since v2.5); with ``=0`` this module is never imported by the model and the
tree is upstream.
Supported table formats: FP8 (global scale) and unquantized -- the two
``Qwen4ExpPLEEmbeddingMethod`` families v0.30.0 itself supports. The E2 NVFP4