[PLE] VLLM_E2_PLE_PULL_TRANSPORT defaults to on when unset
With the switch unset, the PLE took upstream's UVA backend, whose Triton lookup kernel needs fp8e4nv, which sm_86 does not have, so a launcher without the switch failed at profile_run. Unset now means on, the shipped value; 0 still selects upstream's path. Co-Authored-By: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
+5
-2
@@ -395,7 +395,10 @@ if TYPE_CHECKING:
|
||||
# UVA backend is refused (vllm/v1/e2_guards.py). Off: the tree is upstream.
|
||||
# AOT: yes (different module class and forward). CAPTURE: yes (the captured
|
||||
# graph contains the wait/copy ops instead of a table lookup). Hashed.
|
||||
VLLM_E2_PLE_PULL_TRANSPORT: bool = False
|
||||
# UNSET = ON (since v2.5), the shipped value: upstream's UVA backend (the off
|
||||
# path) compiles an fp8e4nv Triton kernel that sm_86 cannot build, so a launcher
|
||||
# without this switch failed at profile_run. "0" still selects upstream's path.
|
||||
VLLM_E2_PLE_PULL_TRANSPORT: bool = True
|
||||
#
|
||||
# RUNTIME-ONLY (registered for validate_environ + a single cached read):
|
||||
# Static K/V scale sidecar: fill_()ed into existing fp32 device buffers
|
||||
@@ -2518,7 +2521,7 @@ environment_variables: dict[str, Callable[[], Any]] = {
|
||||
"MERLIN_TOPK_EXACT_MEDIUM": lambda: os.getenv("MERLIN_TOPK_EXACT_MEDIUM", None),
|
||||
# PLE pull transport switch; strict "0"/"1" (see the TYPE_CHECKING block).
|
||||
"VLLM_E2_PLE_PULL_TRANSPORT": lambda: _e2_strict_switch(
|
||||
"VLLM_E2_PLE_PULL_TRANSPORT"
|
||||
"VLLM_E2_PLE_PULL_TRANSPORT", default=True
|
||||
),
|
||||
# Runtime only; all of the following are in compile_factors.ignored_factors.
|
||||
"VLLM_E2_ALLOW_BREAKABLE": lambda: os.getenv("VLLM_E2_ALLOW_BREAKABLE", "") == "1",
|
||||
|
||||
@@ -15,8 +15,9 @@ class: ``Qwen4ExpNGramEmbeddingPull`` inherits it and ``PleOffloadLayer``.
|
||||
table is upstream's ``Qwen4ExpPLEDeviceEmbedding`` allocated on CPU;
|
||||
``forward_impl`` computes n-gram IDs and gathers raw (FP8) rows.
|
||||
|
||||
Selected only when ``VLLM_E2_PLE_PULL_TRANSPORT=1``; with the switch off this
|
||||
module is never imported by the model and the tree is upstream.
|
||||
Selected when ``VLLM_E2_PLE_PULL_TRANSPORT`` is ``1`` or unset (the fork default
|
||||
since v2.5); with ``=0`` this module is never imported by the model and the
|
||||
tree is upstream.
|
||||
|
||||
Supported table formats: FP8 (global scale) and unquantized -- the two
|
||||
``Qwen4ExpPLEEmbeddingMethod`` families v0.30.0 itself supports. The E2 NVFP4
|
||||
|
||||
Reference in New Issue
Block a user