[Config] The MERLIN_* kernel toggles default to the shipped value when unset

MERLIN_FULL_K, MERLIN_QSA_SORT, MERLIN_TIE_DET, MERLIN_TIE_RECENT and MERLIN_TOPK_SORTED_EMIT defaulted to off in
code while every shipped launcher set them to 1, so a launch without them ran a different kernel configuration.
vllm/env_override.py now sets each to 1 when unset; 0 still turns one off. tests/test_merlin_toggle_defaults.py
covers it.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-09 20:15:11 +07:00
co-authored by Hinata
parent b5f9fdbbac
commit 86d314050f
6 changed files with 26 additions and 10 deletions
@@ -49,7 +49,7 @@ namespace {
// Resolved ONCE per process, not per launch, so it is a launch constant in
// exactly the sense topk.cu's exact_medium_enabled() is: a cudagraph bakes in
// the value that was live at capture time and every replay agrees with it.
// Default OFF -- enabling it by default is a separate release decision.
// Code default off; vllm/env_override.py sets 1 (the shipped value) when unset.
//
// This std::getenv is the SINGLE source of truth. vllm/envs.py carries a raw
// passthrough entry for validate_environ and the compile-factor hash only;
+2 -2
View File
@@ -58,7 +58,7 @@ __device__ __forceinline__ auto convert_to_uint32_v2(float x) -> uint32_t {
// both the host (to report it) and the device (to obey it), so the reported
// policy and the executed policy cannot drift.
//
// `MERLIN_TIE_DET=0` is the shipped default and leaves the buffered paths on
// `MERLIN_TIE_DET=0` (the code default; launchers ship 1) leaves the buffered paths on
// their original racy atomicAdd rule, which has no direction; the medium-path exact
// restart is deterministic whatever happens, so it keeps the most-recent-wins
// direction the medium-path restart shipped and was measured with.
@@ -209,7 +209,7 @@ struct PersistentTopKParams {
// large tail) re-emit their OWN selected set through sorted_emit_row().
// Selection is never re-derived, so the selected set cannot change.
// Resolved once per process on the host, so it is a launch constant a
// cudagraph bakes in. Default OFF.
// cudagraph bakes in. Code default off; vllm/env_override.py sets 1 (the shipped value) when unset.
bool sorted_emit;
};
+4 -4
View File
@@ -39,7 +39,7 @@ namespace {
//
// Our own toggle rather than upstream's VLLM_BATCH_INVARIANT, which aborts
// engine startup on GDN models (upstream #42960) and so can never reach this
// kernel on Flash-Next. Default OFF -- promotion is a release decision.
// kernel on Flash-Next. Code default off; vllm/env_override.py sets 1 (the shipped value) when unset.
//
// This std::getenv is the SINGLE source of truth; vllm/envs.py carries a raw
// passthrough for validate_environ and the compile-factor hash only.
@@ -73,7 +73,7 @@ bool publish_cached(std::atomic<int>& mirror, bool value) {
// most-recent-wins is +1.5 %. Both deterministic. Also the direction
// medium_exact_restart() already implements, so setting this alongside
// MERLIN_TIE_DET makes the medium path agree with itself on overflowing
// rows. Only read when MERLIN_TIE_DET is set. Default OFF.
// rows. Only read when MERLIN_TIE_DET is set; code default OFF; vllm/env_override.py sets 1 (the shipped value) when unset.
bool tie_recent_enabled() {
static const bool enabled = [] {
const char* raw = std::getenv("MERLIN_TIE_RECENT");
@@ -149,8 +149,8 @@ bool exact_decode_enabled() {
// (ascending index, padding last as -1), so qsa_indexer.py can drop that sort.
// See sorted_emit_row() in persistent_topk.cuh for the contract; the selected
// SET is never touched. Resolved once per process like every switch here, so
// it is a launch constant a cudagraph bakes in. Default OFF: the maintainer
// promotes after reference-host qualification.
// it is a launch constant a cudagraph bakes in. Code default OFF; vllm/env_override.py
// sets 1 (the shipped value) when unset.
//
// This std::getenv is the single source of truth. Python does not parse the
// variable to decide anything: qsa_indexer.py asks THIS cache, through
+16
View File
@@ -182,6 +182,22 @@ os.environ.setdefault("TRITON_CACHE_AUTOTUNING", "1")
# debug layout (see https://github.com/vllm-project/vllm/issues/41410).
os.environ.setdefault("TILELANG_CLEANUP_TEMP_FILES", "1")
# Fork (v2.5, knob-absent boot): the MERLIN_* kernel toggles are read by
# getenv in C++ (FULL_K, TIE_DET, TIE_RECENT, TOPK_SORTED_EMIT) and by
# vllm.envs (QSA_SORT) with OFF as the code default, while every shipped
# launcher sets them to 1. Unset now means the shipped value; an explicit "0"
# still turns a toggle off. Set here because this module runs on `import vllm`,
# before any reader, and worker processes inherit the environment.
_SHIPPED_KERNEL_TOGGLE_DEFAULTS = (
"MERLIN_FULL_K",
"MERLIN_QSA_SORT",
"MERLIN_TIE_DET",
"MERLIN_TIE_RECENT",
"MERLIN_TOPK_SORTED_EMIT",
)
for _name in _SHIPPED_KERNEL_TOGGLE_DEFAULTS:
os.environ.setdefault(_name, "1")
# ===================================================
# torch 2.9 Inductor PythonWrapperCodegen monkeypatch
# ===================================================
+1 -1
View File
@@ -520,7 +520,7 @@ if TYPE_CHECKING:
# --- determinism toggles -----------------------------------------
# The three nondeterminism mechanisms (FULL_K, QSA_SORT, TIE_DET/TIE_RECENT).
# Every one of them is
# OFF by default: promotion is the maintainer's decision.
# OFF by default in this file; vllm/env_override.py sets each to 1 (the shipped value) when unset.
#
# Why these are registered at all: validate_environ() only inspects names
# beginning with VLLM_, so a MERLIN_ name never trips the unknown-variable
@@ -45,8 +45,8 @@ _DECODE_BLOCK_N = 64
# on the reference host and is why this version has no Python-visible branch on data.
#
# Resolved once at import so it is a launch constant: a cudagraph captures the
# op sequence that was live at capture time. Default OFF -- promotion is a release
# decision, which is the maintainer's.
# op sequence that was live at capture time. Code default OFF; vllm/env_override.py
# sets 1 (the shipped value) when unset.
# ---------------------------------------------------------------------------
_QSA_SORT = envs.MERLIN_QSA_SORT