The fused pre-indexer could read a stale or uncommitted raw-key ring row and
pass its int64 tail to RoPE as a position (illegal address). The writer now
tags each row with logical_position + 1; the reader checks the tag of every
source row and, on a mismatch, masks the pooled load, the RoPE-tail read and
the compressed-slot store. New guard site 11 (always fatal under warn). The
KV pool size is unchanged.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Landed 0027 with mamba_utils.py and guard_telemetry.py under the
ours-newer rule; that rule saw the slot-ownership check and the
diagnostic gate, not the bias-encoding helpers. decode_slot(),
encode_slot(), GUARD_MAX_ENCODABLE and GUARD_MIN_ENCODABLE are new
names, present in v2.5.1 and absent here, and patch 0049's ring-row
report calls decode_slot(). Replays 0027's guard_telemetry.py section
(3 hunks, patch -F 8) onto the current tree.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
A schema walk fails on any pointer-like int parameter of a custom op on the
PLE path. The FULL_K test child calls the Marlin MoE GEMM by keyword against
its v0.30.0 signature.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Each worker logs its launch identity and, after warm-up and capture, one
e2-toggles line with the effective toggle state read back from the C++
extensions through two new tensor-free query ops. Both extensions must be
rebuilt.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Compile-time defines (default 0, device code unchanged by default) for the
delayed-publication stress, the pre-repair control and a forced-fault cell;
test cells for the OFF-arm controls, the QSA_SORT contract, FULL_K attribution
on the Marlin MoE GEMM and legacy-tail repeated-row membership.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
- Counter enum declared above the decode template (two-phase lookup).
- Emission-fault flag set atomically.
- Close the legacy large tie loop with a block barrier (inherited).
- Large-path fault injectors registered as hashed passthroughs.
- The tensor-free counter-slots op registered for CompositeExplicitAutograd.
- Tie measurements recorded only on rows neither fault verdict failed.
- Tests scoped to the properties the build promises.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
- MERLIN_FULL_K: deterministic Marlin MoE split-K (after unmerged upstream #46639).
- MERLIN_QSA_SORT: canonical order for the QSA selected blocks.
- MERLIN_TIE_DET: deterministic tie-break at the persistent_topk cutoff.
- MERLIN_TIE_RECENT: cutoff ties keep the highest indices.
- Decode candidate truncation is detected; one tie direction across the
medium path; the kernel switches are registered.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
- Inline-PTX acquire/release helpers declare their memory effects.
- The fault verdict is aggregated across the CTA group before emission, so a
faulted row is failed closed by every CTA.
- 64-bit device counters and an int64 reader (counter protocol v2).
- Tests: adversarial distributions, slot-residue coverage, an sm_90 gate for
tests that need it.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Above RADIX_THRESHOLD a row is split across a group of CTAs. The pivot was
already order-independent; cross-CTA placement, intra-CTA placement and tie
membership were arrival-order races. They are now deterministic behind
MERLIN_TOPK_DET_LARGE (default on).
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
In radix_topk's large-path round loop the leader published arrival with no
block barrier after the histogram merge, so a peer CTA could observe an
incomplete histogram. Inherited by earlier releases; fixed below the placement
work so both arms of any comparison carry it.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Reads persistent_topk_exact_counters into {slot: value}, logs one line on a
cadence, and degrades to a one-time warning when the op is absent.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
The wait/copy op took host pointers as Python-int arguments, which were baked
into the AOT compile cache as the tracing process's addresses; the first
forward after a cache-load boot then failed. The op now carries only
process-stable values and resolves pointers from a per-process registry.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
A KV cache group with no layer on a pipeline stage was still given tensors
there, and allocate_kv_cache raised StopIteration on the second stage. The
fix was lost in the v0.30.0 rebase. Test drives the real get_kv_cache_configs
with the production layout.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
With the wait/copy inside a compiled piece, Inductor's post-grad
auto-functionalization decomposition failed on the stage-0 ranks.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
- torch.compile restored on the NVIDIA Qwen4Exp target and MTP so the
production profile (VLLM_COMPILE + FULL_AND_PIECEWISE, breakable off) runs.
- The production pin requires VLLM_USE_BREAKABLE_CUDAGRAPH to be exactly "0".
- assert_resolved_cudagraph_modes() runs at the top of capture_model.
- CPU tests for the combined profile, the exact pin and the resolved modes.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
- VLLM_E2_PLE_PULL_TRANSPORT replaces the fork's old switch name, which
v0.30.0 reuses for its UVA backend with the opposite default; strict parser;
compile/capture factor.
- The pull transport is reachable on v0.30.0 through a pull embedding class
that inherits upstream's without editing it; the upstream UVA backend is
provably off while the pull switch is on.
- resolve_ple_mode()/assert_ple_mode() log and assert the effective PLE mode
for target and drafter, and refuse a mixed configuration.
- Producer log lines for the PLE backend and weight dtype.
- CPU tests for mode resolution and the mixed-config refusal.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Env-gated diagnostics, inert unless set. Allocator telemetry is read only at
step boundaries, armed after warm-up and capture.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Restores the fork's PLE offload package (not upstream) and its pull
transport: the offload process holds no CUDA context; the captured op waits on
a device-mapped host flag and copies host->GPU on the model stream; publication
is fenced; releases are synchronous across dummy/capture work; failures close
the rank. Includes the test-only fault-injection cuts for the transport.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
New vllm/v1/e2_guards.py, called from the target and MTP constructors on every
rank before profiling or capture: the breakable-cudagraph pin is asserted
(three readings of the switch), a graph profile that cannot execute on v0.30.0
is refused, and VLLM_E2_EXPECTED_CUDAGRAPH_MODE is checked by name. Tests
cover the drafter. An inert knob is no longer hashed into the cache key.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
- Drop the PLE-offload half of ple_layer.py that upstream #54371 deleted and a
3-way merge resurrected (it no longer imported on v0.30.0).
- Drop the structured-output helper duplicate that v0.30.0 already provides,
and an unused import.
- Re-sort the import blocks the merges left unsorted (ruff I001).
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
- Every fork knob is registered in envs.py and classified as a compile/capture
factor or runtime-only; envs.py documents that capture identity and AOT
cache identity are different things.
- ruff findings cleared on lines this fork authored; upstream lines untouched.
- A `gpu` pytest marker so the GPU-free tier can be selected, not just skipped.
- Duplicate knob-table entries from two branches reconciled onto one
off|warn|raise normaliser.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
- Host-side Mamba slot-ownership check (off|warn|raise, default off) with a
failing control that proves it detects a wrong block-table row.
- Guard telemetry: int64 encoding limits and the extreme-value pair closed by
test; a decoded-None counter field is now named in the warning; a
captured-replay sentinel probe.
- Both knobs registered in envs.py.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
When the persistent top-k medium-path candidate buffer overflows, the row is
restarted exactly instead of being truncated. Capture-safe counters with atomic
publication and a coherent snapshot record every restart.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Ports upstream vLLM PRs #54442 and #56802 (issue #54437): never sample from an
unmasked row, and key draft-token hand-off snapshots by scheduler step so an
alternating PP microbatch cannot overwrite the slot.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Env-gated per-step log of blocks allocated and freed and scheduled tokens per
request; prefix-cache hits are read behind a schema check so an upstream field
rename fails visibly.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
A circular-buffer ring block could be freed while an earlier, still-executing
step was reading it. The free is deferred until the overlapping step has
committed.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Both callers of the guarded kernel (fused recurrent and KDA) are updated in the
same commit.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Under pipeline parallelism the Mamba/GDN state-copy path indexed a block-table
row gathered for a different step. The align post-process now indexes the
persistent per-request-slot table, and fails closed when it is absent.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Keeps the token embedding table in pinned host memory behind a UVA view for the
stage-0 target and the last-stage MTP drafter, freeing GPU memory for KV. TP
sharding, masking and the all-reduce are retained. The knob is registered as a
compile-cache factor.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Workers set PR_SET_PDEATHSIG(SIGKILL) against their EngineCore parent with a
getppid race check. Start methods other than fork/spawn are refused before any
worker is created. Subprocess tests cover fork, spawn, graceful SIGTERM and the
refusal.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
A stdlib-only, one-shot, rank-selective fault selector shared by the PLE
offload transport and its tests. Inert unless armed: disarmed, every site is a
single None check outside any compiled or captured region.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Each pipeline stage builds only the parts it owns: PLE when every PLE layer
lives on stage 0, MTP under PP (port of vLLM PR #46994 minus its ROCm MLA file
and tests) with the drafter on the last stage, and the vision tower on the
first stage only.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
FP8 E4M3 KV read path for the Qwen4Exp sparse attention (QSA), indexer and
pre-indexer kernels on sm_86, dequantised with a static per-layer scale loaded
from a JSON sidecar, validated against the model's QSA layer set and applied
per pipeline stage. Includes the offline absmax collector that produces the
sidecar and a CUDA-graph-safe clipping counter.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
Shared machinery for device-side bounds guards: persistent int64 counters per
site, written by guarded kernels and read by a background thread. The reader
starts only after warm-up and CUDA-graph capture and never waits on the
compute stream; a read failure arms the stop rather than popping the buffer.
Shutdown does one synchronous final drain so a hit on the last step is
reported before teardown. Release default: off.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
MOVZXandHinata
