MOVZXandHinata
b1fdbde895
Three faults ended a re-run before step 8, so a deleted artifacts/ was never written again. The import check read the first word of stdout, and vLLM logs to stdout. On a box with mixed 3090 and 3090 Ti devices the device-mismatch WARNING comes first, so the build stopped with "vllm reports WARNING, expected 0.30.0". Print a BUILD_SMOKE marker and read the version from that line. Patch 0103 committed .build-products.sha256 into the serving tree, and write_sums rewrites that file, so every re-run stopped at "modified tracked files". Drop the file from the patch and exclude the two build record paths from the check. The series now yields tree a823fa36d3fd640f87b3de87153268d938130a47; OPS_OBJ is unchanged, so the compiled kernels stay valid. Step 8 skipped the pack whenever the kernels came from a tarball, and judged an existing set by file presence alone. Require the tarball, the per-file list, the manifest and the sidecar, verify the sidecar, and pack from the products in the tree when any check fails. Co-Authored-By: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
58 lines
2.8 KiB
Diff
58 lines
2.8 KiB
Diff
From 975db60922a1f8d0a38752724c13f09b6d229c1c Mon Sep 17 00:00:00 2001
|
|
From: MOVZX <movzx@yahoo.com>
|
|
Date: Sun, 11 Oct 2026 05:04:35 +0000
|
|
Subject: [PATCH 103/103] qwen4_exp: abliteration always registers the buffer
|
|
(zeros when off)
|
|
|
|
register() used to set a plain attribute None when QWEN38_ABLITERATION was
|
|
off. The AOT compile cache keeps graphs compiled with the switch on, and
|
|
those graphs resolve _abliteration_r as a registered buffer, so an off boot
|
|
on a warm cache died with KeyError: '_abliteration_r'. Register the buffer
|
|
unconditionally, filled with zeros when the switch is off: projecting along
|
|
a zero direction is an exact no-op, ON and OFF compile to the same graph,
|
|
and toggling the switch never invalidates the compile cache.
|
|
|
|
Co-Authored-By: Hinata <hinata@ihoree.local>
|
|
---
|
|
vllm/models/qwen4_exp/nvidia/abliteration.py | 14 +-
|
|
1 file changed, 6 insertions(+), 8 deletions(-)
|
|
|
|
diff --git a/vllm/models/qwen4_exp/nvidia/abliteration.py b/vllm/models/qwen4_exp/nvidia/abliteration.py
|
|
index 840168b..6ce0528 100644
|
|
--- a/vllm/models/qwen4_exp/nvidia/abliteration.py
|
|
+++ b/vllm/models/qwen4_exp/nvidia/abliteration.py
|
|
@@ -59,24 +59,22 @@ def load_direction(hidden_size: int) -> torch.Tensor | None:
|
|
|
|
|
|
def register(module: torch.nn.Module, hidden_size: int, dtype: torch.dtype) -> None:
|
|
- """Attach the direction as a non-persistent buffer ``_abliteration_r`` (None when the switch is off).
|
|
+ """Attach the direction as a non-persistent buffer ``_abliteration_r`` (all zeros when the switch is off).
|
|
|
|
Call inside the model constructor so the buffer lands on the model's device before CUDA-graph capture. The
|
|
- CPU-only PLE worker process never runs these forwards and gets no buffer.
|
|
+ buffer always exists, so a compiled graph compiled with the switch on resolves when it runs with the switch
|
|
+ off: a zero direction makes the projection an exact no-op. The CPU-only PLE worker process never runs these
|
|
+ forwards and gets the zero buffer.
|
|
"""
|
|
from vllm.model_executor.layers.ple_offload_layer import is_offload_process
|
|
|
|
r = None if is_offload_process() else load_direction(hidden_size)
|
|
- if r is None:
|
|
- module._abliteration_r = None
|
|
- return
|
|
- buf = torch.empty(hidden_size, dtype=dtype)
|
|
- buf.copy_(r)
|
|
+ buf = torch.zeros(hidden_size, dtype=dtype) if r is None else r.to(dtype).flatten().contiguous()
|
|
module.register_buffer("_abliteration_r", buf, persistent=False)
|
|
|
|
|
|
def project_(x: torch.Tensor, r: torch.Tensor | None) -> torch.Tensor:
|
|
- """In place: remove the component along r from x's last dimension. No-op when r is None."""
|
|
+ """In place: remove the component along r from x's last dimension. No-op when r is None or all zeros."""
|
|
if r is None:
|
|
return x
|
|
width = x.shape[-1]
|
|
--
|
|
2.43.0
|
|
|