Three faults ended a re-run before step 8, so a deleted artifacts/ was
never written again.
The import check read the first word of stdout, and vLLM logs to stdout.
On a box with mixed 3090 and 3090 Ti devices the device-mismatch WARNING
comes first, so the build stopped with "vllm reports WARNING, expected
0.30.0". Print a BUILD_SMOKE marker and read the version from that line.
Patch 0103 committed .build-products.sha256 into the serving tree, and
write_sums rewrites that file, so every re-run stopped at "modified
tracked files". Drop the file from the patch and exclude the two build
record paths from the check. The series now yields tree
a823fa36d3fd640f87b3de87153268d938130a47; OPS_OBJ is unchanged, so the
compiled kernels stay valid.
Step 8 skipped the pack whenever the kernels came from a tarball, and
judged an existing set by file presence alone. Require the tarball, the
per-file list, the manifest and the sidecar, verify the sidecar, and
pack from the products in the tree when any check fails.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
git am refreshes every file timestamp, so a Python-only patch series
forced a full extension rebuild and ninja saw everything as changed.
Diff csrc/ and the build files between the recorded tree and HEAD, and
rebuild only when a C++/CUDA source moved; otherwise the shipped .so
stays.
Update the expected tree to the 106-patch series
(c9c526755364d44d8d9923d9acc9d6d1a3d58718).
Add ipc: host, SYS_PTRACE, and seccomp=unconfined to the container so
a profiler or debugger can attach.
Raise --block-size from 2048 to 4096 and drop
--safetensors-load-strategy prefetch.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
The vision tower sat on GPU 0 and 1 all the time, and did work only
when a request carried an image. It now streams from pinned host RAM
and encodes over H2D, which frees 432 MiB on GPU 0 and 1. Stage 1
carries no tower, so GPU 2 and 3 gain nothing, and the KV cache stays
at 924,993 tokens because --kv-cache-memory pins it. The freed memory
is headroom, not capacity. An image request passes: 2223 image tokens
in 11.77 s, with no VRAM growth.
Key the prebuilt ops cache on the compiled inputs, so a Python-only
patch installs from the shipped tarball instead of a 20-40 minute
compile.
Raise --mm-processor-cache-gb to 16. That cache is duplicated for each
API process and engine core process, so 16 GiB costs 32 GiB of host
RAM and no VRAM, and repeated images stop re-processing.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
MOVZXandHinata