git am refreshes every file timestamp, so a Python-only patch series
forced a full extension rebuild and ninja saw everything as changed.
Diff csrc/ and the build files between the recorded tree and HEAD, and
rebuild only when a C++/CUDA source moved; otherwise the shipped .so
stays.
Update the expected tree to the 106-patch series
(c9c526755364d44d8d9923d9acc9d6d1a3d58718).
Add ipc: host, SYS_PTRACE, and seccomp=unconfined to the container so
a profiler or debugger can attach.
Raise --block-size from 2048 to 4096 and drop
--safetensors-load-strategy prefetch.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
MAX_SEQS and the prefill chunk were fixed in the start profile, so any
change to concurrency needed an edit to the script. Read both from .env,
and validate them before the engine starts. The chunk must be at least
the sequence count, the rule vLLM applies at scheduler.py:309. A
MAX_SEQS whose full context exceeds the 924,993-token pool prints a
note, because that shape preempts and recomputes instead of failing.
The captured graph list was a fixed [1,2,4,8,12,16], which covers a
decode batch of 4 requests and nothing wider. One decode step is K+1 = 4
tokens per request, so the list now follows MAX_SEQS: 1, 2, 4, then
every 4 tokens up to MAX_SEQS x 4. At the shipped 4 it produces the old
list exactly. Above 8 graphs a note prints, since each size costs
capture time and graph VRAM.
Drop --mm-processor-cache-gb. This workload sends each image once, so
the cache never had a hit to keep, and vLLM's own 4 GiB default is the
better base.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
The stats line carried throughput, KV usage and queue depth, but the
interval was never set. Unset, vLLM waits 10 s (vllm/envs.py:47), so a
quiet run looked idle for ten seconds and a short one printed nothing.
Export VLLM_LOG_STATS_INTERVAL in start.sh with a default of 1, and pass
it through Compose so .env can raise it back to 10. Add the variable to
.env.example, to the start.sh header, and to the Variables table in both
READMEs.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
The container read the sidecar only inside the MODEL_DIR mount, so a
scale file in another folder needed a manual -v and a container path.
Bind-mount the file named by SCALES to /opt/vllm/qsa_kv_scales.json, and
pass that fixed path to the engine. The default stays
$MODEL_DIR/qsa_kv_scales.json, so the shipped setup does not change. Add
a commented SCALES entry to .env.example.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
MOVZXandHinata