The stats line carried throughput, KV usage and queue depth, but the
interval was never set. Unset, vLLM waits 10 s (vllm/envs.py:47), so a
quiet run looked idle for ten seconds and a short one printed nothing.
Export VLLM_LOG_STATS_INTERVAL in start.sh with a default of 1, and pass
it through Compose so .env can raise it back to 10. Add the variable to
.env.example, to the start.sh header, and to the Variables table in both
READMEs.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
The container read the sidecar only inside the MODEL_DIR mount, so a
scale file in another folder needed a manual -v and a container path.
Bind-mount the file named by SCALES to /opt/vllm/qsa_kv_scales.json, and
pass that fixed path to the engine. The default stays
$MODEL_DIR/qsa_kv_scales.json, so the shipped setup does not change. Add
a commented SCALES entry to .env.example.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
MOVZXandHinata