feat: Mount the KV scale sidecar from any host path

The container read the sidecar only inside the MODEL_DIR mount, so a
scale file in another folder needed a manual -v and a container path.

Bind-mount the file named by SCALES to /opt/vllm/qsa_kv_scales.json, and
pass that fixed path to the engine. The default stays
$MODEL_DIR/qsa_kv_scales.json, so the shipped setup does not change. Add
a commented SCALES entry to .env.example.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-15 18:52:18 +00:00
co-authored by Hinata
parent 1281bf9c03
commit b89e85e99f
2 changed files with 4 additions and 1 deletions
+2
View File
@@ -7,6 +7,8 @@ VLLM_API_KEY=
MODEL_DIR=/data/models/halt95/Swift1.5-Qwen3.8-Flash-Next-W4A16-Merlin
MODEL_MOUNT_PATH=/models/Qwen3.8-Flash-Next
MODEL_NAME=Hinata
# The FP8 KV scale sidecar, a host path. Default: $MODEL_DIR/qsa_kv_scales.json.
# SCALES=
# ── image and container ──────────────────────────────────────────────────────
IMAGE=qwen38-flash-next