feat: Mount the KV scale sidecar from any host path
The container read the sidecar only inside the MODEL_DIR mount, so a scale file in another folder needed a manual -v and a container path. Bind-mount the file named by SCALES to /opt/vllm/qsa_kv_scales.json, and pass that fixed path to the engine. The default stays $MODEL_DIR/qsa_kv_scales.json, so the shipped setup does not change. Add a commented SCALES entry to .env.example. Co-Authored-By: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
@@ -7,6 +7,8 @@ VLLM_API_KEY=
|
||||
MODEL_DIR=/data/models/halt95/Swift1.5-Qwen3.8-Flash-Next-W4A16-Merlin
|
||||
MODEL_MOUNT_PATH=/models/Qwen3.8-Flash-Next
|
||||
MODEL_NAME=Hinata
|
||||
# The FP8 KV scale sidecar, a host path. Default: $MODEL_DIR/qsa_kv_scales.json.
|
||||
# SCALES=
|
||||
|
||||
# ── image and container ──────────────────────────────────────────────────────
|
||||
IMAGE=qwen38-flash-next
|
||||
|
||||
Reference in New Issue
Block a user