iHore/vLLM-4x-RTX-3090
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Set the engine stats interval to 1 s

Browse Source 
The stats line carried throughput, KV usage and queue depth, but the
interval was never set. Unset, vLLM waits 10 s (vllm/envs.py:47), so a
quiet run looked idle for ten seconds and a short one printed nothing.

Export VLLM_LOG_STATS_INTERVAL in start.sh with a default of 1, and pass
it through Compose so .env can raise it back to 10. Add the variable to
.env.example, to the start.sh header, and to the Variables table in both
READMEs.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
MOVZX
2026-10-16 06:01:27 +00:00
co-authored by Hinata
parent c4d88acb4d
commit 3a0a5d1510
5 changed files with 43 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README-ID.md
+18 -17
View File
@@ -277,23 +277,24 @@ Tanpa Compose, mount sendiri sidecar-nya: `-v <host>/scales.json:/opt/vllm/qsa_k
## 9. Variabel
| Variabel | Bawaan | Arti |
| ---------------------- | -------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `MODEL_DIR` | tidak ada, wajib | Direktori host tempat checkpoint. Compose berhenti dengan pesan ketika kosong. |
| `CKPT` | `/data/models/halt95/Swift1.5-Qwen3.8-Flash-Next-W4A16-Merlin` | Folder checkpoint di host. Argumen pertama `start.sh` mengaturnya. |
| `MODEL_MOUNT_PATH` | `/models/Qwen3.8-Flash-Next` | Path checkpoint di dalam container. Compose mem-mount `MODEL_DIR` ke sini. |
| `SCALES` | `$MODEL_DIR/qsa_kv_scales.json` | Path host sidecar scale KV FP8. Compose mem-mount-nya ke container. |
| `MODEL_NAME` | `Hinata` (compose), `qwen38-flash-next` (`start.sh`) | Nama di field `model` pada request. Spasi membuat alias. |
| `HOST_PORT` | `8080` | Port host untuk API. |
| `VLLM_API_KEY` | kosong | Kosong berarti tanpa autentikasi. |
| `CACHE_HOST_DIR` | `.vllm-cache` di direktori ini | Tempat cache compile berada. Jalur host memakai folder yang sama. |
| `CUDA_VISIBLE_DEVICES` | `0,1,2,3` | Jalur GPU, dalam urutan bus. |
| `PLE_HOME` | `0` | Kartu yang menjadi inang PLE offload. Kartu itu tidak mendapat CUDA context untuk itu. |
| `QWEN38_ABLITERATION` | `Qwen3.8-Flash-Next-Uncensored` | Hapus arah refusal saat jalan. Kosong menonaktifkannya. Lihat `patches/0102`. |
| `ABLITERATION_DIR` | `./abliteration` | Folder host dengan file arah itu. Compose mem-mount ke `/opt/qwen38/abliteration`. |
| `GUARD` | `warn` | Telemetri bounds-guard: `off`, `warn`, atau `raise`. |
| `COUNTERS` | `0` | `1` menambah counter kernel khusus log. |
| `MAX_JOBS`, `ARCH` | `32`, `8.6` | Job compile dan arsitektur GPU, hanya untuk build. |
| Variabel | Bawaan | Arti |
| ------------------------- | -------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `MODEL_DIR` | tidak ada, wajib | Direktori host tempat checkpoint. Compose berhenti dengan pesan ketika kosong. |
| `CKPT` | `/data/models/halt95/Swift1.5-Qwen3.8-Flash-Next-W4A16-Merlin` | Folder checkpoint di host. Argumen pertama `start.sh` mengaturnya. |
| `MODEL_MOUNT_PATH` | `/models/Qwen3.8-Flash-Next` | Path checkpoint di dalam container. Compose mem-mount `MODEL_DIR` ke sini. |
| `SCALES` | `$MODEL_DIR/qsa_kv_scales.json` | Path host sidecar scale KV FP8. Compose mem-mount-nya ke container. |
| `MODEL_NAME` | `Hinata` (compose), `qwen38-flash-next` (`start.sh`) | Nama di field `model` pada request. Spasi membuat alias. |
| `HOST_PORT` | `8080` | Port host untuk API. |
| `VLLM_API_KEY` | kosong | Kosong berarti tanpa autentikasi. |
| `CACHE_HOST_DIR` | `.vllm-cache` di direktori ini | Tempat cache compile berada. Jalur host memakai folder yang sama. |
| `CUDA_VISIBLE_DEVICES` | `0,1,2,3` | Jalur GPU, dalam urutan bus. |
| `PLE_HOME` | `0` | Kartu yang menjadi inang PLE offload. Kartu itu tidak mendapat CUDA context untuk itu. |
| `QWEN38_ABLITERATION` | `Qwen3.8-Flash-Next-Uncensored` | Hapus arah refusal saat jalan. Kosong menonaktifkannya. Lihat `patches/0102`. |
| `ABLITERATION_DIR` | `./abliteration` | Folder host dengan file arah itu. Compose mem-mount ke `/opt/qwen38/abliteration`. |
| `GUARD` | `warn` | Telemetri bounds-guard: `off`, `warn`, atau `raise`. |
| `COUNTERS` | `0` | `1` menambah counter kernel khusus log. |
| `VLLM_LOG_STATS_INTERVAL` | `1` | Jeda antar baris stats engine, dalam detik. Default vLLM: 10 s. |
| `MAX_JOBS`, `ARCH` | `32`, `8.6` | Job compile dan arsitektur GPU, hanya untuk build. |
## 10. Benchmark