[Bugfix][Core] Reserve the KV null block when validating max_model_len (#47272)

Signed-off-by: hyungjun oh <92hyungjun@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
92hyungjun
2026-08-20 02:49:23 +00:00
committed by GitHub
co-authored by Claude Nick Hill
parent c0233fcf01
commit 76fb6d210a
6 changed files with 109 additions and 16 deletions
+6 -2
View File
@@ -77,8 +77,12 @@ def test_auto_fit_max_model_len_rejects_oversized_input(
# Use a small KV cache budget to force auto-fit to a small
# max_model_len. Pin block_size=16 so the budget is independent
# of the platform's default block size.
kv_cache_bytes = 1_000_000 # 1 MB
# of the platform's default block size. One block for this model is
# 2 (K/V) * 16 (block) * 12 (heads) * 64 (head_dim) * 2 (fp16)
# * 12 (layers) = 589,824 bytes, and the pool must cover at least
# two: one is reserved as the null block, so a 1 MB single-block
# pool cannot serve any tokens at all.
kv_cache_bytes = 2_000_000 # 2 MB = 3 blocks -> 2 usable
with vllm_runner(
model_name=model,