[Metrics] Add group-aware KV cache capacity to vllm:cache_config_info (#42206)

The startup log already reports the correct group-aware KV cache capacity for
hybrid models, but Prometheus did not expose matching info in 'vllm:cache_config_info`.

This PR adds kv_cache_size_tokens and kv_cache_max_concurrency.

Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
This commit is contained in:
Ethan Feng
2026-06-12 11:49:44 +00:00
committed by GitHub
parent 8af550b399
commit b7f9b6ab27
7 changed files with 75 additions and 26 deletions
+14 -2
View File
@@ -720,14 +720,26 @@ class MPClient(EngineCoreClient):
)
# Setup KV cache config with initialization state from
# engine core process. Sum values from all engines in DP case.
# engine core process. Sum num_gpu_blocks from all engines in DP case.
num_gpu_blocks = vllm_config.cache_config.num_gpu_blocks or 0
num_gpu_blocks += response.num_gpu_blocks
vllm_config.cache_config.num_gpu_blocks = num_gpu_blocks
# Sync block_size: may be enlarged by _align_hybrid_block_size in the
# worker for hybrid Mamba models.
vllm_config.cache_config.block_size = response.block_size
cache_config = vllm_config.cache_config
cache_config.block_size = response.block_size
# Keep these as per-engine cache_config_info values; do not sum across DP.
cache_config.kv_cache_size_tokens = (
getattr(cache_config, "kv_cache_size_tokens", None)
if getattr(cache_config, "kv_cache_size_tokens", None) is not None
else response.kv_cache_size_tokens
)
cache_config.kv_cache_max_concurrency = (
getattr(cache_config, "kv_cache_max_concurrency", None)
if getattr(cache_config, "kv_cache_max_concurrency", None) is not None
else response.kv_cache_max_concurrency
)
# In external DP LB mode, the coordinator address that the
# front-end procs connect to is obtained by each engine via it's