[Docs] Fix VLLM_CPU_KVCACHE_SPACE default and units (#60053)

Signed-off-by: Praneeth <praneethreddyarikatla@gmail.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
This commit is contained in:
Praneeth Reddy A
2026-10-09 19:43:55 +08:00
committed by GitHub
co-authored by Li, Jiang <jiang1.li@intel.com>
parent d095e61c0b
commit 1c47ebfc57
3 changed files with 3 additions and 3 deletions
+1 -1
View File
@@ -80,7 +80,7 @@ llm = LLM(model="meta-llama/Llama-3.1-8B-Instruct", enforce_eager=True)
If you run out of CPU RAM, try the following options:
- (Multi-modal models only) you can set the size of multi-modal cache by setting `mm_processor_cache_gb` engine argument (default 4 GiB).
- (CPU backend only) you can set the size of KV cache using `VLLM_CPU_KVCACHE_SPACE` environment variable (default 4 GiB).
- (CPU backend only) you can set the size of KV cache using `VLLM_CPU_KVCACHE_SPACE` environment variable, in GiB (default 0, meaning vLLM doesn't override the KV cache size through this variable).
## Multi-modal input limits