[Startup][UX] Enable CUDAGraph memory profiling by default (#38284)

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
This commit is contained in:
Matthew Bonanni
2026-04-21 18:16:59 -04:00
committed by GitHub
co-authored by Tyler Michael Smith
parent 9db4650e5e
commit 96a85c5750
8 changed files with 26 additions and 27 deletions
+2 -2
View File
@@ -51,10 +51,10 @@ class CacheConfig:
"""Whether block_size was explicitly provided. Derived automatically."""
user_specified_mamba_block_size: bool = field(default=False, init=False)
"""Whether mamba_block_size was explicitly provided. Derived automatically."""
gpu_memory_utilization: float = Field(default=0.9, gt=0, le=1)
gpu_memory_utilization: float = Field(default=0.92, gt=0, le=1)
"""The fraction of GPU memory to be used for the model executor, which can
range from 0 to 1. For example, a value of 0.5 would imply 50% GPU memory
utilization. If unspecified, will use the default value of 0.9. This is a
utilization. If unspecified, will use the default value of 0.92. This is a
per-instance limit, and only applies to the current vLLM instance. It does
not matter if you have another vLLM instance running on the same GPU. For
example, if you have two vLLM instances running on the same GPU, you can