[Doc] Surface the --kv-cache-memory suggestion at INFO and document fast-startup knobs (#47374)
Signed-off-by: Nils Matteson <nils@thaw.sh> Signed-off-by: Nils Matteson <nilsmatteson@icloud.com> Co-authored-by: Nils Matteson <nils@thaw.sh> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Nils Matteson
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
55da232db6
commit
dd0d74cd92
@@ -832,7 +832,7 @@ class Worker(WorkerBase):
|
||||
f"{format_gib(self.available_kv_cache_memory_bytes)} GiB."
|
||||
)
|
||||
|
||||
logger.debug(msg)
|
||||
logger.info(msg)
|
||||
|
||||
if self.use_v2_model_runner:
|
||||
# V2: Run full execute_model + sample_tokens to JIT compile triton kernels.
|
||||
|
||||
Reference in New Issue
Block a user