[Core] Model console logging as CLI configuration (#57205)
Add `--logging-config` CLI argument which can be supplied as JSON or using dotted arguments. The `--log-level` argument is provided for convenience, and `--log-config-file` is deprecated in favor of `--logging-config.pylogging_config_file`. Signed-off-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: AI Assistant <noreply@openai.com>
This commit is contained in:
co-authored by
AI Assistant
parent
fa6c407d15
commit
45d56c0baf
@@ -3,46 +3,60 @@
|
||||
vLLM leverages Python's `logging.config.dictConfig` functionality to enable
|
||||
robust and flexible configuration of the various loggers used by vLLM.
|
||||
|
||||
vLLM offers two environment variables that can be used to accommodate a range
|
||||
of logging configurations that range from simple-and-inflexible to
|
||||
more-complex-and-more-flexible.
|
||||
For `vllm serve`, configure logging with CLI arguments:
|
||||
|
||||
- No vLLM logging (simple and inflexible)
|
||||
- Set `VLLM_CONFIGURE_LOGGING=0` (leaving `VLLM_LOGGING_CONFIG_PATH` unset)
|
||||
- vLLM's default logging configuration (simple and inflexible)
|
||||
- Leave `VLLM_CONFIGURE_LOGGING` unset or set `VLLM_CONFIGURE_LOGGING=1`
|
||||
- Fine-grained custom logging configuration (more complex, more flexible)
|
||||
- Leave `VLLM_CONFIGURE_LOGGING` unset or set `VLLM_CONFIGURE_LOGGING=1` and
|
||||
set `VLLM_LOGGING_CONFIG_PATH=<path-to-logging-config.json>`
|
||||
- Use the built-in configuration, optionally with `--log-level`.
|
||||
- Use a custom Python logging configuration file with
|
||||
`--logging-config.pylogging_config_file`.
|
||||
- Disable vLLM logging configuration with
|
||||
`--logging-config.configure_logging false`.
|
||||
|
||||
## Logging Configuration Environment Variables
|
||||
## CLI logging configuration
|
||||
|
||||
### `VLLM_CONFIGURE_LOGGING`
|
||||
`--logging-config` accepts a JSON object. Its fields are `log_level`,
|
||||
`configure_logging`, and `pylogging_config_file`. For example:
|
||||
|
||||
`VLLM_CONFIGURE_LOGGING` controls whether or not vLLM takes any action to
|
||||
configure the loggers used by vLLM. This functionality is enabled by default,
|
||||
but can be disabled by setting `VLLM_CONFIGURE_LOGGING=0` when running vLLM.
|
||||
```bash
|
||||
vllm serve mistralai/Mistral-7B-v0.1 \
|
||||
--logging-config '{"log_level":"DEBUG","configure_logging":true}'
|
||||
```
|
||||
|
||||
If `VLLM_CONFIGURE_LOGGING` is enabled and no value is given for
|
||||
`VLLM_LOGGING_CONFIG_PATH`, vLLM will use built-in default configuration to
|
||||
configure the root vLLM logger. By default, no other vLLM loggers are
|
||||
configured and, as such, all vLLM loggers defer to the root vLLM logger to make
|
||||
all logging decisions.
|
||||
Fields can also be set individually with dotted arguments:
|
||||
|
||||
If `VLLM_CONFIGURE_LOGGING` is disabled and a value is given for
|
||||
`VLLM_LOGGING_CONFIG_PATH`, an error will occur while starting vLLM.
|
||||
```bash
|
||||
vllm serve mistralai/Mistral-7B-v0.1 \
|
||||
--logging-config.log_level DEBUG
|
||||
```
|
||||
|
||||
### `VLLM_LOGGING_CONFIG_PATH`
|
||||
`--log-level` is a shortcut for `--logging-config.log_level` and takes
|
||||
precedence if both are supplied. It sets the level of vLLM's built-in logging
|
||||
configuration. `--log-config-file` is deprecated and will be removed in
|
||||
v0.33.0; use `--logging-config.pylogging_config_file` in new commands.
|
||||
|
||||
`VLLM_LOGGING_CONFIG_PATH` allows users to specify a path to a JSON file of
|
||||
alternative, custom logging configuration that will be used instead of vLLM's
|
||||
built-in default logging configuration. The logging configuration should be
|
||||
provided in JSON format following the schema specified by Python's [logging
|
||||
If `configure_logging` is `false`, vLLM does not apply a logging
|
||||
configuration. It cannot be combined with `pylogging_config_file`. This has
|
||||
the same effect as the legacy `VLLM_CONFIGURE_LOGGING=0` setting.
|
||||
|
||||
The custom logging configuration file must be JSON following Python's [logging
|
||||
configuration dictionary
|
||||
schema](https://docs.python.org/3/library/logging.config.html#dictionary-schema-details).
|
||||
|
||||
If `VLLM_LOGGING_CONFIG_PATH` is specified, but `VLLM_CONFIGURE_LOGGING` is
|
||||
disabled, an error will occur while starting vLLM.
|
||||
!!! note "Custom configurations override `--log-level`"
|
||||
When `pylogging_config_file` is set, vLLM loads that JSON file and replaces
|
||||
its built-in `dictConfig`; it does not merge the two. Therefore,
|
||||
`--log-level` applies only when no custom configuration file is provided.
|
||||
Set logger and handler levels in the custom file itself. vLLM applies the
|
||||
resolved configuration in its child processes as well.
|
||||
|
||||
## Environment variables
|
||||
|
||||
The CLI configuration is recommended for `vllm serve`. The legacy
|
||||
`VLLM_CONFIGURE_LOGGING`, `VLLM_LOGGING_LEVEL`, and
|
||||
`VLLM_LOGGING_CONFIG_PATH` variables remain supported as defaults. The default
|
||||
handler's stream, prefix, and color are currently controlled only through
|
||||
environment variables. Values from a YAML `--config` file or the command line
|
||||
override those defaults. See [Environment Variables](https://docs.vllm.ai/en/latest/configuration/env_vars/)
|
||||
for those settings.
|
||||
|
||||
## Examples
|
||||
|
||||
@@ -83,12 +97,11 @@ To begin, first, create an appropriate JSON logging configuration file:
|
||||
}
|
||||
```
|
||||
|
||||
Finally, run vLLM with the `VLLM_LOGGING_CONFIG_PATH` environment variable set
|
||||
to the path of the custom logging configuration JSON file:
|
||||
Finally, run vLLM with the custom logging configuration JSON file:
|
||||
|
||||
```bash
|
||||
VLLM_LOGGING_CONFIG_PATH=/path/to/logging_config.json \
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
|
||||
--logging-config.pylogging_config_file /path/to/logging_config.json
|
||||
```
|
||||
|
||||
### Example 2: Silence a particular vLLM logger
|
||||
@@ -137,26 +150,27 @@ configuration for the root vLLM logger and for the logger you wish to silence:
|
||||
}
|
||||
```
|
||||
|
||||
Finally, run vLLM with the `VLLM_LOGGING_CONFIG_PATH` environment variable set
|
||||
to the path of the custom logging configuration JSON file:
|
||||
Finally, run vLLM with the custom logging configuration JSON file:
|
||||
|
||||
```bash
|
||||
VLLM_LOGGING_CONFIG_PATH=/path/to/logging_config.json \
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
|
||||
--logging-config.pylogging_config_file /path/to/logging_config.json
|
||||
```
|
||||
|
||||
### Example 3: Disable vLLM default logging configuration
|
||||
|
||||
To disable vLLM's default logging configuration and silence all vLLM loggers,
|
||||
simple set `VLLM_CONFIGURE_LOGGING=0` when running vLLM. This will prevent vLLM
|
||||
for configuring the root vLLM logger, which in turn, silences all other vLLM
|
||||
loggers.
|
||||
To disable vLLM's default logging configuration and silence vLLM log output,
|
||||
set `--logging-config.configure_logging false` when running vLLM. This prevents
|
||||
vLLM from configuring the root vLLM logger, which in turn silences other vLLM
|
||||
loggers unless the application or Python root logger configures a handler.
|
||||
|
||||
```bash
|
||||
VLLM_CONFIGURE_LOGGING=0 \
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
|
||||
--logging-config.configure_logging false
|
||||
```
|
||||
|
||||
For legacy launch scripts, `VLLM_CONFIGURE_LOGGING=0` has the same effect.
|
||||
|
||||
### Example 4: Disable access logs for health check endpoints
|
||||
|
||||
In production environments, health check endpoints like `/health`, `/metrics`,
|
||||
|
||||
Reference in New Issue
Block a user