[Core] Model console logging as CLI configuration (#57205)
Add `--logging-config` CLI argument which can be supplied as JSON or using dotted arguments. The `--log-level` argument is provided for convenience, and `--log-config-file` is deprecated in favor of `--logging-config.pylogging_config_file`. Signed-off-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: AI Assistant <noreply@openai.com>
This commit is contained in:
co-authored by
AI Assistant
parent
fa6c407d15
commit
45d56c0baf
@@ -3,46 +3,60 @@
|
||||
vLLM leverages Python's `logging.config.dictConfig` functionality to enable
|
||||
robust and flexible configuration of the various loggers used by vLLM.
|
||||
|
||||
vLLM offers two environment variables that can be used to accommodate a range
|
||||
of logging configurations that range from simple-and-inflexible to
|
||||
more-complex-and-more-flexible.
|
||||
For `vllm serve`, configure logging with CLI arguments:
|
||||
|
||||
- No vLLM logging (simple and inflexible)
|
||||
- Set `VLLM_CONFIGURE_LOGGING=0` (leaving `VLLM_LOGGING_CONFIG_PATH` unset)
|
||||
- vLLM's default logging configuration (simple and inflexible)
|
||||
- Leave `VLLM_CONFIGURE_LOGGING` unset or set `VLLM_CONFIGURE_LOGGING=1`
|
||||
- Fine-grained custom logging configuration (more complex, more flexible)
|
||||
- Leave `VLLM_CONFIGURE_LOGGING` unset or set `VLLM_CONFIGURE_LOGGING=1` and
|
||||
set `VLLM_LOGGING_CONFIG_PATH=<path-to-logging-config.json>`
|
||||
- Use the built-in configuration, optionally with `--log-level`.
|
||||
- Use a custom Python logging configuration file with
|
||||
`--logging-config.pylogging_config_file`.
|
||||
- Disable vLLM logging configuration with
|
||||
`--logging-config.configure_logging false`.
|
||||
|
||||
## Logging Configuration Environment Variables
|
||||
## CLI logging configuration
|
||||
|
||||
### `VLLM_CONFIGURE_LOGGING`
|
||||
`--logging-config` accepts a JSON object. Its fields are `log_level`,
|
||||
`configure_logging`, and `pylogging_config_file`. For example:
|
||||
|
||||
`VLLM_CONFIGURE_LOGGING` controls whether or not vLLM takes any action to
|
||||
configure the loggers used by vLLM. This functionality is enabled by default,
|
||||
but can be disabled by setting `VLLM_CONFIGURE_LOGGING=0` when running vLLM.
|
||||
```bash
|
||||
vllm serve mistralai/Mistral-7B-v0.1 \
|
||||
--logging-config '{"log_level":"DEBUG","configure_logging":true}'
|
||||
```
|
||||
|
||||
If `VLLM_CONFIGURE_LOGGING` is enabled and no value is given for
|
||||
`VLLM_LOGGING_CONFIG_PATH`, vLLM will use built-in default configuration to
|
||||
configure the root vLLM logger. By default, no other vLLM loggers are
|
||||
configured and, as such, all vLLM loggers defer to the root vLLM logger to make
|
||||
all logging decisions.
|
||||
Fields can also be set individually with dotted arguments:
|
||||
|
||||
If `VLLM_CONFIGURE_LOGGING` is disabled and a value is given for
|
||||
`VLLM_LOGGING_CONFIG_PATH`, an error will occur while starting vLLM.
|
||||
```bash
|
||||
vllm serve mistralai/Mistral-7B-v0.1 \
|
||||
--logging-config.log_level DEBUG
|
||||
```
|
||||
|
||||
### `VLLM_LOGGING_CONFIG_PATH`
|
||||
`--log-level` is a shortcut for `--logging-config.log_level` and takes
|
||||
precedence if both are supplied. It sets the level of vLLM's built-in logging
|
||||
configuration. `--log-config-file` is deprecated and will be removed in
|
||||
v0.33.0; use `--logging-config.pylogging_config_file` in new commands.
|
||||
|
||||
`VLLM_LOGGING_CONFIG_PATH` allows users to specify a path to a JSON file of
|
||||
alternative, custom logging configuration that will be used instead of vLLM's
|
||||
built-in default logging configuration. The logging configuration should be
|
||||
provided in JSON format following the schema specified by Python's [logging
|
||||
If `configure_logging` is `false`, vLLM does not apply a logging
|
||||
configuration. It cannot be combined with `pylogging_config_file`. This has
|
||||
the same effect as the legacy `VLLM_CONFIGURE_LOGGING=0` setting.
|
||||
|
||||
The custom logging configuration file must be JSON following Python's [logging
|
||||
configuration dictionary
|
||||
schema](https://docs.python.org/3/library/logging.config.html#dictionary-schema-details).
|
||||
|
||||
If `VLLM_LOGGING_CONFIG_PATH` is specified, but `VLLM_CONFIGURE_LOGGING` is
|
||||
disabled, an error will occur while starting vLLM.
|
||||
!!! note "Custom configurations override `--log-level`"
|
||||
When `pylogging_config_file` is set, vLLM loads that JSON file and replaces
|
||||
its built-in `dictConfig`; it does not merge the two. Therefore,
|
||||
`--log-level` applies only when no custom configuration file is provided.
|
||||
Set logger and handler levels in the custom file itself. vLLM applies the
|
||||
resolved configuration in its child processes as well.
|
||||
|
||||
## Environment variables
|
||||
|
||||
The CLI configuration is recommended for `vllm serve`. The legacy
|
||||
`VLLM_CONFIGURE_LOGGING`, `VLLM_LOGGING_LEVEL`, and
|
||||
`VLLM_LOGGING_CONFIG_PATH` variables remain supported as defaults. The default
|
||||
handler's stream, prefix, and color are currently controlled only through
|
||||
environment variables. Values from a YAML `--config` file or the command line
|
||||
override those defaults. See [Environment Variables](https://docs.vllm.ai/en/latest/configuration/env_vars/)
|
||||
for those settings.
|
||||
|
||||
## Examples
|
||||
|
||||
@@ -83,12 +97,11 @@ To begin, first, create an appropriate JSON logging configuration file:
|
||||
}
|
||||
```
|
||||
|
||||
Finally, run vLLM with the `VLLM_LOGGING_CONFIG_PATH` environment variable set
|
||||
to the path of the custom logging configuration JSON file:
|
||||
Finally, run vLLM with the custom logging configuration JSON file:
|
||||
|
||||
```bash
|
||||
VLLM_LOGGING_CONFIG_PATH=/path/to/logging_config.json \
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
|
||||
--logging-config.pylogging_config_file /path/to/logging_config.json
|
||||
```
|
||||
|
||||
### Example 2: Silence a particular vLLM logger
|
||||
@@ -137,26 +150,27 @@ configuration for the root vLLM logger and for the logger you wish to silence:
|
||||
}
|
||||
```
|
||||
|
||||
Finally, run vLLM with the `VLLM_LOGGING_CONFIG_PATH` environment variable set
|
||||
to the path of the custom logging configuration JSON file:
|
||||
Finally, run vLLM with the custom logging configuration JSON file:
|
||||
|
||||
```bash
|
||||
VLLM_LOGGING_CONFIG_PATH=/path/to/logging_config.json \
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
|
||||
--logging-config.pylogging_config_file /path/to/logging_config.json
|
||||
```
|
||||
|
||||
### Example 3: Disable vLLM default logging configuration
|
||||
|
||||
To disable vLLM's default logging configuration and silence all vLLM loggers,
|
||||
simple set `VLLM_CONFIGURE_LOGGING=0` when running vLLM. This will prevent vLLM
|
||||
for configuring the root vLLM logger, which in turn, silences all other vLLM
|
||||
loggers.
|
||||
To disable vLLM's default logging configuration and silence vLLM log output,
|
||||
set `--logging-config.configure_logging false` when running vLLM. This prevents
|
||||
vLLM from configuring the root vLLM logger, which in turn silences other vLLM
|
||||
loggers unless the application or Python root logger configures a handler.
|
||||
|
||||
```bash
|
||||
VLLM_CONFIGURE_LOGGING=0 \
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048
|
||||
vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
|
||||
--logging-config.configure_logging false
|
||||
```
|
||||
|
||||
For legacy launch scripts, `VLLM_CONFIGURE_LOGGING=0` has the same effect.
|
||||
|
||||
### Example 4: Disable access logs for health check endpoints
|
||||
|
||||
In production environments, health check endpoints like `/health`, `/metrics`,
|
||||
|
||||
+8
-4
@@ -1430,16 +1430,20 @@ def temporary_enable_log_propagate():
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger("vllm")
|
||||
previous_propagate = logger.propagate
|
||||
logger.propagate = True
|
||||
yield
|
||||
logger.propagate = False
|
||||
logger.propagate = previous_propagate
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def caplog_vllm(temporary_enable_log_propagate, caplog):
|
||||
# To capture vllm log, we should enable propagate=True temporarily
|
||||
# because caplog depends on logs propagated to the root logger.
|
||||
yield caplog
|
||||
# caplog depends on logs propagated to the root logger. The vLLM logger
|
||||
# inherits this INFO level until runtime initialization configures it.
|
||||
import logging
|
||||
|
||||
with caplog.at_level(logging.INFO):
|
||||
yield caplog
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
|
||||
@@ -8,6 +8,7 @@ import pytest
|
||||
|
||||
import vllm.entrypoints.launchers.cli_args as cli_args_module
|
||||
from tests.utils import VLLM_PATH
|
||||
from vllm.engine.arg_utils import AsyncEngineArgs
|
||||
from vllm.entrypoints.launchers.cli_args import (
|
||||
make_arg_parser,
|
||||
validate_parsed_serve_args,
|
||||
@@ -69,6 +70,24 @@ def test_config_arg_parsing(serve_parser, cli_config_file):
|
||||
assert args.port == 9000
|
||||
|
||||
|
||||
def test_logging_config_cli_args(serve_parser):
|
||||
with pytest.warns(UserWarning, match="--log-config-file is deprecated"):
|
||||
args = serve_parser.parse_args(
|
||||
[
|
||||
"--logging-config",
|
||||
'{"log_level":"WARNING","pylogging_config_file":"/tmp/json.json"}',
|
||||
"--log-level",
|
||||
"DEBUG",
|
||||
"--log-config-file",
|
||||
"/tmp/flat.json",
|
||||
]
|
||||
)
|
||||
|
||||
config = AsyncEngineArgs.from_cli_args(args).create_logging_config()
|
||||
assert config.log_level == "DEBUG"
|
||||
assert config.pylogging_config_file == "/tmp/flat.json"
|
||||
|
||||
|
||||
### Tests for LoRA module parsing
|
||||
def test_valid_key_value_format(serve_parser):
|
||||
# Test old format: name=path
|
||||
|
||||
+60
-9
@@ -16,11 +16,13 @@ from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
|
||||
from vllm.config import LoggingConfig
|
||||
from vllm.logger import (
|
||||
_DATE_FORMAT,
|
||||
_FORMAT,
|
||||
_configure_vllm_root_logger,
|
||||
_use_color,
|
||||
configure_logging,
|
||||
enable_trace_function_call,
|
||||
init_logger,
|
||||
)
|
||||
@@ -50,16 +52,27 @@ def test_trace_function_call():
|
||||
os.remove(path)
|
||||
|
||||
|
||||
def test_caplog_vllm_captures_info_before_runtime_logging_is_configured(caplog_vllm):
|
||||
message = "Capture this unconfigured INFO record"
|
||||
logger = init_logger(f"vllm.test_logger.{uuid4()}")
|
||||
|
||||
logger.info(message)
|
||||
|
||||
assert message in caplog_vllm.text
|
||||
|
||||
|
||||
def test_default_vllm_root_logger_configuration(monkeypatch):
|
||||
"""This test presumes that VLLM_CONFIGURE_LOGGING (default: True) and
|
||||
VLLM_LOGGING_CONFIG_PATH (default: None) are not configured and default
|
||||
behavior is activated."""
|
||||
monkeypatch.setenv("VLLM_LOGGING_COLOR", "0")
|
||||
_configure_vllm_root_logger()
|
||||
early_logger = init_logger(f"vllm.test_logger.{uuid4()}")
|
||||
configure_logging(LoggingConfig())
|
||||
|
||||
logger = logging.getLogger("vllm")
|
||||
assert logger.level == logging.INFO
|
||||
assert not logger.propagate
|
||||
assert early_logger.isEnabledFor(logging.INFO)
|
||||
|
||||
handler = logger.handlers[0]
|
||||
assert isinstance(handler, logging.StreamHandler)
|
||||
@@ -74,6 +87,29 @@ def test_default_vllm_root_logger_configuration(monkeypatch):
|
||||
assert formatter.datefmt == _DATE_FORMAT
|
||||
|
||||
|
||||
def test_offline_llm_configures_logging_before_logging_args(monkeypatch):
|
||||
import vllm.entrypoints.llm as llm_module
|
||||
|
||||
class StopInitialization(Exception):
|
||||
pass
|
||||
|
||||
configured = False
|
||||
|
||||
def configure_logging(_):
|
||||
nonlocal configured
|
||||
configured = True
|
||||
|
||||
def log_args(_):
|
||||
assert configured
|
||||
raise StopInitialization
|
||||
|
||||
monkeypatch.setattr(llm_module, "configure_logging_if_needed", configure_logging)
|
||||
monkeypatch.setattr(llm_module, "log_non_default_args", log_args)
|
||||
|
||||
with pytest.raises(StopInitialization):
|
||||
llm_module.LLM(model="facebook/opt-125m")
|
||||
|
||||
|
||||
def test_use_color_force_color(monkeypatch):
|
||||
"""FORCE_COLOR forces colored logs without a TTY, while NO_COLOR and an
|
||||
explicit VLLM_LOGGING_COLOR=0 take precedence over it."""
|
||||
@@ -133,7 +169,7 @@ def test_logger_configuring_can_be_disabled(monkeypatch):
|
||||
monkeypatch.delenv("VLLM_LOGGING_CONFIG_PATH", raising=False)
|
||||
|
||||
with patch("vllm.logger.dictConfig") as dict_config_mock:
|
||||
_configure_vllm_root_logger()
|
||||
configure_logging(LoggingConfig(configure_logging=False))
|
||||
dict_config_mock.assert_not_called()
|
||||
|
||||
|
||||
@@ -214,9 +250,11 @@ def test_custom_logging_config_is_parsed_and_used_when_provided(monkeypatch):
|
||||
with NamedTemporaryFile(encoding="utf-8", mode="w") as logging_config_file:
|
||||
logging_config_file.write(json.dumps(valid_logging_config))
|
||||
logging_config_file.flush()
|
||||
monkeypatch.setenv("VLLM_LOGGING_CONFIG_PATH", logging_config_file.name)
|
||||
monkeypatch.delenv("VLLM_LOGGING_CONFIG_PATH", raising=False)
|
||||
with patch("vllm.logger.dictConfig") as dict_config_mock:
|
||||
_configure_vllm_root_logger()
|
||||
configure_logging(
|
||||
LoggingConfig(pylogging_config_file=logging_config_file.name)
|
||||
)
|
||||
dict_config_mock.assert_called_with(valid_logging_config)
|
||||
|
||||
|
||||
@@ -237,13 +275,18 @@ def test_custom_logging_config_causes_an_error_if_configure_logging_is_off(monke
|
||||
with NamedTemporaryFile(encoding="utf-8", mode="w") as logging_config_file:
|
||||
logging_config_file.write(json.dumps(valid_logging_config))
|
||||
logging_config_file.flush()
|
||||
monkeypatch.setenv("VLLM_LOGGING_CONFIG_PATH", logging_config_file.name)
|
||||
monkeypatch.delenv("VLLM_LOGGING_CONFIG_PATH", raising=False)
|
||||
with pytest.raises(RuntimeError) as ex_info:
|
||||
_configure_vllm_root_logger()
|
||||
configure_logging(
|
||||
LoggingConfig(
|
||||
configure_logging=False,
|
||||
pylogging_config_file=logging_config_file.name,
|
||||
)
|
||||
)
|
||||
assert ex_info.type is RuntimeError
|
||||
expected_message_snippet = (
|
||||
"VLLM_CONFIGURE_LOGGING evaluated to false, but "
|
||||
"VLLM_LOGGING_CONFIG_PATH was given."
|
||||
"Logging configuration is disabled, but a Python logging config "
|
||||
"file was given."
|
||||
)
|
||||
assert expected_message_snippet in str(ex_info)
|
||||
|
||||
@@ -297,6 +340,8 @@ test_logger = init_logger("vllm.test_logger")
|
||||
|
||||
def mp_function(**kwargs):
|
||||
# This function runs in a subprocess
|
||||
if logging_config := kwargs.pop("logging_config", None):
|
||||
configure_logging(logging_config)
|
||||
|
||||
test_logger.warning("This is a subprocess: %s", kwargs.get("a"))
|
||||
test_logger.error("This is a subprocess error.")
|
||||
@@ -328,7 +373,13 @@ def test_caplog_mp_spawn(caplog_mp_spawn):
|
||||
p = ctx.Process(
|
||||
target=mp_function,
|
||||
name=f"SubProcess{1}",
|
||||
kwargs={"a": "AAAA", "b": "BBBBB"},
|
||||
kwargs={
|
||||
"a": "AAAA",
|
||||
"b": "BBBBB",
|
||||
"logging_config": LoggingConfig(
|
||||
pylogging_config_file=os.environ["VLLM_LOGGING_CONFIG_PATH"]
|
||||
),
|
||||
},
|
||||
)
|
||||
p.start()
|
||||
p.join()
|
||||
|
||||
@@ -11,7 +11,7 @@ import pytest
|
||||
import vllm.v1.engine.async_llm as async_llm_module
|
||||
from vllm import SamplingParams
|
||||
from vllm.assets.image import ImageAsset
|
||||
from vllm.config import VllmConfig
|
||||
from vllm.config import LoggingConfig, VllmConfig
|
||||
from vllm.engine.arg_utils import AsyncEngineArgs
|
||||
from vllm.entrypoints.openai.chat_completion.protocol import (
|
||||
ChatCompletionRequest,
|
||||
@@ -89,6 +89,7 @@ def _mock_async_llm_dependencies(monkeypatch: pytest.MonkeyPatch):
|
||||
|
||||
def test_cuda_profiler_requests_reach_engine_core(monkeypatch: pytest.MonkeyPatch):
|
||||
vllm_config = MagicMock()
|
||||
vllm_config.logging_config = LoggingConfig()
|
||||
vllm_config.observability_config.otlp_traces_endpoint = None
|
||||
vllm_config.scheduler_config.stream_interval = 1
|
||||
vllm_config.profiler_config.profiler = "cuda"
|
||||
@@ -111,6 +112,7 @@ def test_cuda_only_torch_profiler_skips_frontend_cpu_trace(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
):
|
||||
vllm_config = MagicMock()
|
||||
vllm_config.logging_config = LoggingConfig()
|
||||
vllm_config.observability_config.otlp_traces_endpoint = None
|
||||
vllm_config.scheduler_config.stream_interval = 1
|
||||
vllm_config.profiler_config.profiler = "torch"
|
||||
|
||||
@@ -25,6 +25,7 @@ from __future__ import annotations
|
||||
import dataclasses
|
||||
import functools
|
||||
import gc
|
||||
import logging
|
||||
import threading
|
||||
import weakref
|
||||
from collections.abc import Callable
|
||||
@@ -285,7 +286,7 @@ class BreakableCUDAGraphWrapper:
|
||||
self.runtime_mode = runtime_mode
|
||||
self.compilation_config = vllm_config.compilation_config
|
||||
self.graph_pool = current_platform.get_global_graph_pool()
|
||||
self.is_debugging_mode = envs.VLLM_LOGGING_LEVEL == "DEBUG"
|
||||
self.is_debugging_mode = logger.isEnabledFor(logging.DEBUG)
|
||||
|
||||
self.entries: dict[BatchDescriptor, _BreakableEntry] = {}
|
||||
BreakableCUDAGraphWrapper._all_instances.add(self)
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
|
||||
import dataclasses
|
||||
import logging
|
||||
import weakref
|
||||
from collections import Counter
|
||||
from collections.abc import Callable
|
||||
@@ -11,7 +12,6 @@ from unittest.mock import patch
|
||||
|
||||
import torch
|
||||
|
||||
import vllm.envs as envs
|
||||
from vllm.compilation.counter import compilation_counter
|
||||
from vllm.compilation.monitor import validate_cudagraph_capturing_enabled
|
||||
from vllm.config import CUDAGraphMode, VllmConfig
|
||||
@@ -164,7 +164,7 @@ class CUDAGraphWrapper:
|
||||
assumption on the dynamic shape (batch size) of the runtime inputs, as a
|
||||
trade-off for staying orthogonal to compilation logic. Nevertheless,
|
||||
tracing and checking the input addresses to be consistent during replay is
|
||||
guaranteed when VLLM_LOGGING_LEVEL == "DEBUG".
|
||||
guaranteed when vLLM debug logging is enabled.
|
||||
"""
|
||||
|
||||
_all_instances: ClassVar[weakref.WeakSet["CUDAGraphWrapper"]] = weakref.WeakSet()
|
||||
@@ -188,7 +188,7 @@ class CUDAGraphWrapper:
|
||||
self.compilation_config = vllm_config.compilation_config
|
||||
|
||||
self.first_run_finished = False
|
||||
self.is_debugging_mode = envs.VLLM_LOGGING_LEVEL == "DEBUG"
|
||||
self.is_debugging_mode = logger.isEnabledFor(logging.DEBUG)
|
||||
self._runnable_str = str(runnable) if self.is_debugging_mode else None
|
||||
|
||||
# assert runtime_mode is not NONE(no cudagraph), otherwise, we don't
|
||||
|
||||
@@ -20,6 +20,7 @@ from vllm.config.kernel import KernelConfig
|
||||
from vllm.config.kv_events import KVEventsConfig
|
||||
from vllm.config.kv_transfer import KVTransferConfig
|
||||
from vllm.config.load import LoadConfig
|
||||
from vllm.config.logging import LoggingConfig
|
||||
from vllm.config.lora import LoRAConfig
|
||||
from vllm.config.mamba import MambaConfig
|
||||
from vllm.config.model import (
|
||||
@@ -97,6 +98,8 @@ __all__ = [
|
||||
"KVTransferConfig",
|
||||
# From vllm.config.load
|
||||
"LoadConfig",
|
||||
# From vllm.config.logging
|
||||
"LoggingConfig",
|
||||
# From vllm.config.lora
|
||||
"LoRAConfig",
|
||||
# From vllm.config.mamba
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
|
||||
from typing import Literal, cast
|
||||
|
||||
from pydantic import Field
|
||||
|
||||
import vllm.envs as envs
|
||||
from vllm.config.utils import config
|
||||
|
||||
LogLevel = Literal["CRITICAL", "ERROR", "WARNING", "INFO", "DEBUG", "NOTSET"]
|
||||
|
||||
|
||||
@config
|
||||
class LoggingConfig:
|
||||
"""vLLM logging. Default ``dictConfig`` adds a formatted stream handler.
|
||||
|
||||
Supply a JSON object with ``--logging-config`` or individual fields with
|
||||
dotted arguments such as ``--logging-config.log_level DEBUG`` and
|
||||
``--logging-config.pylogging_config_file logging.json``. ``--log-level``
|
||||
and legacy ``--log-config-file`` override their matching fields. Set
|
||||
``configure_logging`` to false to skip applying a ``dictConfig``.
|
||||
"""
|
||||
|
||||
log_level: LogLevel = Field(
|
||||
default_factory=lambda: cast(LogLevel, envs.VLLM_LOGGING_LEVEL)
|
||||
)
|
||||
"""Log level used when no custom logging configuration is provided."""
|
||||
|
||||
configure_logging: bool = Field(default_factory=lambda: envs.VLLM_CONFIGURE_LOGGING)
|
||||
"""Whether to apply a Python logging configuration.
|
||||
|
||||
With no ``pylogging_config_file``, the default config adds a formatted
|
||||
stream handler to the ``vllm`` logger. A custom Python logging configuration
|
||||
file also requires this to be enabled.
|
||||
"""
|
||||
|
||||
pylogging_config_file: str | None = Field(
|
||||
default_factory=lambda: envs.VLLM_LOGGING_CONFIG_PATH
|
||||
)
|
||||
"""Path to a Python logging JSON file using
|
||||
[``dictConfig`` schema](https://docs.python.org/3/library/logging.config.html#configuration-file-format).
|
||||
|
||||
A custom ``dictConfig`` is authoritative over ``log_level`` for logger,
|
||||
handler, and formatter settings.
|
||||
"""
|
||||
@@ -40,6 +40,7 @@ from .kernel import KernelConfig
|
||||
from .kv_events import KVEventsConfig
|
||||
from .kv_transfer import KVTransferConfig
|
||||
from .load import LoadConfig
|
||||
from .logging import LoggingConfig
|
||||
from .lora import LoRAConfig
|
||||
from .mamba import MambaBackendEnum, MambaConfig
|
||||
from .model import ModelConfig
|
||||
@@ -402,6 +403,8 @@ class VllmConfig:
|
||||
default_factory=ObservabilityConfig
|
||||
)
|
||||
"""Observability configuration."""
|
||||
logging_config: LoggingConfig = Field(default_factory=LoggingConfig)
|
||||
"""Logging configuration."""
|
||||
quant_config: QuantizationConfig | None = None
|
||||
"""Quantization configuration."""
|
||||
compilation_config: CompilationConfig = Field(default_factory=CompilationConfig)
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
import atexit
|
||||
import contextlib
|
||||
import logging
|
||||
import tempfile
|
||||
from typing import Any
|
||||
|
||||
@@ -83,7 +84,7 @@ def compile_nccl_allocator():
|
||||
cpp_sources=nccl_allocator_source,
|
||||
with_cuda=True,
|
||||
extra_ldflags=ldflags,
|
||||
verbose=envs.VLLM_LOGGING_LEVEL == "DEBUG",
|
||||
verbose=logger.isEnabledFor(logging.DEBUG),
|
||||
is_python_module=False,
|
||||
build_directory=out_dir,
|
||||
extra_include_paths=nccl_include_paths,
|
||||
|
||||
@@ -8,6 +8,7 @@ import functools
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import warnings
|
||||
from collections.abc import Callable
|
||||
from dataclasses import MISSING, asdict, dataclass, fields, is_dataclass
|
||||
from types import UnionType
|
||||
@@ -50,6 +51,7 @@ from vllm.config import (
|
||||
KVEventsConfig,
|
||||
KVTransferConfig,
|
||||
LoadConfig,
|
||||
LoggingConfig,
|
||||
LoRAConfig,
|
||||
MambaConfig,
|
||||
ModelConfig,
|
||||
@@ -84,6 +86,7 @@ from vllm.config.kernel import (
|
||||
SparseIndexerTopkBackend,
|
||||
)
|
||||
from vllm.config.load import SafetensorsLoadStrategy
|
||||
from vllm.config.logging import LogLevel
|
||||
from vllm.config.lora import MaxLoRARanks
|
||||
from vllm.config.mamba import MambaBackendEnum, MambaSSUAlgorithm
|
||||
from vllm.config.model import (
|
||||
@@ -113,7 +116,7 @@ from vllm.config.scheduler import SchedulerPolicy
|
||||
from vllm.config.utils import get_field
|
||||
from vllm.config.vllm import OptimizationLevel, PerformanceMode
|
||||
from vllm.config.watermarking import WatermarkConfig
|
||||
from vllm.logger import init_logger, suppress_logging
|
||||
from vllm.logger import configure_logging_if_needed, init_logger, suppress_logging
|
||||
from vllm.platforms import CpuArchEnum, current_platform
|
||||
from vllm.plugins import load_general_plugins
|
||||
from vllm.ray.lazy_utils import is_in_ray_actor, is_ray_initialized
|
||||
@@ -443,6 +446,20 @@ def get_kwargs(cls: ConfigType) -> dict[str, dict[str, Any]]:
|
||||
return copy.deepcopy(_compute_kwargs(cls))
|
||||
|
||||
|
||||
_LOG_CONFIG_FILE_DEPRECATION_MESSAGE = (
|
||||
"--log-config-file is deprecated and will be removed in v0.33.0. "
|
||||
"Use --logging-config.pylogging_config_file instead."
|
||||
)
|
||||
|
||||
|
||||
class DeprecatedLogConfigFileAction(argparse.Action):
|
||||
"""Warn when the legacy ``--log-config-file`` option is used."""
|
||||
|
||||
def __call__(self, parser, namespace, values, option_string=None):
|
||||
warnings.warn(_LOG_CONFIG_FILE_DEPRECATION_MESSAGE, stacklevel=1)
|
||||
setattr(namespace, self.dest, values)
|
||||
|
||||
|
||||
@dataclass
|
||||
class EngineArgs:
|
||||
"""Arguments for vLLM engine."""
|
||||
@@ -733,6 +750,10 @@ class EngineArgs:
|
||||
|
||||
profiler_config: ProfilerConfig = get_field(VllmConfig, "profiler_config")
|
||||
|
||||
logging_config: LoggingConfig | None = None
|
||||
log_level: LogLevel | None = None
|
||||
log_config_file: str | None = None
|
||||
|
||||
kv_transfer_config: KVTransferConfig | None = None
|
||||
kv_events_config: KVEventsConfig | None = None
|
||||
|
||||
@@ -1551,6 +1572,34 @@ class EngineArgs:
|
||||
**lora_kwargs["enable_moe_shared_loras"],
|
||||
)
|
||||
|
||||
# Logging arguments
|
||||
logging_group = parser.add_argument_group(
|
||||
title="LoggingConfig",
|
||||
description=LoggingConfig.__doc__,
|
||||
)
|
||||
logging_config_kwargs = get_kwargs(VllmConfig)["logging_config"]
|
||||
logging_config_kwargs["default"] = argparse.SUPPRESS
|
||||
logging_group.add_argument("--logging-config", **logging_config_kwargs)
|
||||
logging_group.add_argument(
|
||||
"--log-level",
|
||||
choices=get_args(LogLevel),
|
||||
default=argparse.SUPPRESS,
|
||||
help=(
|
||||
"Shortcut for --logging-config.log_level. "
|
||||
"Overrides that field if both are specified."
|
||||
),
|
||||
)
|
||||
logging_group.add_argument(
|
||||
"--log-config-file",
|
||||
dest="log_config_file",
|
||||
default=argparse.SUPPRESS,
|
||||
metavar="PYLOGGING_CONFIG_FILE",
|
||||
action=DeprecatedLogConfigFileAction,
|
||||
help=_LOG_CONFIG_FILE_DEPRECATION_MESSAGE,
|
||||
)
|
||||
# Retain the warning in v0.31.0 and v0.32.0. Remove this option and its
|
||||
# compatibility mapping in create_logging_config() in v0.33.0.
|
||||
|
||||
# Observability arguments
|
||||
observability_kwargs = get_kwargs(ObservabilityConfig)
|
||||
observability_group = parser.add_argument_group(
|
||||
@@ -2081,6 +2130,16 @@ class EngineArgs:
|
||||
jit_monitor_verbose=self.jit_monitor_verbose,
|
||||
)
|
||||
|
||||
def create_logging_config(self) -> LoggingConfig:
|
||||
config = self.logging_config or LoggingConfig()
|
||||
if self.log_level is not None:
|
||||
config = dataclasses.replace(config, log_level=self.log_level)
|
||||
if self.log_config_file is not None:
|
||||
config = dataclasses.replace(
|
||||
config, pylogging_config_file=self.log_config_file
|
||||
)
|
||||
return config
|
||||
|
||||
def create_engine_config(
|
||||
self,
|
||||
usage_context: UsageContext | None = None,
|
||||
@@ -2090,6 +2149,9 @@ class EngineArgs:
|
||||
|
||||
NOTE: If VllmConfig is incompatible, we raise an error.
|
||||
"""
|
||||
logging_config = self.create_logging_config()
|
||||
configure_logging_if_needed(logging_config)
|
||||
|
||||
current_platform.pre_register_and_update()
|
||||
|
||||
device_config = DeviceConfig(device=cast(Device, current_platform.device_type))
|
||||
@@ -2721,6 +2783,7 @@ class EngineArgs:
|
||||
diffusion_config=diffusion_config,
|
||||
structured_outputs_config=self.structured_outputs_config,
|
||||
observability_config=observability_config,
|
||||
logging_config=logging_config,
|
||||
compilation_config=compilation_config,
|
||||
kv_transfer_config=self.kv_transfer_config,
|
||||
kv_events_config=self.kv_events_config,
|
||||
|
||||
@@ -9,7 +9,7 @@ import importlib.metadata
|
||||
import sys
|
||||
from importlib.util import find_spec
|
||||
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging_from_args, init_logger
|
||||
|
||||
logger = init_logger(__name__)
|
||||
|
||||
@@ -110,7 +110,9 @@ def main():
|
||||
cmds[cmd.name] = cmd
|
||||
args = parser.parse_args()
|
||||
if args.subparser in cmds:
|
||||
cmds[args.subparser].validate(args)
|
||||
cmd = cmds[args.subparser]
|
||||
configure_logging_from_args(args)
|
||||
cmd.validate(args)
|
||||
|
||||
if hasattr(args, "dispatch_function"):
|
||||
args.dispatch_function(args)
|
||||
|
||||
@@ -15,7 +15,7 @@ from typing import Any
|
||||
import vllm.envs as envs
|
||||
from vllm.engine.arg_utils import AsyncEngineArgs
|
||||
from vllm.engine.protocol import EngineClient
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging_from_args, init_logger
|
||||
from vllm.reasoning import ReasoningParserManager
|
||||
from vllm.tool_parsers import ToolParserManager
|
||||
from vllm.usage.usage_lib import UsageContext
|
||||
@@ -216,6 +216,7 @@ def main():
|
||||
)
|
||||
parser = make_arg_parser(parser)
|
||||
args = parser.parse_args()
|
||||
configure_logging_from_args(args)
|
||||
validate_parsed_serve_args(args)
|
||||
|
||||
uvloop.run(run_server(args))
|
||||
|
||||
@@ -134,8 +134,6 @@ class BaseFrontendArgs:
|
||||
The `demo` Python tool executes model-generated code in Docker without
|
||||
network isolation by default. See the security guide for more
|
||||
information."""
|
||||
log_config_file: str | None = envs.VLLM_LOGGING_CONFIG_PATH
|
||||
"""Path to logging config JSON file for both vllm and uvicorn"""
|
||||
max_log_len: int | None = None
|
||||
"""Max number of prompt characters or prompt ID numbers being printed in
|
||||
log. The default of None means unlimited."""
|
||||
|
||||
@@ -23,7 +23,7 @@ import uvloop
|
||||
from fastapi import FastAPI, Response
|
||||
|
||||
import vllm.envs as envs
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging_from_args, init_logger
|
||||
from vllm.utils.system_utils import (
|
||||
decorate_logs,
|
||||
kill_process_tree,
|
||||
@@ -251,6 +251,8 @@ def _run_rust_vllm_dp_server(child_args: argparse.Namespace) -> None:
|
||||
|
||||
def _run_vllm_dp_server(child_args: argparse.Namespace) -> None:
|
||||
"""Entrypoint function for the vLLM DP Server."""
|
||||
configure_logging_from_args(child_args)
|
||||
|
||||
# Create a fresh process group for the vLLM DP Server,
|
||||
# so that CTRL-C is propagated cleanly.
|
||||
os.setpgrp()
|
||||
@@ -558,4 +560,5 @@ class DPSupervisor:
|
||||
|
||||
|
||||
def run_dp_supervisor(args: argparse.Namespace) -> None:
|
||||
configure_logging_from_args(args)
|
||||
uvloop.run(DPSupervisor(args).run())
|
||||
|
||||
@@ -44,7 +44,7 @@ import uvloop
|
||||
from vllm import envs
|
||||
from vllm.engine.arg_utils import AsyncEngineArgs
|
||||
from vllm.entrypoints.serve.utils.api_utils import log_version_and_model
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging_from_args, init_logger
|
||||
from vllm.usage.usage_lib import UsageContext
|
||||
from vllm.utils.argparse_utils import FlexibleArgumentParser
|
||||
from vllm.v1.engine.async_llm import AsyncLLM
|
||||
@@ -195,6 +195,7 @@ def main():
|
||||
parser = AsyncEngineArgs.add_cli_args(parser)
|
||||
|
||||
args = parser.parse_args()
|
||||
configure_logging_from_args(args)
|
||||
|
||||
# Run server
|
||||
try:
|
||||
|
||||
@@ -8,7 +8,7 @@ from argparse import Namespace
|
||||
|
||||
from vllm import AsyncEngineArgs, envs
|
||||
from vllm.config import VllmConfig
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging_from_args, init_logger
|
||||
|
||||
from ..app import build_app
|
||||
from ..launcher import serve_http, setup_server
|
||||
@@ -87,7 +87,10 @@ async def run_launch_fastapi(args: argparse.Namespace) -> None:
|
||||
# cache space warning from CpuPlatform.check_and_update_config.
|
||||
envs.VLLM_CPU_KVCACHE_SPACE = 0
|
||||
|
||||
vllm_config = VllmConfig(model_config=model_config)
|
||||
vllm_config = VllmConfig(
|
||||
model_config=model_config,
|
||||
logging_config=engine_args.create_logging_config(),
|
||||
)
|
||||
shutdown_task = await build_and_serve_renderer(
|
||||
vllm_config, listen_address, sock, args
|
||||
)
|
||||
@@ -115,6 +118,7 @@ if __name__ == "__main__":
|
||||
)
|
||||
parser = make_arg_parser(parser)
|
||||
args = parser.parse_args()
|
||||
configure_logging_from_args(args)
|
||||
validate_parsed_serve_args(args)
|
||||
|
||||
uvloop.run(run_launch_fastapi(args))
|
||||
|
||||
@@ -61,7 +61,7 @@ from vllm.entrypoints.speech_to_text.translation.protocol import (
|
||||
TranslationResponseVerbose,
|
||||
)
|
||||
from vllm.exceptions import VLLMValidationError
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging_from_args, init_logger
|
||||
from vllm.reasoning import ReasoningParserManager
|
||||
from vllm.utils import random_uuid
|
||||
from vllm.utils.argparse_utils import FlexibleArgumentParser
|
||||
@@ -289,6 +289,7 @@ def make_arg_parser(parser: FlexibleArgumentParser):
|
||||
def parse_args():
|
||||
parser = FlexibleArgumentParser(description="vLLM OpenAI-Compatible batch runner.")
|
||||
args = make_arg_parser(parser).parse_args()
|
||||
configure_logging_from_args(args)
|
||||
|
||||
# Backward compatibility: If --url is set, use it for host
|
||||
url_explicit = any(arg == "--url" or arg.startswith("--url=") for arg in sys.argv)
|
||||
|
||||
@@ -38,7 +38,13 @@ def get_uvicorn_log_config(args: Namespace) -> dict | None:
|
||||
3. Otherwise, return None (use uvicorn defaults)
|
||||
"""
|
||||
# First, try to load from file if specified
|
||||
log_config = load_log_config(args.log_config_file)
|
||||
logging_config = getattr(args, "logging_config", None)
|
||||
log_config_file = (
|
||||
logging_config.pylogging_config_file
|
||||
if logging_config is not None
|
||||
else getattr(args, "log_config_file", None)
|
||||
)
|
||||
log_config = load_log_config(log_config_file)
|
||||
if log_config is not None:
|
||||
return log_config
|
||||
|
||||
|
||||
@@ -42,7 +42,7 @@ from vllm.entrypoints.generate.beam_search.offline import BeamSearchOfflineMixin
|
||||
from vllm.entrypoints.pooling.offline import PoolingOfflineMixin
|
||||
from vllm.entrypoints.serve.utils.api_utils import log_non_default_args
|
||||
from vllm.inputs import PromptType
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging_if_needed, init_logger
|
||||
from vllm.lora.request import LoRARequest
|
||||
from vllm.model_executor.layers.quantization import QuantizationMethods
|
||||
from vllm.outputs import PoolingRequestOutput, RequestOutput
|
||||
@@ -338,6 +338,7 @@ class LLM(BeamSearchOfflineMixin, PoolingOfflineMixin, OfflineInferenceMixin):
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
configure_logging_if_needed(engine_args.create_logging_config())
|
||||
log_non_default_args(engine_args)
|
||||
|
||||
self.llm_engine = LLMEngine.from_engine_args(
|
||||
|
||||
@@ -31,7 +31,7 @@ class RequestLogger:
|
||||
"`--enable-log-requests` is set but "
|
||||
"the minimum log level is higher than DEBUG. "
|
||||
"Only limited information will be logged to minimize overhead. "
|
||||
"To view more details, set `VLLM_LOGGING_LEVEL=DEBUG`."
|
||||
"To view more details, set `--log-level DEBUG`."
|
||||
)
|
||||
|
||||
def log_inputs(
|
||||
|
||||
+86
-32
@@ -9,16 +9,21 @@ import os
|
||||
import sys
|
||||
from collections.abc import Generator, Hashable
|
||||
from contextlib import contextmanager
|
||||
from copy import deepcopy
|
||||
from dataclasses import replace
|
||||
from functools import lru_cache, partial
|
||||
from logging import Logger
|
||||
from logging.config import dictConfig
|
||||
from os import path
|
||||
from types import MethodType
|
||||
from typing import Any, Literal, cast
|
||||
from typing import TYPE_CHECKING, Any, Literal, cast
|
||||
|
||||
import vllm.envs as envs
|
||||
from vllm.logging_utils import ColoredFormatter, NewLineFormatter
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from vllm.config.logging import LoggingConfig
|
||||
|
||||
_FORMAT = (
|
||||
f"{envs.VLLM_LOGGING_PREFIX}%(levelname)s %(asctime)s "
|
||||
"[%(fileinfo)s:%(lineno)d] %(message)s"
|
||||
@@ -40,15 +45,18 @@ def _use_color() -> bool:
|
||||
|
||||
DEFAULT_LOGGING_CONFIG: dict[str, dict[str, Any] | Any] = {
|
||||
"formatters": {
|
||||
# The "()" factory form forwards custom kwargs such as log_level.
|
||||
"vllm": {
|
||||
"class": "vllm.logging_utils.NewLineFormatter",
|
||||
"()": "vllm.logging_utils.NewLineFormatter",
|
||||
"datefmt": _DATE_FORMAT,
|
||||
"format": _FORMAT,
|
||||
"log_level": envs.VLLM_LOGGING_LEVEL,
|
||||
},
|
||||
"vllm_color": {
|
||||
"class": "vllm.logging_utils.ColoredFormatter",
|
||||
"()": "vllm.logging_utils.ColoredFormatter",
|
||||
"datefmt": _DATE_FORMAT,
|
||||
"format": _FORMAT,
|
||||
"log_level": envs.VLLM_LOGGING_LEVEL,
|
||||
},
|
||||
},
|
||||
"handlers": {
|
||||
@@ -71,6 +79,8 @@ DEFAULT_LOGGING_CONFIG: dict[str, dict[str, Any] | Any] = {
|
||||
"disable_existing_loggers": False,
|
||||
}
|
||||
|
||||
_last_configured_logging_config: "LoggingConfig | None" = None
|
||||
|
||||
|
||||
@lru_cache
|
||||
def _print_debug_once(logger: Logger, msg: str, *args: Hashable) -> None:
|
||||
@@ -165,36 +175,46 @@ _METHODS_TO_PATCH = {
|
||||
}
|
||||
|
||||
|
||||
def _configure_vllm_root_logger() -> None:
|
||||
def _configure_vllm_root_logger(config: "LoggingConfig | None" = None) -> None:
|
||||
"""Configure logging from explicit config or bootstrap environment values."""
|
||||
logging_config: dict[str, dict[str, Any] | Any] = {}
|
||||
if config is None:
|
||||
configure_logging = envs.VLLM_CONFIGURE_LOGGING
|
||||
log_level = envs.VLLM_LOGGING_LEVEL
|
||||
log_config_file = envs.VLLM_LOGGING_CONFIG_PATH
|
||||
else:
|
||||
configure_logging = config.configure_logging
|
||||
log_level = config.log_level
|
||||
log_config_file = config.pylogging_config_file
|
||||
|
||||
if not envs.VLLM_CONFIGURE_LOGGING and envs.VLLM_LOGGING_CONFIG_PATH:
|
||||
if not configure_logging and log_config_file:
|
||||
raise RuntimeError(
|
||||
"VLLM_CONFIGURE_LOGGING evaluated to false, but "
|
||||
"VLLM_LOGGING_CONFIG_PATH was given. VLLM_LOGGING_CONFIG_PATH "
|
||||
"implies VLLM_CONFIGURE_LOGGING. Please enable "
|
||||
"VLLM_CONFIGURE_LOGGING or unset VLLM_LOGGING_CONFIG_PATH."
|
||||
"Logging configuration is disabled, but a Python logging config "
|
||||
"file was given. pylogging_config_file requires "
|
||||
"configure_logging to be enabled."
|
||||
)
|
||||
|
||||
if envs.VLLM_CONFIGURE_LOGGING:
|
||||
logging_config = DEFAULT_LOGGING_CONFIG
|
||||
if configure_logging:
|
||||
logging_config = deepcopy(DEFAULT_LOGGING_CONFIG)
|
||||
|
||||
vllm_handler = logging_config["handlers"]["vllm"]
|
||||
# Refresh these values in case env vars have changed.
|
||||
vllm_handler["level"] = envs.VLLM_LOGGING_LEVEL
|
||||
vllm_handler["level"] = log_level
|
||||
vllm_handler["stream"] = envs.VLLM_LOGGING_STREAM
|
||||
vllm_handler["formatter"] = "vllm_color" if _use_color() else "vllm"
|
||||
|
||||
vllm_loggers = logging_config["loggers"]["vllm"]
|
||||
vllm_loggers["level"] = envs.VLLM_LOGGING_LEVEL
|
||||
vllm_loggers["level"] = log_level
|
||||
for formatter in logging_config["formatters"].values():
|
||||
formatter["log_level"] = log_level
|
||||
|
||||
if envs.VLLM_LOGGING_CONFIG_PATH:
|
||||
if not path.exists(envs.VLLM_LOGGING_CONFIG_PATH):
|
||||
if log_config_file:
|
||||
if not path.exists(log_config_file):
|
||||
raise RuntimeError(
|
||||
"Could not load logging config. File does not exist: %s",
|
||||
envs.VLLM_LOGGING_CONFIG_PATH,
|
||||
log_config_file,
|
||||
)
|
||||
with open(envs.VLLM_LOGGING_CONFIG_PATH, encoding="utf-8") as file:
|
||||
with open(log_config_file, encoding="utf-8") as file:
|
||||
custom_config = json.loads(file.read())
|
||||
|
||||
if not isinstance(custom_config, dict):
|
||||
@@ -212,11 +232,57 @@ def _configure_vllm_root_logger() -> None:
|
||||
if logging_config:
|
||||
dictConfig(logging_config)
|
||||
|
||||
# Transformers uses httpx to access the Hugging Face Hub. httpx is quite verbose,
|
||||
# so we set its logging level to WARNING when vLLM's logging level is INFO.
|
||||
# httpx2 is the successor huggingface_hub switches to in its 2.x releases.
|
||||
if log_level == "INFO":
|
||||
logging.getLogger("httpx").setLevel(logging.WARNING)
|
||||
logging.getLogger("httpx2").setLevel(logging.WARNING)
|
||||
|
||||
|
||||
def configure_logging(config: "LoggingConfig") -> None:
|
||||
"""Apply a logging configuration in the current process."""
|
||||
_configure_vllm_root_logger(config)
|
||||
global _last_configured_logging_config
|
||||
_last_configured_logging_config = config
|
||||
_log_platform_warnings(config)
|
||||
|
||||
|
||||
def _log_platform_warnings(config: "LoggingConfig") -> None:
|
||||
"""Emit platform diagnostics only after an enabled config is active."""
|
||||
if not config.configure_logging:
|
||||
return
|
||||
|
||||
# Import lazily because platform modules use init_logger during import.
|
||||
from vllm.platforms import current_platform
|
||||
|
||||
current_platform.log_warnings()
|
||||
|
||||
|
||||
def configure_logging_if_needed(config: "LoggingConfig") -> None:
|
||||
"""Apply a logging configuration unless it is already active in this process."""
|
||||
if config != _last_configured_logging_config:
|
||||
configure_logging(config)
|
||||
|
||||
|
||||
def configure_logging_from_args(args: Any) -> "LoggingConfig":
|
||||
"""Apply parsed logging arguments and retain them for child processes."""
|
||||
from vllm.config.logging import LoggingConfig
|
||||
|
||||
config = getattr(args, "logging_config", None) or LoggingConfig()
|
||||
if hasattr(args, "log_level"):
|
||||
config = replace(config, log_level=args.log_level)
|
||||
if hasattr(args, "log_config_file"):
|
||||
config = replace(config, pylogging_config_file=args.log_config_file)
|
||||
|
||||
configure_logging_if_needed(config)
|
||||
args.log_config_file = config.pylogging_config_file
|
||||
args.logging_config = config
|
||||
return config
|
||||
|
||||
|
||||
def init_logger(name: str) -> _VllmLogger:
|
||||
"""The main purpose of this function is to ensure that loggers are
|
||||
retrieved in such a way that we can be sure the root vllm logger has
|
||||
already been configured."""
|
||||
"""Retrieve a logger and add vLLM's convenience logging methods."""
|
||||
logger = logging.getLogger(name)
|
||||
|
||||
for method_name, method in _METHODS_TO_PATCH.items():
|
||||
@@ -246,18 +312,6 @@ def current_formatter_type(logger: Logger) -> Literal["color", "newline", None]:
|
||||
return None
|
||||
|
||||
|
||||
# The root logger is initialized when the module is imported.
|
||||
# This is thread-safe as the module is only imported once,
|
||||
# guaranteed by the Python GIL.
|
||||
_configure_vllm_root_logger()
|
||||
|
||||
# Transformers uses httpx to access the Hugging Face Hub. httpx is quite verbose,
|
||||
# so we set its logging level to WARNING when vLLM's logging level is INFO.
|
||||
# httpx2 is the successor huggingface_hub switches to in its 2.x releases.
|
||||
if envs.VLLM_LOGGING_LEVEL == "INFO":
|
||||
logging.getLogger("httpx").setLevel(logging.WARNING)
|
||||
logging.getLogger("httpx2").setLevel(logging.WARNING)
|
||||
|
||||
logger = init_logger(__name__)
|
||||
|
||||
|
||||
|
||||
@@ -4,16 +4,14 @@
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from vllm import envs
|
||||
|
||||
|
||||
class NewLineFormatter(logging.Formatter):
|
||||
"""Adds logging prefix to newlines to align multi-line messages."""
|
||||
|
||||
def __init__(self, fmt, datefmt=None, style="%"):
|
||||
def __init__(self, fmt, datefmt=None, style="%", *, log_level: str | None = None):
|
||||
super().__init__(fmt, datefmt, style)
|
||||
|
||||
self.use_relpath = envs.VLLM_LOGGING_LEVEL == "DEBUG"
|
||||
self.use_relpath = log_level == "DEBUG"
|
||||
if self.use_relpath:
|
||||
self.root_dir = Path(__file__).resolve().parent.parent.parent
|
||||
|
||||
@@ -95,7 +93,7 @@ class ColoredFormatter(NewLineFormatter):
|
||||
GREY = "\033[90m" # Grey for timestamp and file info
|
||||
RESET = "\033[0m"
|
||||
|
||||
def __init__(self, fmt, datefmt=None, style="%"):
|
||||
def __init__(self, fmt, datefmt=None, style="%", *, log_level: str | None = None):
|
||||
# Inject grey color codes into format string for timestamp and file info
|
||||
if fmt:
|
||||
# Wrap %(asctime)s with grey
|
||||
@@ -107,7 +105,7 @@ class ColoredFormatter(NewLineFormatter):
|
||||
)
|
||||
|
||||
# Call parent __init__ with potentially modified format string
|
||||
super().__init__(fmt, datefmt, style)
|
||||
super().__init__(fmt, datefmt, style, log_level=log_level)
|
||||
|
||||
def format(self, record):
|
||||
# Store original levelname to restore later (in case record is reused)
|
||||
|
||||
@@ -298,10 +298,6 @@ class CudaPlatformBase(Platform):
|
||||
def is_fully_connected(cls, device_ids: list[int]) -> bool:
|
||||
raise NotImplementedError
|
||||
|
||||
@classmethod
|
||||
def log_warnings(cls):
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def is_pin_memory_available(cls) -> bool:
|
||||
if in_wsl():
|
||||
@@ -1009,11 +1005,12 @@ class NvmlCudaPlatform(CudaPlatformBase):
|
||||
len(set(device_names)) > 1
|
||||
and os.environ.get("CUDA_DEVICE_ORDER") != "PCI_BUS_ID"
|
||||
):
|
||||
logger.warning(
|
||||
logger.warning_once(
|
||||
"Detected different devices in the system: %s. Please"
|
||||
" make sure to set `CUDA_DEVICE_ORDER=PCI_BUS_ID` to "
|
||||
"avoid unexpected behavior.",
|
||||
", ".join(device_names),
|
||||
scope="process",
|
||||
)
|
||||
|
||||
|
||||
@@ -1065,5 +1062,3 @@ finally:
|
||||
pynvml.nvmlShutdown()
|
||||
|
||||
CudaPlatform = NvmlCudaPlatform if nvml_available else NonNvmlCudaPlatform
|
||||
|
||||
CudaPlatform.log_warnings()
|
||||
|
||||
@@ -147,6 +147,10 @@ class Platform:
|
||||
# empty string means the device does not support ray
|
||||
ray_device_key: str = ""
|
||||
|
||||
@classmethod
|
||||
def log_warnings(cls) -> None:
|
||||
"""Log platform-specific diagnostics after logging is configured."""
|
||||
|
||||
# platform-agnostic way to specify the device control environment variable,
|
||||
# .e.g. CUDA_VISIBLE_DEVICES for CUDA.
|
||||
# hint: search for "get_visible_accelerator_ids_env_var" in
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import logging
|
||||
import multiprocessing
|
||||
import os
|
||||
import signal
|
||||
@@ -63,7 +64,7 @@ def suppress_stdout():
|
||||
"""Suppress stdout from C libraries at the file descriptor level.
|
||||
|
||||
Only suppresses stdout, not stderr, to preserve error messages.
|
||||
Suppression is disabled when VLLM_LOGGING_LEVEL is set to DEBUG.
|
||||
Suppression is disabled when vLLM debug logging is enabled.
|
||||
|
||||
Example:
|
||||
with suppress_stdout():
|
||||
@@ -72,7 +73,7 @@ def suppress_stdout():
|
||||
|
||||
"""
|
||||
# Don't suppress if logging level is DEBUG
|
||||
if envs.VLLM_LOGGING_LEVEL == "DEBUG":
|
||||
if logger.isEnabledFor(logging.DEBUG):
|
||||
yield
|
||||
return
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ from vllm.exceptions import (
|
||||
VLLMValidationError,
|
||||
)
|
||||
from vllm.inputs import EngineInput, PromptType
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging_if_needed, init_logger
|
||||
from vllm.lora.request import LoRARequest
|
||||
from vllm.multimodal import MULTIMODAL_REGISTRY, MultiModalRegistry
|
||||
from vllm.outputs import STREAM_FINISHED, PoolingRequestOutput, RequestOutput
|
||||
@@ -122,6 +122,8 @@ class AsyncLLM(EngineClient):
|
||||
None
|
||||
|
||||
"""
|
||||
configure_logging_if_needed(vllm_config.logging_config)
|
||||
|
||||
# Ensure we can serialize custom transformer configs
|
||||
maybe_register_config_serialize_by_value()
|
||||
|
||||
|
||||
@@ -9,8 +9,8 @@ import weakref
|
||||
import msgspec.msgpack
|
||||
import zmq
|
||||
|
||||
from vllm.config import ParallelConfig
|
||||
from vllm.logger import init_logger
|
||||
from vllm.config import LoggingConfig, ParallelConfig
|
||||
from vllm.logger import configure_logging, init_logger
|
||||
from vllm.utils.network_utils import make_zmq_socket
|
||||
from vllm.utils.system_utils import get_mp_context, set_process_title
|
||||
from vllm.v1.engine import EngineCoreOutputs, EngineCoreRequestType
|
||||
@@ -77,7 +77,10 @@ class DPCoordinator:
|
||||
zmq_addr_pipe.close()
|
||||
|
||||
def __init__(
|
||||
self, parallel_config: ParallelConfig, enable_wave_coordination: bool = True
|
||||
self,
|
||||
parallel_config: ParallelConfig,
|
||||
enable_wave_coordination: bool = True,
|
||||
logging_config: LoggingConfig | None = None,
|
||||
):
|
||||
dp_size = parallel_config.data_parallel_size
|
||||
assert dp_size > 1, "Coordinator only used for data parallel"
|
||||
@@ -108,6 +111,7 @@ class DPCoordinator:
|
||||
"back_publish_address": back_publish_address,
|
||||
"zmq_addr_pipe": child_zmq_addr_pipe,
|
||||
"enable_wave_coordination": enable_wave_coordination,
|
||||
"logging_config": logging_config,
|
||||
},
|
||||
daemon=True,
|
||||
)
|
||||
@@ -167,7 +171,11 @@ class DPCoordinatorProc:
|
||||
zmq_addr_pipe=None,
|
||||
min_stats_update_interval_ms: int = 100,
|
||||
enable_wave_coordination: bool = True,
|
||||
logging_config: LoggingConfig | None = None,
|
||||
):
|
||||
if logging_config is not None:
|
||||
configure_logging(logging_config)
|
||||
|
||||
coordinator = DPCoordinatorProc(
|
||||
engine_count=engine_count,
|
||||
min_stats_update_interval_ms=min_stats_update_interval_ms,
|
||||
|
||||
@@ -28,7 +28,7 @@ from vllm.distributed import (
|
||||
stateless_destroy_torch_distributed_process_group,
|
||||
)
|
||||
from vllm.envs import enable_envs_cache
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging, init_logger
|
||||
from vllm.logging_utils.dump_input import dump_engine_exception
|
||||
from vllm.lora.request import LoRARequest
|
||||
from vllm.multimodal.cache import (
|
||||
@@ -1350,6 +1350,10 @@ class EngineCoreProc(EngineCore):
|
||||
@staticmethod
|
||||
def run_engine_core(*args, dp_rank: int = 0, local_dp_rank: int = 0, **kwargs):
|
||||
"""Launch EngineCore busy loop in background process."""
|
||||
vllm_config: VllmConfig = kwargs["vllm_config"]
|
||||
if logging_config := getattr(vllm_config, "logging_config", None):
|
||||
configure_logging(logging_config)
|
||||
|
||||
# Ensure we can serialize transformer config after spawning
|
||||
maybe_register_config_serialize_by_value()
|
||||
|
||||
@@ -1357,7 +1361,6 @@ class EngineCoreProc(EngineCore):
|
||||
signal_callback: SignalCallback | None = None
|
||||
clean_shutdown = False
|
||||
try:
|
||||
vllm_config: VllmConfig = kwargs["vllm_config"]
|
||||
parallel_config: ParallelConfig = vllm_config.parallel_config
|
||||
data_parallel = parallel_config.data_parallel_size > 1 or dp_rank > 0
|
||||
if data_parallel:
|
||||
@@ -2473,6 +2476,9 @@ class EngineCoreActorMixin:
|
||||
dp_rank: int = 0,
|
||||
local_dp_rank: int = 0,
|
||||
):
|
||||
if logging_config := getattr(vllm_config, "logging_config", None):
|
||||
configure_logging(logging_config)
|
||||
|
||||
# Initialize tracer for distributed tracing if configured.
|
||||
maybe_init_worker_tracer(
|
||||
instrumenting_module_name="vllm.engine_core",
|
||||
|
||||
@@ -16,7 +16,7 @@ from vllm.distributed import stateless_destroy_torch_distributed_process_group
|
||||
from vllm.distributed.parallel_state import get_dp_group
|
||||
from vllm.engine.arg_utils import EngineArgs
|
||||
from vllm.inputs import EngineInput, PromptType
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging_if_needed, init_logger
|
||||
from vllm.lora.request import LoRARequest
|
||||
from vllm.multimodal import MULTIMODAL_REGISTRY, MultiModalRegistry
|
||||
from vllm.outputs import PoolingRequestOutput, RequestOutput
|
||||
@@ -60,6 +60,7 @@ class LLMEngine:
|
||||
mm_registry: MultiModalRegistry = MULTIMODAL_REGISTRY,
|
||||
multiprocess_mode: bool = False,
|
||||
) -> None:
|
||||
configure_logging_if_needed(vllm_config.logging_config)
|
||||
self.vllm_config = vllm_config
|
||||
self.model_config = vllm_config.model_config
|
||||
self.observability_config = vllm_config.observability_config
|
||||
|
||||
@@ -1128,6 +1128,7 @@ def launch_core_engines(
|
||||
coordinator = DPCoordinator(
|
||||
parallel_config,
|
||||
enable_wave_coordination=vllm_config.model_config.is_moe,
|
||||
logging_config=vllm_config.logging_config,
|
||||
)
|
||||
|
||||
addresses.coordinator_input, addresses.coordinator_output = (
|
||||
|
||||
@@ -42,7 +42,7 @@ from vllm.distributed.parallel_state import (
|
||||
model_parallel_is_initialized,
|
||||
)
|
||||
from vllm.envs import enable_envs_cache
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging, init_logger
|
||||
from vllm.platforms import current_platform
|
||||
from vllm.tracing import instrument, maybe_init_worker_tracer
|
||||
from vllm.utils import numa_utils
|
||||
@@ -853,6 +853,9 @@ class WorkerProc:
|
||||
def worker_main(*args, **kwargs):
|
||||
"""Worker initialization and execution loops.
|
||||
This runs a background process"""
|
||||
if logging_config := getattr(kwargs["vllm_config"], "logging_config", None):
|
||||
configure_logging(logging_config)
|
||||
|
||||
# Signal handler used for graceful termination.
|
||||
# SystemExit exception is only raised once to allow this and worker
|
||||
# processes to terminate without error
|
||||
|
||||
@@ -16,7 +16,7 @@ from vllm.distributed.device_communicators.shm_broadcast import (
|
||||
Handle,
|
||||
MessageQueue,
|
||||
)
|
||||
from vllm.logger import init_logger
|
||||
from vllm.logger import configure_logging, init_logger
|
||||
from vllm.platforms import current_platform
|
||||
from vllm.utils.network_utils import (
|
||||
get_distributed_init_method,
|
||||
@@ -113,6 +113,9 @@ class RayWorkerProc(WorkerProc):
|
||||
is_driver_worker: bool,
|
||||
is_driver_node: bool = False,
|
||||
):
|
||||
if logging_config := getattr(vllm_config, "logging_config", None):
|
||||
configure_logging(logging_config)
|
||||
|
||||
# Defer WorkerProc.__init__ until GPU IDs are known.
|
||||
self._is_driver_node = is_driver_node
|
||||
self._parallel_config = vllm_config.parallel_config
|
||||
|
||||
@@ -520,6 +520,11 @@ def run_api_server_worker_proc(
|
||||
listen_address, sock, args, client_config=None, **uvicorn_kwargs
|
||||
) -> None:
|
||||
"""Entrypoint for individual API server worker processes."""
|
||||
if logging_config := getattr(args, "logging_config", None):
|
||||
from vllm.logger import configure_logging
|
||||
|
||||
configure_logging(logging_config)
|
||||
|
||||
from vllm.entrypoints.launchers.api_server.entry import run_server_worker
|
||||
|
||||
client_config = client_config or {}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
|
||||
import logging
|
||||
import threading
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
@@ -8,7 +9,6 @@ from typing import Any
|
||||
|
||||
import torch
|
||||
|
||||
import vllm.envs as envs
|
||||
from vllm.compilation.cuda_graph import CUDAGraphWrapper
|
||||
from vllm.config import CUDAGraphMode, VllmConfig
|
||||
from vllm.distributed.device_communicators.pynccl_allocator import set_graph_pool_id
|
||||
@@ -89,7 +89,7 @@ class UBatchWrapper:
|
||||
|
||||
self.sm_control = create_sm_control_context(vllm_config.parallel_config)
|
||||
self.device = device
|
||||
self.is_debugging_mode = envs.VLLM_LOGGING_LEVEL == "DEBUG"
|
||||
self.is_debugging_mode = logger.isEnabledFor(logging.DEBUG)
|
||||
self._runnable_str = str(runnable) if self.is_debugging_mode else None
|
||||
|
||||
@property
|
||||
|
||||
Reference in New Issue
Block a user