[Core][Logging] Add built-in JSON formatter (#58739)

Add `--logging-config.formatter=json` to enable the python-json-logger
formatter without having to use a complete Python logging dictConfig.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Mark McLoughlin
2026-10-01 09:11:46 +01:00
committed by GitHub
co-authored by OpenAI Codex
parent c4fc3c7d32
commit 91fcec1235
11 changed files with 148 additions and 46 deletions
+2 -2
View File
@@ -530,7 +530,7 @@ steps:
- vllm/v1/ - vllm/v1/
- vllm/version.py - vllm/version.py
- cmake/utils.cmake - cmake/utils.cmake
- tests/test_access_log_filter.py - tests/test_uvicorn_logging.py
- tests/test_attention_backend_registry.py - tests/test_attention_backend_registry.py
- tests/test_audio_media_size_precheck.py - tests/test_audio_media_size_precheck.py
- tests/test_cmake_utils.py - tests/test_cmake_utils.py
@@ -571,7 +571,7 @@ steps:
- pip install cmake # test_cmake_utils.py shells out to cmake; the CPU image has none - pip install cmake # test_cmake_utils.py shells out to cmake; the CPU image has none
- python3 standalone_tests/lazy_imports.py - python3 standalone_tests/lazy_imports.py
- pytest -v -s device_allocator - pytest -v -s device_allocator
- pytest -v -s test_access_log_filter.py - pytest -v -s test_uvicorn_logging.py
- pytest -v -s test_attention_backend_registry.py - pytest -v -s test_attention_backend_registry.py
- pytest -v -s test_audio_media_size_precheck.py - pytest -v -s test_audio_media_size_precheck.py
- pytest -v -s test_cmake_utils.py - pytest -v -s test_cmake_utils.py
+53 -17
View File
@@ -6,6 +6,7 @@ robust and flexible configuration of the various loggers used by vLLM.
For `vllm serve`, configure logging with CLI arguments: For `vllm serve`, configure logging with CLI arguments:
- Use the built-in configuration, optionally with `--log-level`. - Use the built-in configuration, optionally with `--log-level`.
- Emit structured JSON with `--logging-config.formatter json`.
- Use a custom Python logging configuration file with - Use a custom Python logging configuration file with
`--logging-config.pylogging_config_file`. `--logging-config.pylogging_config_file`.
- Disable vLLM logging configuration with - Disable vLLM logging configuration with
@@ -14,7 +15,7 @@ For `vllm serve`, configure logging with CLI arguments:
## CLI logging configuration ## CLI logging configuration
`--logging-config` accepts a JSON object. Its fields are `log_level`, `--logging-config` accepts a JSON object. Its fields are `log_level`,
`configure_logging`, and `pylogging_config_file`. For example: `formatter`, `configure_logging`, and `pylogging_config_file`. For example:
```bash ```bash
vllm serve mistralai/Mistral-7B-v0.1 \ vllm serve mistralai/Mistral-7B-v0.1 \
@@ -28,6 +29,25 @@ vllm serve mistralai/Mistral-7B-v0.1 \
--logging-config.log_level DEBUG --logging-config.log_level DEBUG
``` ```
The built-in `text` formatter is selected by default. To emit one JSON object
per vLLM and Uvicorn log record, select the built-in `json` formatter:
```bash
vllm serve mistralai/Mistral-7B-v0.1 \
--logging-config.formatter json
```
The built-in JSON output includes `asctime`, `levelname`, `name`,
`processName`, `process`, `message`, and `vllm_process_name`. `processName` is
Python's process name, `vllm_process_name` identifies vLLM's logical process
(including worker ranks where applicable), and `process` is the
operating-system PID.
The built-in profile formats records emitted through vLLM's Python loggers and
Uvicorn's server and access loggers when using `vllm serve`. It does not convert
direct writes to `stdout` or `stderr`. Use a custom configuration to format
other loggers.
`--log-level` is a shortcut for `--logging-config.log_level` and takes `--log-level` is a shortcut for `--logging-config.log_level` and takes
precedence if both are supplied. It sets the level of vLLM's built-in logging precedence if both are supplied. It sets the level of vLLM's built-in logging
configuration. `--log-config-file` is deprecated and will be removed in configuration. `--log-config-file` is deprecated and will be removed in
@@ -41,11 +61,11 @@ The custom logging configuration file must be JSON following Python's [logging
configuration dictionary configuration dictionary
schema](https://docs.python.org/3/library/logging.config.html#dictionary-schema-details). schema](https://docs.python.org/3/library/logging.config.html#dictionary-schema-details).
!!! note "Custom configurations override `--log-level`" !!! note "Custom configurations override built-in formatter settings"
When `pylogging_config_file` is set, vLLM loads that JSON file and replaces When `pylogging_config_file` is set, vLLM loads that JSON file and replaces
its built-in `dictConfig`; it does not merge the two. Therefore, its built-in `dictConfig`; it does not merge the two. Therefore, `log_level`
`--log-level` applies only when no custom configuration file is provided. and `formatter` apply only when no custom configuration file is provided.
Set logger and handler levels in the custom file itself. vLLM applies the Set levels and formatters in the custom file itself. vLLM applies the
resolved configuration in its child processes as well. resolved configuration in its child processes as well.
## Environment variables ## Environment variables
@@ -62,10 +82,11 @@ for those settings.
### Example 1: Customize vLLM root logger ### Example 1: Customize vLLM root logger
For this example, we will customize the vLLM root logger to use For more control over fields and handlers, configure the vLLM root logger and
Uvicorn's server and access loggers with
[`python-json-logger`](https://github.com/nhairs/python-json-logger) [`python-json-logger`](https://github.com/nhairs/python-json-logger)
(which is part of the container image) to log to (which is part of the container image) to log to STDOUT in JSON format at
STDOUT of the console in JSON format with a log level of `INFO`. `INFO` level.
To begin, first, create an appropriate JSON logging configuration file: To begin, first, create an appropriate JSON logging configuration file:
@@ -92,8 +113,24 @@ To begin, first, create an appropriate JSON logging configuration file:
"handlers": ["console"], "handlers": ["console"],
"level": "INFO", "level": "INFO",
"propagate": false "propagate": false
},
"uvicorn": {
"handlers": ["console"],
"level": "INFO",
"propagate": false
},
"uvicorn.error": {
"handlers": ["console"],
"level": "INFO",
"propagate": false
},
"uvicorn.access": {
"handlers": ["console"],
"level": "INFO",
"propagate": false
} }
}, },
"disable_existing_loggers": false,
"version": 1 "version": 1
} }
``` ```
@@ -105,21 +142,20 @@ vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
--logging-config.pylogging_config_file /path/to/logging_config.json --logging-config.pylogging_config_file /path/to/logging_config.json
``` ```
Each vLLM log record is one JSON object and includes `vllm_process_name`, which With this custom configuration, each vLLM and Uvicorn log record is one JSON
identifies vLLM's logical process (including worker ranks where applicable). object. The example selects `vllm_process_name` and `process` explicitly; both
The standard `process` record attribute contains the operating-system PID. fields are also present in the built-in JSON output described above. vLLM
vLLM avoids altering `stdout` or `stderr`, so no text is prepended to JSON avoids altering `stdout` or `stderr`, so no text is prepended to JSON log
log records. records.
This applies to records emitted through the configured Python loggers. A JSON This applies to records emitted through the configured Python loggers. A JSON
formatter cannot convert unrelated output into JSON, such as a third-party formatter cannot convert unrelated output into JSON, such as a third-party
library writing directly to `stdout` or `stderr`; configure or route such library writing directly to `stdout` or `stderr`; configure or route such
output separately when a consumer requires every collected line to be JSON. output separately when a consumer requires every collected line to be JSON.
When serving, `VLLM_LOGGING_CONFIG_PATH` is also used as Uvicorn's logging When serving, vLLM also passes this file to Uvicorn as its logging
configuration. This example configures only `vllm`; configure `uvicorn`, configuration. The `uvicorn`, `uvicorn.error`, and `uvicorn.access` entries
`uvicorn.error`, and `uvicorn.access` with a JSON handler when those server format its server and access logs as JSON.
logs must also be structured.
### Example 2: Silence a particular vLLM logger ### Example 2: Silence a particular vLLM logger
@@ -76,6 +76,8 @@ def test_logging_config_cli_args(serve_parser):
[ [
"--logging-config", "--logging-config",
'{"log_level":"WARNING","pylogging_config_file":"/tmp/json.json"}', '{"log_level":"WARNING","pylogging_config_file":"/tmp/json.json"}',
"--logging-config.formatter",
"json",
"--log-level", "--log-level",
"DEBUG", "DEBUG",
"--log-config-file", "--log-config-file",
@@ -85,6 +87,7 @@ def test_logging_config_cli_args(serve_parser):
config = AsyncEngineArgs.from_cli_args(args).create_logging_config() config = AsyncEngineArgs.from_cli_args(args).create_logging_config()
assert config.log_level == "DEBUG" assert config.log_level == "DEBUG"
assert config.formatter == "json"
assert config.pylogging_config_file == "/tmp/flat.json" assert config.pylogging_config_file == "/tmp/flat.json"
+31
View File
@@ -4,6 +4,7 @@ import enum
import io import io
import json import json
import logging import logging
import multiprocessing
import os import os
import sys import sys
import tempfile import tempfile
@@ -21,6 +22,7 @@ from vllm.config import LoggingConfig
from vllm.logger import ( from vllm.logger import (
_DATE_FORMAT, _DATE_FORMAT,
_FORMAT, _FORMAT,
_JSON_FORMAT,
_configure_vllm_root_logger, _configure_vllm_root_logger,
_use_color, _use_color,
configure_logging, configure_logging,
@@ -204,6 +206,35 @@ def test_configure_logging_preserves_application_record_factory(
logging.setLogRecordFactory(original_factory) logging.setLogRecordFactory(original_factory)
def test_builtin_json_formatter(monkeypatch):
output = io.StringIO()
try:
with monkeypatch.context() as context:
context.setattr(sys, "stdout", output)
context.setattr(vllm_logger, "_vllm_process_info", None)
_configure_vllm_root_logger(LoggingConfig(formatter="json"))
decorate_logs("Worker_DP0")
init_logger("vllm.structured_log_probe").info("structured log probe")
log = json.loads(output.getvalue())
formatter = logging.getLogger("vllm").handlers[0].formatter
finally:
_configure_vllm_root_logger(LoggingConfig())
assert log == {
"asctime": log["asctime"],
"levelname": "INFO",
"name": "vllm.structured_log_probe",
"processName": multiprocessing.current_process().name,
"process": os.getpid(),
"message": "structured log probe",
"vllm_process_name": "Worker_DP0",
}
assert formatter is not None
assert formatter._fmt == _JSON_FORMAT
def test_use_color_force_color(monkeypatch): def test_use_color_force_color(monkeypatch):
"""FORCE_COLOR forces colored logs without a TTY, while NO_COLOR and an """FORCE_COLOR forces colored logs without a TTY, while NO_COLOR and an
explicit VLLM_LOGGING_COLOR=0 take precedence over it.""" explicit VLLM_LOGGING_COLOR=0 take precedence over it."""
@@ -1,10 +1,10 @@
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
"""Tests for the UvicornAccessLogFilter class.""" """Tests for Uvicorn logging configuration and access log filtering."""
import logging import logging
from vllm.logging_utils.access_log_filter import ( from vllm.logging_utils.uvicorn_logging import (
UvicornAccessLogFilter, UvicornAccessLogFilter,
create_uvicorn_log_config, create_uvicorn_log_config,
) )
+8 -4
View File
@@ -9,6 +9,7 @@ import vllm.envs as envs
from vllm.config.utils import config from vllm.config.utils import config
LogLevel = Literal["CRITICAL", "ERROR", "WARNING", "INFO", "DEBUG", "NOTSET"] LogLevel = Literal["CRITICAL", "ERROR", "WARNING", "INFO", "DEBUG", "NOTSET"]
LogFormatter = Literal["text", "json"]
@config @config
@@ -17,8 +18,8 @@ class LoggingConfig:
Supply a JSON object with ``--logging-config`` or individual fields with Supply a JSON object with ``--logging-config`` or individual fields with
dotted arguments such as ``--logging-config.log_level DEBUG`` and dotted arguments such as ``--logging-config.log_level DEBUG`` and
``--logging-config.pylogging_config_file logging.json``. ``--log-level`` ``--logging-config.formatter json``. ``--log-level`` and legacy
and legacy ``--log-config-file`` override their matching fields. Set ``--log-config-file`` override their matching fields. Set
``configure_logging`` to false to skip applying a ``dictConfig``. ``configure_logging`` to false to skip applying a ``dictConfig``.
""" """
@@ -27,6 +28,9 @@ class LoggingConfig:
) )
"""Log level used when no custom logging configuration is provided.""" """Log level used when no custom logging configuration is provided."""
formatter: LogFormatter = "text"
"""Formatter used by the built-in logging configuration."""
configure_logging: bool = Field(default_factory=lambda: envs.VLLM_CONFIGURE_LOGGING) configure_logging: bool = Field(default_factory=lambda: envs.VLLM_CONFIGURE_LOGGING)
"""Whether to apply a Python logging configuration. """Whether to apply a Python logging configuration.
@@ -41,6 +45,6 @@ class LoggingConfig:
"""Path to a Python logging JSON file using """Path to a Python logging JSON file using
[``dictConfig`` schema](https://docs.python.org/3/library/logging.config.html#configuration-file-format). [``dictConfig`` schema](https://docs.python.org/3/library/logging.config.html#configuration-file-format).
A custom ``dictConfig`` is authoritative over ``log_level`` for logger, A custom ``dictConfig`` is authoritative over ``log_level`` and
handler, and formatter settings. ``formatter`` for logger, handler, and formatter settings.
""" """
@@ -8,6 +8,7 @@ from contextlib import asynccontextmanager
from fastapi import FastAPI from fastapi import FastAPI
from vllm import envs from vllm import envs
from vllm.config.logging import LogFormatter
from vllm.engine.protocol import EngineClient from vllm.engine.protocol import EngineClient
from vllm.logger import init_logger from vllm.logger import init_logger
from vllm.utils.gc_utils import freeze_gc_heap from vllm.utils.gc_utils import freeze_gc_heap
@@ -33,8 +34,8 @@ def get_uvicorn_log_config(args: Namespace) -> dict | None:
Priority: Priority:
1. If log_config_file is specified, use it 1. If log_config_file is specified, use it
2. If disable_access_log_for_endpoints is specified, create a config with 2. If disable_access_log_for_endpoints or JSON formatting is specified,
the access log filter create a config with the requested filter and format
3. Otherwise, return None (use uvicorn defaults) 3. Otherwise, return None (use uvicorn defaults)
""" """
# First, try to load from file if specified # First, try to load from file if specified
@@ -48,19 +49,21 @@ def get_uvicorn_log_config(args: Namespace) -> dict | None:
if log_config is not None: if log_config is not None:
return log_config return log_config
# If endpoints to filter are specified, create a config with the filter formatter: LogFormatter = (
if args.disable_access_log_for_endpoints: logging_config.formatter if logging_config is not None else "text"
)
if args.disable_access_log_for_endpoints or formatter == "json":
from vllm.logging_utils import create_uvicorn_log_config from vllm.logging_utils import create_uvicorn_log_config
# Parse comma-separated string into list
excluded_paths = [ excluded_paths = [
p.strip() p.strip()
for p in args.disable_access_log_for_endpoints.split(",") for p in (args.disable_access_log_for_endpoints or "").split(",")
if p.strip() if p.strip()
] ]
return create_uvicorn_log_config( return create_uvicorn_log_config(
excluded_paths=excluded_paths, excluded_paths=excluded_paths,
log_level=args.uvicorn_log_level, log_level=args.uvicorn_log_level,
formatter=formatter,
) )
return None return None
+16 -4
View File
@@ -20,6 +20,7 @@ from typing import TYPE_CHECKING, Any, Literal, cast
import vllm.envs as envs import vllm.envs as envs
from vllm.logging_utils import ColoredFormatter, NewLineFormatter from vllm.logging_utils import ColoredFormatter, NewLineFormatter
from vllm.logging_utils.formatter import JSON_FORMAT as _JSON_FORMAT
if TYPE_CHECKING: if TYPE_CHECKING:
from vllm.config.logging import LoggingConfig from vllm.config.logging import LoggingConfig
@@ -30,7 +31,6 @@ _FORMAT = (
"[%(fileinfo)s:%(lineno)d] %(message)s" "[%(fileinfo)s:%(lineno)d] %(message)s"
) )
_DATE_FORMAT = "%m-%d %H:%M:%S" _DATE_FORMAT = "%m-%d %H:%M:%S"
_vllm_process_info: tuple[str, int] | None = None _vllm_process_info: tuple[str, int] | None = None
@@ -211,10 +211,12 @@ def _configure_vllm_root_logger(config: "LoggingConfig | None" = None) -> None:
configure_logging = envs.VLLM_CONFIGURE_LOGGING configure_logging = envs.VLLM_CONFIGURE_LOGGING
log_level = envs.VLLM_LOGGING_LEVEL log_level = envs.VLLM_LOGGING_LEVEL
log_config_file = envs.VLLM_LOGGING_CONFIG_PATH log_config_file = envs.VLLM_LOGGING_CONFIG_PATH
log_formatter = "text"
else: else:
configure_logging = config.configure_logging configure_logging = config.configure_logging
log_level = config.log_level log_level = config.log_level
log_config_file = config.pylogging_config_file log_config_file = config.pylogging_config_file
log_formatter = config.formatter
if not configure_logging and log_config_file: if not configure_logging and log_config_file:
raise RuntimeError( raise RuntimeError(
@@ -233,12 +235,22 @@ def _configure_vllm_root_logger(config: "LoggingConfig | None" = None) -> None:
# Refresh these values in case env vars have changed. # Refresh these values in case env vars have changed.
vllm_handler["level"] = log_level vllm_handler["level"] = log_level
vllm_handler["stream"] = envs.VLLM_LOGGING_STREAM vllm_handler["stream"] = envs.VLLM_LOGGING_STREAM
vllm_handler["formatter"] = "vllm_color" if _use_color() else "vllm" for formatter in logging_config["formatters"].values():
formatter["log_level"] = log_level
if log_formatter == "json":
logging_config["formatters"] = {
"vllm_json": {
"class": "pythonjsonlogger.jsonlogger.JsonFormatter",
"format": _JSON_FORMAT,
}
}
vllm_handler["formatter"] = "vllm_json"
else:
vllm_handler["formatter"] = "vllm_color" if _use_color() else "vllm"
vllm_loggers = logging_config["loggers"]["vllm"] vllm_loggers = logging_config["loggers"]["vllm"]
vllm_loggers["level"] = log_level vllm_loggers["level"] = log_level
for formatter in logging_config["formatters"].values():
formatter["log_level"] = log_level
if log_config_file: if log_config_file:
if not path.exists(log_config_file): if not path.exists(log_config_file):
+4 -4
View File
@@ -1,14 +1,14 @@
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
from vllm.logging_utils.access_log_filter import (
UvicornAccessLogFilter,
create_uvicorn_log_config,
)
from vllm.logging_utils.formatter import ColoredFormatter, NewLineFormatter from vllm.logging_utils.formatter import ColoredFormatter, NewLineFormatter
from vllm.logging_utils.lazy import lazy from vllm.logging_utils.lazy import lazy
from vllm.logging_utils.log_time import logtime from vllm.logging_utils.log_time import logtime
from vllm.logging_utils.torch_tensor import tensors_str_no_data from vllm.logging_utils.torch_tensor import tensors_str_no_data
from vllm.logging_utils.uvicorn_logging import (
UvicornAccessLogFilter,
create_uvicorn_log_config,
)
__all__ = [ __all__ = [
"NewLineFormatter", "NewLineFormatter",
+4
View File
@@ -4,6 +4,10 @@
import logging import logging
from pathlib import Path from pathlib import Path
JSON_FORMAT = (
"%(asctime)s %(levelname)s %(name)s %(processName)s %(process)d %(message)s"
)
class NewLineFormatter(logging.Formatter): class NewLineFormatter(logging.Formatter):
"""Adds logging prefix to newlines to align multi-line messages.""" """Adds logging prefix to newlines to align multi-line messages."""
@@ -1,15 +1,13 @@
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
"""Access log filter for uvicorn to exclude specific endpoints from logging. """Uvicorn logging configuration and access log filtering."""
This module provides a logging filter that can be used to suppress access logs
for specific endpoints (e.g., /health, /metrics) to reduce log noise in
production environments.
"""
import logging import logging
from typing import Literal
from urllib.parse import urlparse from urllib.parse import urlparse
from vllm.logging_utils.formatter import JSON_FORMAT
class UvicornAccessLogFilter(logging.Filter): class UvicornAccessLogFilter(logging.Filter):
"""A logging filter that excludes access logs for specified endpoint paths. """A logging filter that excludes access logs for specified endpoint paths.
@@ -70,8 +68,9 @@ class UvicornAccessLogFilter(logging.Filter):
def create_uvicorn_log_config( def create_uvicorn_log_config(
excluded_paths: list[str] | None = None, excluded_paths: list[str] | None = None,
log_level: str = "info", log_level: str = "info",
formatter: Literal["text", "json"] = "text",
) -> dict: ) -> dict:
"""Create a uvicorn logging configuration with access log filtering. """Create a uvicorn logging configuration with optional access filtering.
This function generates a logging configuration dictionary that can be This function generates a logging configuration dictionary that can be
passed to uvicorn's `log_config` parameter. It sets up the access log passed to uvicorn's `log_config` parameter. It sets up the access log
@@ -80,6 +79,7 @@ def create_uvicorn_log_config(
Args: Args:
excluded_paths: List of URL paths to exclude from access logs. excluded_paths: List of URL paths to exclude from access logs.
log_level: The log level for uvicorn loggers. log_level: The log level for uvicorn loggers.
formatter: Format used for uvicorn logs.
Returns: Returns:
A dictionary containing the logging configuration. A dictionary containing the logging configuration.
@@ -140,4 +140,13 @@ def create_uvicorn_log_config(
}, },
}, },
} }
if formatter == "json":
json_formatter = {
"class": "pythonjsonlogger.jsonlogger.JsonFormatter",
"format": JSON_FORMAT,
}
config["formatters"] = {
"default": json_formatter.copy(),
"access": json_formatter.copy(),
}
return config return config