diff --git a/vllm/parser/AGENTS.md b/vllm/parser/AGENTS.md new file mode 100644 index 0000000000..d9e6b79707 --- /dev/null +++ b/vllm/parser/AGENTS.md @@ -0,0 +1,41 @@ +# vllm/parser + +Model-output parsing for the serving layer. `Parser` (`abstract_parser.py`) +unifies reasoning extraction and tool-call extraction behind one object; the +serving layer calls `parse` / `parse_delta` and `is_reasoning_end` on it. + +## Layout + +- `abstract_parser.py` — `Parser`, and `DelegatingParser` for wrapping a legacy + `ReasoningParser` + `ToolParser` pair. +- `parser_manager.py` — resolves parser names to classes. +- `engine/` — the declarative parser engine. A `ParserEngineConfig` + (`parser_engine_config.py`) is a set of terminals, transitions and content + events; `parser_engine.py` and `streaming_parser_engine.py` drive it for the + non-streaming and streaming paths; `token_id_scanner.py` pre-lexes + special-token ids and `incremental_lexer.py` lexes text; `adapters.py` + exposes an engine through the legacy reasoning/tool parser interfaces. +- `.py` — one file per model family. Most subclass `ParserEngine` + (`deepseek_v4.py`, `qwen3.py`, ...); a few are `DelegatingParser` wrappers + over legacy parsers (`harmony.py`, `kimi_k3.py`). + +## Guidelines + +- A new parser either subclasses `ParserEngine` with a declarative config, or + subclasses `Parser` and implements `parse` / `parse_delta` directly. Use the + engine when the format is expressible as terminals and transitions, and + subclass an existing config when it is a variant of one (`seed_oss.py`, + `ling3.py`). `DelegatingParser` exists for backward compatibility with the + separate `ReasoningParser` / `ToolParser` classes; do not use it for new + formats. +- Keep a model-specific fix inside that model's `.py` where possible. + Change `engine/` or `abstract_parser.py` only when the behavior is wrong for + every parser, and say so in the PR. +- Text emitted in a state with no `content_events` entry is discarded. Make + that deliberate when adding a state. +- Special tokens reach the engine as token ids, not text. A text terminal that + spans a special token never matches in streaming; list it in + `token_id_terminals`. +- Comment only where the code alone would leave a reader with a question — + a non-obvious transition, a deliberate swallow, a model quirk being worked + around. Keep it minimal and concise. diff --git a/vllm/parser/CLAUDE.md b/vllm/parser/CLAUDE.md new file mode 100644 index 0000000000..43c994c2d3 --- /dev/null +++ b/vllm/parser/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/vllm/reasoning/AGENTS.md b/vllm/reasoning/AGENTS.md new file mode 100644 index 0000000000..1f7c8b3557 --- /dev/null +++ b/vllm/reasoning/AGENTS.md @@ -0,0 +1,54 @@ +# vllm/reasoning + +Every `--reasoning-parser` name resolves here. The parser has two consumers: +`ParserManager.get_parser` composes it with the tool parser into the unified +`Parser` the serving layer uses, and the structured-output manager +(`vllm/v1/structured_output/__init__.py`) instantiates it directly and calls +`is_reasoning_end` / `is_reasoning_end_streaming` every decode step to decide +when the grammar bitmask starts applying. The `ReasoningParser` interface +itself is legacy: it is kept so existing in-tree parsers and out-of-tree +plugins keep working, not as the base for new ones. + +## Layout + +- `abs_reasoning_parsers.py` — `ReasoningParser` (`extract_reasoning`, + `extract_reasoning_streaming`, `is_reasoning_end`, + `is_reasoning_end_streaming`, `extract_content_ids`, `adjust_request`) and + `ReasoningParserManager`. +- `__init__.py` — `_REASONING_PARSERS_TO_REGISTER`, the name → (file, class) + table that lazily registers every built-in parser. +- `basic_parsers.py` — `BaseThinkingReasoningParser`, the base for parsers + that only need a start/end token pair. +- `_reasoning_parser.py` — one file per family. Either a subclass of a + `ParserReasoningAdapter` generated from a `vllm/parser/.py` engine + grammar (`mistral_reasoning_parser.py`, `kimi_k2_reasoning_parser.py`; the + `_engine_` suffix is not applied consistently), or a self-contained + `ReasoningParser` subclass. + +## Guidelines + +- Do not change the `ReasoningParser` interface in `abs_reasoning_parsers.py`. + Out-of-tree plugins loaded through + `ReasoningParserManager.import_reasoning_parser` subclass it. New + capabilities go on `Parser` in `vllm/parser/`; if a legacy parser needs the + same capability, `DelegatingParser` supplies the default. +- For a new model, implement a unified `Parser` in `vllm/parser/` first, then + expose it here as a thin adapter subclass plus an entry in + `_REASONING_PARSERS_TO_REGISTER`. +- The `_REASONING_PARSERS_TO_REGISTER` key is what users pass on the command + line; treat renames as breaking. +- Keep adapters thin: registration, `adjust_request` tweaks, and class + attributes. Parsing logic belongs in `vllm/parser/`. +- Keep a model-specific fix inside that model's `_reasoning_parser.py`, + or its grammar in `vllm/parser/.py`, where possible. Touch + `abs_reasoning_parsers.py` or `basic_parsers.py` only for behavior every + parser needs, and say so in the PR. +- `is_reasoning_end_streaming` runs on the engine core for every decode step + of every structured-output request. Work on `delta_ids`; re-scanning the + full sequence makes the step cost grow with sequence length. +- The structured-output gate and the frontend's extraction must agree on + where reasoning ends, whether the turn continues with content or a tool + call. Test the end-detection methods with the same inputs as extraction. +- Comment only where the code alone would leave a reader with a question — + a request tweak in `adjust_request`, a marker handled unusually, a model + quirk being worked around. Keep it minimal and concise. diff --git a/vllm/reasoning/CLAUDE.md b/vllm/reasoning/CLAUDE.md new file mode 100644 index 0000000000..43c994c2d3 --- /dev/null +++ b/vllm/reasoning/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/vllm/tool_parsers/AGENTS.md b/vllm/tool_parsers/AGENTS.md new file mode 100644 index 0000000000..2bd487997c --- /dev/null +++ b/vllm/tool_parsers/AGENTS.md @@ -0,0 +1,45 @@ +# vllm/tool_parsers + +Every `--tool-call-parser` name resolves here. `ParserManager.get_parser` +composes the tool parser chosen by name with the reasoning parser into the +unified `Parser` the serving layer uses. The `ToolParser` interface itself is +legacy: it is kept so existing in-tree parsers and out-of-tree plugins keep +working, not as the base for new ones. + +## Layout + +- `abstract_tool_parser.py` — `ToolParser` (`extract_tool_calls`, + `extract_tool_calls_streaming`, `adjust_request`) and `ToolParserManager`. +- `__init__.py` — `_TOOL_PARSERS_TO_REGISTER`, the name → (file, class) table + that lazily registers every built-in parser. +- `structural_tag_registry.py` — structural-tag builders used to constrain + tool-call output for `tool_choice="required"` / named tools. +- `_tool_parser.py` — one file per family. Either a subclass of a + `ParserToolAdapter` generated from a `vllm/parser/.py` engine grammar + (`mistral_tool_parser.py`, `kimi_k2_tool_parser.py`; the `_engine_` suffix + is not applied consistently), or a self-contained `ToolParser` subclass. + +## Guidelines + +- Do not change the `ToolParser` interface in `abstract_tool_parser.py`. + Out-of-tree plugins loaded through `ToolParserManager.import_tool_parser` + subclass it. New capabilities go on `Parser` in `vllm/parser/`; if a legacy + parser needs the same capability, `DelegatingParser` supplies the default. +- For a new model, implement a unified `Parser` in `vllm/parser/` first, then + expose it here as a thin adapter subclass plus an entry in + `_TOOL_PARSERS_TO_REGISTER`. +- The `_TOOL_PARSERS_TO_REGISTER` key is what users pass on the command line; + treat renames as breaking. +- Keep adapters thin: registration, `adjust_request` tweaks, and class + attributes. Parsing logic belongs in `vllm/parser/`. +- Keep a model-specific fix inside that model's `_tool_parser.py`, or + its grammar in `vllm/parser/.py`, where possible. Touch + `abstract_tool_parser.py` or `structural_tag_registry.py` only for behavior + every parser needs, and say so in the PR. +- Set `structural_tag_model` only to a key in `SUPPORTED_STRUCTURAL_TAG_MODELS` + (an xgrammar builtin, or one registered with `register_vllm_structural_tag`). + It only takes effect under `VLLM_ENFORCE_STRICT_TOOL_CALLING`, where it also + disables the generic required/named handling for that parser. +- Comment only where the code alone would leave a reader with a question — + a request tweak in `adjust_request`, a marker handled unusually, a model + quirk being worked around. Keep it minimal and concise. diff --git a/vllm/tool_parsers/CLAUDE.md b/vllm/tool_parsers/CLAUDE.md new file mode 100644 index 0000000000..43c994c2d3 --- /dev/null +++ b/vllm/tool_parsers/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md