[Bugfix][Parser] Scope reasoning-end detection to the current turn via turn-boundary tokens (#54089)

Signed-off-by: Mikhail Podvitskii <podvitskiymichael@gmail.com>
This commit is contained in:
Mikhail Podvitskii
2026-08-27 16:57:32 -04:00
committed by GitHub
parent 7d5769b2d9
commit 3a9bfc209f
5 changed files with 147 additions and 4 deletions
@@ -84,6 +84,10 @@ class ParserEngineConfig:
# Special tokens exempt from auto-drop but not state-machine terminals.
preserve_tokens: frozenset[str] = field(default_factory=frozenset)
# Special tokens delimiting conversation turns in the prompt only
# considers reasoning markers after the last boundary token.
turn_boundary_tokens: frozenset[str] = field(default_factory=frozenset)
# Prevents trailing-whitespace accumulation across multi-turn conversations.
strip_trailing_reasoning_whitespace: bool = True