[Bugfix][Frontend] Reject stop strings on --tokens-only servers instead of silently ignoring them (#57058)

Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Shimi Bandiel
2026-09-17 22:02:00 +00:00
committed by GitHub
co-authored by Claude Cyrus Leung mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent ac78445bc9
commit e8a2a0a8bb
2 changed files with 35 additions and 0 deletions
@@ -134,6 +134,16 @@ class ServingTokens(GenerateBaseServing):
f"sampling_params.n must be at most the server's max_num_seqs "
f"({max_num_seqs}), got {sampling_params.n}."
)
if self.force_no_detokenize and sampling_params.stop:
# SamplingParams rejects stop with detokenize=False at request
# validation, but this server forces detokenize=False afterwards,
# so the combination must be rejected here or stop strings are
# silently never applied.
return self.create_error_response(
"stop strings are not supported on a --tokens-only server "
"because detokenization is disabled. Check stop strings on "
"the coordinator, or use stop_token_ids."
)
try:
msgspec.msgpack.encode(
(