[Bugfix] Don't let a structured-output request sample from an unmasked row (#54442)

Signed-off-by: ArcheyChen <45146080+ArcheyChen@users.noreply.github.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: GPT-5.6 <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
This commit is contained in:
Ausar
2026-10-01 02:11:44 +00:00
committed by GitHub
co-authored by GPT-5.6 Claude Opus 5 Wei Zhao Wentao Ye
parent 31ec1c3aa1
commit 34c458d27f
8 changed files with 142 additions and 3 deletions
+7
View File
@@ -339,9 +339,16 @@ def _shard_grammar_output(
cursor += num_req_logits
if not local_ids:
return None
# num_acceptable_drafts is ordered with structured_output_request_ids.
num_acceptable = grammar_output.num_acceptable_drafts
if num_acceptable is not None:
owned = set(local_ids)
ids = grammar_output.structured_output_request_ids
num_acceptable = [n for i, n in zip(ids, num_acceptable) if i in owned]
return GrammarOutput(
structured_output_request_ids=local_ids,
grammar_bitmask=grammar_output.grammar_bitmask[keep_indices],
num_acceptable_drafts=num_acceptable,
)