[Render][Speculator] Add return_loss_mask to render endpoint for training data generation (#46846)

Signed-off-by: Ranran Haoran Zhang <ranzhang@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
This commit is contained in:
Ranran
2026-06-28 00:07:33 -07:00
committed by GitHub
co-authored by Benjamin Chislett
parent a2a92cbbaa
commit 09841ae705
7 changed files with 288 additions and 20 deletions
+10
View File
@@ -42,6 +42,11 @@ class TokensInput(_InputOptions):
"""Char-level (start, end) offsets per token, propagated from the
renderer's TokensPrompt when offsets were computed."""
assistant_tokens_mask: NotRequired[list[int] | None]
"""Per-token 0/1 mask marking assistant-generated tokens.
Populated when ``return_assistant_tokens_mask=True`` is set on the
render request and the chat template supports ``{% generation %}``."""
def tokens_input(
prompt_token_ids: list[int],
@@ -151,6 +156,11 @@ class MultiModalInput(_InputOptions):
`prompt_token_ids`.
"""
assistant_tokens_mask: NotRequired[list[int] | None]
"""Per-token 0/1 mask marking assistant-generated tokens.
Populated when ``return_assistant_tokens_mask=True`` is set on the
render request and the chat template supports ``{% generation %}``."""
def mm_input(
prompt_token_ids: list[int],