[Doc] Document /inference/v1/generate request/response fields and DP rank header (#62305)

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
aoshen02
2026-10-18 15:26:36 +00:00
committed by GitHub
co-authored by Claude Opus 5.5
parent e80b6c642f
commit fe8f2140ee
3 changed files with 49 additions and 4 deletions
+3 -2
View File
@@ -139,8 +139,9 @@ Each position then carries the sampled token's logprob and the top-128
drop them. The finite entries are the sampling mask when it has at most `k`
tokens.
- The sampled token is always the first entry and may repeat inside the top-k.
- `/inference/v1/generate` encodes IDs as `token_id:<id>`; the OpenAI
endpoints need `return_tokens_as_token_ids`.
- The `token_id:<id>` strings come from the OpenAI endpoints with
`return_tokens_as_token_ids`; `/inference/v1/generate` returns an integer
`token_id` instead.
## Limitations