[Doc] Document /inference/v1/generate request/response fields and DP rank header (#62305)
Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
e80b6c642f
commit
fe8f2140ee
@@ -139,8 +139,9 @@ Each position then carries the sampled token's logprob and the top-128
|
||||
drop them. The finite entries are the sampling mask when it has at most `k`
|
||||
tokens.
|
||||
- The sampled token is always the first entry and may repeat inside the top-k.
|
||||
- `/inference/v1/generate` encodes IDs as `token_id:<id>`; the OpenAI
|
||||
endpoints need `return_tokens_as_token_ids`.
|
||||
- The `token_id:<id>` strings come from the OpenAI endpoints with
|
||||
`return_tokens_as_token_ids`; `/inference/v1/generate` returns an integer
|
||||
`token_id` instead.
|
||||
|
||||
## Limitations
|
||||
|
||||
|
||||
Reference in New Issue
Block a user