[DisaggEverything] Tokens in<>out /generate endpoint (#24261)

Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Nicolò Lucchesi
2025-11-14 09:58:01 -07:00
committed by GitHub
co-authored by Harry Mellor
parent d54a18a47e
commit 6f1e7f7226
12 changed files with 822 additions and 9 deletions
@@ -58,6 +58,8 @@ from vllm.entrypoints.openai.protocol import (
ErrorResponse,
FunctionCall,
FunctionDefinition,
GenerateRequest,
GenerateResponse,
IOProcessorRequest,
PoolingResponse,
RerankRequest,
@@ -134,6 +136,7 @@ AnyRequest: TypeAlias = (
| SpeechToTextRequest
| ResponsesRequest
| IOProcessorRequest
| GenerateRequest
)
AnyResponse: TypeAlias = (
@@ -145,6 +148,7 @@ AnyResponse: TypeAlias = (
| PoolingResponse
| ClassificationResponse
| ScoreResponse
| GenerateResponse
)