[Benchmark] Add Responses API backend to vllm bench serve (#54628)
Signed-off-by: QHarshil <harshil_c@hotmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
This commit is contained in:
co-authored by
Claude
Chauncey
parent
0da126676e
commit
af7f9488c2
@@ -2148,7 +2148,9 @@ def get_samples(
|
||||
dataset_path=args.dataset_path, disable_shuffle=args.disable_shuffle
|
||||
)
|
||||
# For the "sonnet" dataset, formatting depends on the backend.
|
||||
if args.backend == "openai-chat":
|
||||
# Chat-style backends leave templating to the server; completions
|
||||
# backends need the prompt rendered client side.
|
||||
if args.backend in ("openai-chat", "openai-responses"):
|
||||
input_requests = sonnet_dataset.sample(
|
||||
num_requests=args.num_prompts,
|
||||
input_len=args.sonnet_input_len,
|
||||
|
||||
Reference in New Issue
Block a user