[Pooling] Fix Rust pooling endpoint for benchmark (#53352)

Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
This commit is contained in:
Taneem Ibrahim
2026-08-22 15:45:09 -04:00
committed by GitHub
parent 811d12ee26
commit a014e35f38
3 changed files with 3 additions and 3 deletions
+1 -1
View File
@@ -419,7 +419,7 @@ vllm-bench \
| --------- | ------------- | ------------- |
| `openai-embeddings` | `/v1/embeddings` | Text embedding (accepts text or token IDs) |
| `openai-embeddings-chat` | `/v1/embeddings` | Chat-format embedding (supports multimodal content) |
| `vllm-pooling` | `/v1/pooling` | vLLM native pooling endpoint |
| `vllm-pooling` | `/pooling` | vLLM native pooling endpoint |
| `vllm-rerank` | `/v1/rerank` | vLLM reranking (query from prompt, documents via `--extra-body`) |
Pooling backends are non-streaming and report E2EL (end-to-end latency) only. Use `--dataset-name sharegpt`, `sonnet`, or `hf` for text-based embedding/rerank benchmarks, or `random` for token-ID-based embedding benchmarks.
+1 -1
View File
@@ -8,7 +8,7 @@
//! - `openai-embeddings`: Standard OpenAI `/v1/embeddings` with text input
//! - `openai-embeddings-chat`: OpenAI `/v1/embeddings` with chat message format (supports
//! multimodal)
//! - `vllm-pooling`: vLLM `/v1/pooling` endpoint
//! - `vllm-pooling`: vLLM `/pooling` endpoint
//! - `vllm-rerank`: vLLM `/v1/rerank` endpoint (query + documents)
use std::time::Instant;
+1 -1
View File
@@ -682,7 +682,7 @@ impl BenchServeArgs {
BackendKind::OpenaiEmbeddings | BackendKind::OpenaiEmbeddingsChat => {
"/v1/embeddings".to_string()
}
BackendKind::VllmPooling => "/v1/pooling".to_string(),
BackendKind::VllmPooling => "/pooling".to_string(),
BackendKind::VllmRerank => "/v1/rerank".to_string(),
}
}