Forward Anthropic vllm_xargs to sampling params (#53308)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
This commit is contained in:
@@ -19,7 +19,7 @@ This allows arguments which are not already part of `SamplingParams` to be passe
|
||||
|
||||
## Online Custom Arguments
|
||||
|
||||
The vLLM REST API allows custom arguments to be passed to the vLLM server via `vllm_xargs`. The example below integrates custom arguments into a vLLM REST API request:
|
||||
The OpenAI-compatible REST API and Anthropic-compatible `/v1/messages` endpoint allow custom arguments to be passed to the vLLM server via `vllm_xargs`. The example below integrates custom arguments into a vLLM REST API request:
|
||||
|
||||
``` bash
|
||||
curl http://localhost:8000/v1/completions \
|
||||
|
||||
Reference in New Issue
Block a user