From 887e900b773172ddec085149c9bdc702640b1fa4 Mon Sep 17 00:00:00 2001 From: "rongfu.leng" Date: Fri, 26 Dec 2025 23:48:15 +0800 Subject: [PATCH] [Docs] Add profiler user docs for http request (#31370) Signed-off-by: rongfu.leng --- docs/contributing/profiling.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/contributing/profiling.md b/docs/contributing/profiling.md index cbce14ce99..ce10adaf0c 100644 --- a/docs/contributing/profiling.md +++ b/docs/contributing/profiling.md @@ -54,6 +54,29 @@ vllm bench serve \ --num-prompts 2 ``` +Or use http request: + +```shell +# We need first call /start_profile api to start profile. +$ curl -X POST http://localhost:8000/start_profile + +# Call model generate. +curl -X POST http://localhost:8000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "meta-llama/Llama-3.1-8B-Instruct", + "messages": [ + { + "role": "user", + "content": "San Francisco is a" + } + ] + }' + +# After need call /stop_profile api to stop profile. +$ curl -X POST http://localhost:8000/stop_profile +``` + ## Profile with NVIDIA Nsight Systems Nsight systems is an advanced tool that exposes more profiling details, such as register and shared memory usage, annotated code regions and low-level CUDA APIs and events.