diff --git a/docs/design/arch_overview.md b/docs/design/arch_overview.md index e419104bae..c19ea49d96 100644 --- a/docs/design/arch_overview.md +++ b/docs/design/arch_overview.md @@ -178,7 +178,7 @@ incoming requests. The `AsyncLLMEngine` is designed for online serving, where it can handle multiple concurrent requests and stream outputs to clients. The OpenAI-compatible API server uses the `AsyncLLMEngine`. There is also a demo -API server that serves as a simpler example in [vllm/entrypoints/api_server.py](../../vllm/entrypoints/api_server.py). +API server that serves as a simpler example in [examples/applications/api_server/server.py](../../examples/applications/api_server/server.py). The code for `AsyncLLMEngine` can be found in [vllm/engine/async_llm_engine.py](../../vllm/engine/async_llm_engine.py). diff --git a/docs/examples/README.md b/docs/examples/README.md index 9d6126a65c..5569db9119 100644 --- a/docs/examples/README.md +++ b/docs/examples/README.md @@ -9,7 +9,7 @@ vLLM's examples are organized into the following categories: - **[`features/`](../../examples/features)** – Demonstrations of individual vLLM features: automatic prefix caching, speculative decoding, LoRA, structured outputs, prompt embedding, pause/resume, batch invariance, KV events, data parallelism, and more. - **[`reasoning/`](../../examples/reasoning)** – Examples for reasoning with vLLM. - **[`tool_calling/`](../../examples/tool_calling)** – Examples for function/tool calling with vLLM. -- **[`applications/`](../../examples/applications)** – Application examples such as chatbots and RAG (Retrieval-Augmented Generation). +- **[`applications/`](../../examples/applications)** – Application examples such as simpler api server, chatbots and RAG (Retrieval-Augmented Generation). - **[`rl/`](../../examples/rl)** – Reinforcement learning examples. - **[`deployment/`](../../examples/deployment)** – Examples for deploying vLLM in production. - **[`ray_serving/`](../../examples/ray_serving)** – Scalable serving using Ray. diff --git a/examples/applications/chatbot/api_client.py b/examples/applications/api_server/client.py similarity index 94% rename from examples/applications/chatbot/api_client.py rename to examples/applications/api_server/client.py index 84854911ba..89207d854c 100644 --- a/examples/applications/chatbot/api_client.py +++ b/examples/applications/api_server/client.py @@ -1,8 +1,8 @@ # SPDX-License-Identifier: Apache-2.0 # SPDX-FileCopyrightText: Copyright contributors to the vLLM project -"""Example Python client for `vllm.entrypoints.api_server` +"""Example Python client for `examples/applications/api_server/server.py` Start the demo server: - python -m vllm.entrypoints.api_server --model + python examples/applications/api_server/server.py --model NOTE: The API server is used only for demonstration and simple performance benchmarks. It is not intended for production use. diff --git a/vllm/entrypoints/api_server.py b/examples/applications/api_server/server.py similarity index 99% rename from vllm/entrypoints/api_server.py rename to examples/applications/api_server/server.py index f950b52d88..adac413321 100644 --- a/vllm/entrypoints/api_server.py +++ b/examples/applications/api_server/server.py @@ -31,7 +31,7 @@ from vllm.utils.argparse_utils import FlexibleArgumentParser from vllm.utils.system_utils import set_ulimit from vllm.version import __version__ as VLLM_VERSION -logger = init_logger("vllm.entrypoints.api_server") +logger = init_logger("api_server") app = FastAPI() engine = None diff --git a/examples/applications/chatbot/gradio_webserver.py b/examples/applications/chatbot/gradio_webserver.py index f75636409c..005bb7c68c 100644 --- a/examples/applications/chatbot/gradio_webserver.py +++ b/examples/applications/chatbot/gradio_webserver.py @@ -2,7 +2,7 @@ # SPDX-FileCopyrightText: Copyright contributors to the vLLM project """Example for starting a Gradio Webserver Start vLLM API server: - python -m vllm.entrypoints.api_server \ + python examples/applications/api_server/server.py \ --model meta-llama/Llama-2-7b-chat-hf Start Webserver: