[Frontend][RFC] Rust front-end integration (#40848)
Signed-off-by: Nick Hill <nickhill123@gmail.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>
This commit is contained in:
@@ -43,6 +43,13 @@ If you are only developing vLLM's Python code, install vLLM using:
|
||||
VLLM_USE_PRECOMPILED=1 uv pip install -e .
|
||||
```
|
||||
|
||||
To rebuild only the Rust frontend binary:
|
||||
|
||||
```bash
|
||||
./build_rust.sh # release build
|
||||
./build_rust.sh --debug # faster build for development
|
||||
```
|
||||
|
||||
If you are developing vLLM's Python and CUDA/C++ code, install Pytorch first:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -136,10 +136,10 @@ When installing vLLM with `VLLM_USE_PRECOMPILED=1`, the `setup.py` script:
|
||||
3. **Selects compatible wheel** based on:
|
||||
- Package name (`vllm`)
|
||||
- Platform tag (architecture match)
|
||||
4. **Downloads and extracts** precompiled binaries from the wheel:
|
||||
- C++ extension modules (`.so` files)
|
||||
- Flash Attention Python modules
|
||||
- Triton kernel Python files
|
||||
4. **Downloads and extracts** precompiled artifacts from the wheel:
|
||||
- Native extension modules (`.so` files)
|
||||
- The `vllm-rs` Rust frontend binary
|
||||
- Flash Attention Python modules and Triton/FlashMLA Python files
|
||||
5. **Patches package_data** to include extracted files in the installation
|
||||
|
||||
!!! note "What is the base commit?"
|
||||
|
||||
@@ -101,12 +101,22 @@ This command will do the following:
|
||||
1. Look for the current branch in your vLLM clone.
|
||||
1. Identify the corresponding base commit in the main branch.
|
||||
1. Download the pre-built wheel of the base commit.
|
||||
1. Use its compiled libraries in the installation.
|
||||
1. Use its compiled libraries and `vllm-rs` binary in the installation.
|
||||
|
||||
!!! note
|
||||
1. If you change C++ or kernel code, you cannot use Python-only build; otherwise you will see an import error about library not found or undefined symbol.
|
||||
2. If you rebase your dev branch, it is recommended to uninstall vllm and re-run the above command to make sure your libraries are up to date.
|
||||
|
||||
!!! tip "Rebuilding the Rust frontend"
|
||||
If you need to recompile the `vllm-rs` Rust frontend binary, you can rebuild and install it without re-running the full pip install:
|
||||
|
||||
```bash
|
||||
./build_rust.sh # release build
|
||||
./build_rust.sh --debug # faster build for development
|
||||
```
|
||||
|
||||
This will install the required Rust toolchain if needed, build the binary, and place it in `vllm/vllm-rs`.
|
||||
|
||||
In case you see an error about wheel not found when running the above command, it might be because the commit you based on in the `main` branch was just merged and its precompiled wheel is not available yet. You can wait around an hour and retry, or set `VLLM_PRECOMPILED_WHEEL_COMMIT=nightly` to automatically select the most recent already-built commit on `main`.
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user