[Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels (#58765)
Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: Misha Goin <mgoin64@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
be0823357f
commit
29e4d2d42e
+16
-1
@@ -9,7 +9,7 @@ vllm --help
|
||||
Available Commands:
|
||||
|
||||
```bash
|
||||
vllm {chat,complete,serve,launch,bench,collect-env,run-batch,preload}
|
||||
vllm {chat,complete,serve,launch,bench,collect-env,download-kernels,run-batch,preload}
|
||||
```
|
||||
|
||||
## serve
|
||||
@@ -188,6 +188,21 @@ Start collecting environment information.
|
||||
vllm collect-env
|
||||
```
|
||||
|
||||
## download-kernels
|
||||
|
||||
Install the precompiled kernels that vLLM's dependencies would otherwise download
|
||||
or compile at startup. Run it after installing or upgrading vLLM. It currently
|
||||
installs FlashInfer's kernels and does nothing when FlashInfer is not installed.
|
||||
|
||||
```bash
|
||||
vllm download-kernels
|
||||
|
||||
# Print the install commands without running them
|
||||
vllm download-kernels --dry-run
|
||||
```
|
||||
|
||||
See [vllm download-kernels](./download-kernels.md) for the full reference of all available arguments.
|
||||
|
||||
## run-batch
|
||||
|
||||
Run batch prompts and write results to file.
|
||||
|
||||
Reference in New Issue
Block a user