[Doc] Add MatrixHub as a model loading source (#50492)

Signed-off-by: yiting.jiang <yiting.jiang@daocloud.io>
This commit is contained in:
Yiting Jiang
2026-08-17 02:32:23 +00:00
committed by GitHub
parent a6a2a93f9b
commit 502af5ed00
+15
View File
@@ -289,6 +289,21 @@ os.environ["http_proxy"] = "http://your.proxy.server:port"
os.environ["https_proxy"] = "http://your.proxy.server:port"
```
### MatrixHub
[MatrixHub](https://github.com/matrixhub-ai/matrixhub) is a self-hosted model registry and distribution layer that caches models from upstream hubs and serves them over a Hugging Face-compatible API inside your own network.
Since the API is Hugging Face-compatible, you only need to point `HF_ENDPOINT` at your MatrixHub instance:
```shell
export HF_ENDPOINT="http://<your-matrixhub-address>"
vllm serve Qwen/Qwen3-0.6B
```
vLLM then downloads model weights from MatrixHub over the internal network instead of the public Hugging Face Hub, which is useful for air-gapped clusters and for avoiding repeated downloads across nodes.
See the [MatrixHub guide for vLLM](https://matrixhub.ai/docs/guides/use-with-vllm/) for an end-to-end walkthrough, including Docker and Kubernetes deployment examples.
### ModelScope
To use models from [ModelScope](https://www.modelscope.cn) instead of Hugging Face Hub, set an environment variable: