iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,991 Commits 3 Branches 0 Tags
c19c263ac8ea3fd6e1d2d689f0e4e725855d89cf
Commit Graph
199 Commits
Author SHA1 Message Date
Misha GoinandClaude Opus 5.5 29e4d2d42e [Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels (#58765) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 21:22:35 -07:00
Tiezhen WANGandKevin H. Luu 295c062db8 Remove sudo from dockerfile command (#60488) 
Signed-off-by: Tiezhen WANG <38108242+xianbaoqian@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-08 19:24:56 -07:00
Artem Perevedentsev 4ac5a36a8e [CI/Build][NVIDIA] Build NIXL with NIXL EP from source in Rubin image (#60090) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-10-08 02:44:17 +04:00
Shang WangClaude Opus 5.5Tyler Michael SmithShengqi Chen
 		8e443871f0 [CI/Build][NVIDIA] Build Rubin images on the public nvidia/cuda base image (#59288) 
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-10-03 00:12:33 +00:00
Nils MattesonSimon Momergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		76d3929916 [Docker] Compile Python bytecode at image build time (#55422) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 02:20:17 +00:00
Wei Zhao 678baf5372 [Dependency] Upgrade FlashInfer to 0.7.0.post1 (#59323) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-09-30 02:34:36 +00:00
Kevin H. LuuandClaude Opus 5.5 aedaba8664 [CI/Build] Skip the snapshot runtime on CUDA 12.x images (#59118) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 20:17:59 -07:00
Anton Peganovmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>OpenAI Codex
 		73859fec58 [Frontend] Switch Python Harmony dependency to oss-harmony (#55128) 
Signed-off-by: Anton Peganov <apeganov@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-28 00:43:56 +04:00
Misha Goin 58aa2f1ef1 [Refactor] Move auxiliary files out of the repository root (#58572) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-24 12:38:56 -07:00
Wei ZhaoCodexYongye ZhuKevin H. Luu
 		44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-09-23 22:45:23 -07:00
Jee Jee Li d90f0eade5 [Build] Fix CUDA 12 KV connector dependency selection (#57945) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-09-22 06:13:07 -07:00
Kevin H. LuuandClaude Fable 5.1 64a48b19b4 [CI] Emit a kernel symbol map from the csrc build (opt-in, for test selection) (#58097) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-22 01:37:34 -07:00
AlecCodexBugen Zhao
 		f07e227ee9 [Docker] Expose bundled vllm-rs on PATH (#57606) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Alec <35311602+alec-flowers@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-21 20:03:03 -07:00
Nils MattesonTaneem IbrahimelehayymAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		13058faa49 [Frontend] Add reusable TP1 initialized-engine snapshots (#51360) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 23:18:49 +00:00
Tyler Michael SmithOpenAI CodexOpenAI CodexMisha Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		4f9a41ac94 [CI/Build][Hardware][NVIDIA] Add Rubin CUDA 13.4 nightly images (#55953) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-17 08:05:31 -07:00
Shang WangandCodex 031f5810c1 [Build][NVIDIA] Update public Rubin dependencies and MSA compatibility (#56545) 
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-15 16:29:13 -07:00
Yongye ZhuandClaude Opus 5 8263ea12bd [Build] Move DeepGEMM pin to the vLLM fork and update the Mega MoE call convention (#56876) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-15 00:57:18 -07:00
Wei ZhaoandOpenAI Codex 8c87c333b8 [Perf] Fix TRTLLM ragged prefill perf regression (#55499) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-09 20:38:20 +00:00
meena-at-workShang WangTyler Michael Smith
 		385dce36bc [CI/Build][Hardware][NVIDIA] Add public CUDA 13.4 Rubin build path (#54640) 
Signed-off-by: Meenakshi Venkataraman <meenakshiv@nvidia.com>
Signed-off-by: meena-at-work <80416898+meena-at-work@users.noreply.github.com>
Signed-off-by: Shang Wang <shangw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Shang Wang <shangw@nvidia.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-09-09 04:53:12 +00:00
Wei ZhaoJee Jee LiYongye ZhuJiangyun Zhu
 		7ab2923489 [Flashinfer] Upgrade Flashinfer version to 0.6.18 (#54313) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-08-30 16:01:05 -07:00
Shang Wangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 9650dc73e2 [CI/Build][Hardware][NVIDIA] Add opt-in Rubin Docker builds (#53443) 
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-27 16:06:44 +00:00
Misha GoinandOpenAI Codex c9331d8064 [CI] Preserve Rust Docker cache across commits (#53290) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-25 11:13:36 -07:00
Kevin H. LuuandOpenAI Codex e25c586b90 [CI/Build] Pin Cython below 3.3 for arm64 tilelang sdist (#53358) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-23 01:09:27 -07:00
Bugen ZhaoOpenAI CodexNick Hill
 		83c5d59209 [Rust Frontend] Replace external protoc with pure Rust lib protox (#52892) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 18:43:51 -07:00
Andrey Talman 54ba80d961 [CI][Docker] Pin manylinux2_28-builder:cuda13.0 to the release/2.13 image (#52994) 
Signed-off-by: Andrey Talman <atalman@fb.com>
 2026-08-20 15:19:40 -07:00
Wei Zhao 689be2bcd3 Upgrade Flashinfer version to 0.6.17 (#52681) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-08-18 23:04:36 +08:00
Bugen ZhaoNick HillAndreas Karatzas
 		c89d692cb5 [Build] Propagate vLLM version to Rust binaries (#52593) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-17 23:37:37 -07:00
Nick HillandClaude Opus 4.8 615d4cfade [Core] Check for GPU<->CPU syncs during CI (#43107) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-14 17:42:05 -07:00
ovidiusm 5658391af7 Remove NIXL reinstall step (#51882) 
Signed-off-by: Ovidiu Mara <ovidium@nvidia.com>
 2026-08-12 22:44:05 -07:00
Michael GoinOpenAI CodexOpenAI Codex
 		21c667aa64 [Docker] Cache test dependencies before vLLM install (#51184) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-10 10:34:59 -07:00
Wei ZhaojiahancKevin H. Luu
 		f18e10a7e1 Bump Flashinfer version to 0.6.16.post3 (#50892) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-08-09 00:54:07 -07:00
Isotr0py 073c510c91 [Migration] Migrate bitsandbytes support to OOT plugin (#43529) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-09 10:27:31 +08:00
Tyler Michael SmithOpenAI CodexMichael Goin
 		cb1a52aee3 [Build] Upgrade runtime image to Ubuntu 24.04, pick up rdma-core > 44 (#51058) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-08-08 18:17:50 -07:00
Zhewen LiZhewen LiClaude Opus 5
 		14e57ad47d [Docker][KVConnector] Install mooncake from official wheels instead of a custom build (#51067) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-05 11:58:44 -07:00
Michael GoinandShengqi Chen e8586b2878 [Build] Remove Ubuntu build-stage option from the CUDA dockerfile (#51060) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-08-05 10:50:58 -07:00
Kevin H. LuuandMichael Goin 5f8f72866c [Build] Fix CUDA release wheel builds (#50243) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-07-30 11:48:09 -07:00
Tyler Michael SmithandOpenAI Codex 1ad5182ba9 [CI/Build] Limit wheel size check to CUDA 13 (#50357) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-29 20:10:42 -07:00
Tyler Michael SmithClaudeIlya MarkovOpenAI CodexCodex
 		5d8e90a966 [WideEP] Update NCCL to 2.30.7 to enable DeepEPv2 in the vllm/vllm-openai image (#45321) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Tyler Michael Smith <tyler@vllm.ai>
Signed-off-by: Tyler Michael Smith <tyler@tylermsmith.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ilya Markov <ilmarkov@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-24 13:00:02 -07:00
Andrey TalmanAndrey TalmanClaude
 		75ccdf3145 [Core] Update PyTorch to 2.13.0, torchvision to 0.28.0, triton to 3.7.1 (#48155) 
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-23 11:09:36 -07:00
Wei ZhaoWei ZhaoCyrus Leung
 		61a09532f2 Bump Flashinfer version to 0.6.15 (#48914) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <weizha@oci-aga-slurm-1-vscode-02.cm.cluster>
Co-authored-by: Wei Zhao <weizha@oci-aga-slurm-1-vscode-02.cm.cluster>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-07-22 13:32:05 -07:00
Artem PerevedentsevandMatthew Bonanni 7cd1d57b74 [CI/Build][Docker] Bump nvidia-cutlass-dsl to 4.6.0 and drop packaging workarounds (#47442) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-16 13:51:41 +00:00
Ameen PatelAmeenPJee Jee LiClaudePavani Majety
 		ba47bb5be1 Bump flashinfer version to 0.6.14 (#47669) 
Signed-off-by: AmeenP <ameenp360@gmail.com>
Signed-off-by: AmeenP <ameen@primeintellect.ai>
Co-authored-by: AmeenP <ameen@primeintellect.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-07-15 21:00:40 -07:00
Bugen Zhao b790c84cde [CI] Enable sccache for Rust build under CUDA/ROCm (#45246) 2026-07-02 23:45:41 -07:00
Nick Hill d85601c20f [CI] Pin modelscope version to fix test breakage (#47465) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-02 19:33:08 -07:00
Wei ZhaoandJee Jee Li 4708292d48 Bump flashinfer version to 0.6.13 (#46683) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-06-29 09:30:57 -07:00
weizhoublue 2e058851d3 fix(docker): eliminate race conditions in shared buildkit cache mounts (#44984) 2026-06-26 19:43:17 -07:00
Michael GoinandClaude 7c2e08451a [Docker] Remove redundant flashinfer download-cubin step (#46517) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-23 12:16:51 -07:00
Shengqi ChenXin LiShawRongChange72
 		78e7293bb1 [Build] Fix CUDA arch build coverage gaps (#45277) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Xin Li <xinli-sw@users.noreply.github.com>
Co-authored-by: ShawRong <ShawRong@users.noreply.github.com>
Co-authored-by: Change72 <Change72@users.noreply.github.com>
 2026-06-13 22:09:20 -07:00
achyuthan.sandShengqi Chen 54bbf51668 [Bugfix] nightly Docker images crash with ImportError: AnthropicOutputConfig since May 28 (#44795) 
Signed-off-by: achyuthan.s <113010327+Achyuthan-S@users.noreply.github.com>
Signed-off-by: Achyuthan S <achyuthan.sivasankar@gmail.com>
Signed-off-by: Achyuthan Sivasankar <achyuthan.sivasankar@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-06-13 21:45:29 -07:00
Richard BarnesandShengqi Chen 05d9848267 [Build] Upgrade CUDA Dockerfiles from GCC 10 to GCC 12 for C++20 compatibility (#44923) 
Signed-off-by: Richard Barnes <rbarnes@meta.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-06-11 12:26:52 +00:00