iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
20,626 Commits 3 Branches 0 Tags
65ce85fcdcd79770eeb8cd5eeb4e4ba06120bf6d
Commit Graph
257 Commits
Author SHA1 Message Date
Taneem Ibrahim fd5d3aea94 [Mypy] Fix typing for J models (#54130) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-29 08:50:15 -04:00
Taneem IbrahimandWentao Ye 21fa2c5a2a [Mypy] Fix mypy typing for model interfaces and H/I models (#54079) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-08-28 16:49:19 -04:00
Shang Wangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 9650dc73e2 [CI/Build][Hardware][NVIDIA] Add opt-in Rubin Docker builds (#53443) 
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-27 16:06:44 +00:00
Louie Tsai 79651d6085 [Tools][Recipes] Improve sweep recommendations and short-alias parsing (#53946) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-08-27 06:58:23 +00:00
Jensen ChenClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c47ca4aaeb [Mypy Fix] Mypy fix for "vllm/model_executor/models/[tT]" (#53466) 
Signed-off-by: Jensen Chen <a1043904820@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-26 14:12:11 +00:00
Wentao Ye 70021a0081 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[gG]" (#53616) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-25 09:47:16 -07:00
Louie Tsai 4af586e185 Vllm recipes tool improve (#53325) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-08-25 05:53:47 +00:00
Aaron Hao 6cd9713463 [RL] P2P RDT weight sync (#43375) 2026-08-23 09:42:22 -07:00
Wentao Ye 811d12ee26 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[eE][fF]" (#53381) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-22 15:41:27 -04:00
Bugen ZhaoOpenAI CodexNick Hill
 		83c5d59209 [Rust Frontend] Replace external protoc with pure Rust lib protox (#52892) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 18:43:51 -07:00
Turner JabbourandClaude Sonnet 5 1eab6fef01 [CI] replace shellcheck script with shellcheck-py hook (#52572) 
Signed-off-by: Turner <doubleujabbour@gmail.com>
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-08-20 10:09:59 +00:00
jcotant-inferactClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c205726108 [CI] Fix and extend PR/issue auto-labeling (#51459) 
Signed-off-by: Joe Cotant <joe@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 21:45:44 +00:00
Bugen ZhaoNick HillAndreas Karatzas
 		c89d692cb5 [Build] Propagate vLLM version to Rust binaries (#52593) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-17 23:37:37 -07:00
fxmarty-amdAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c5b7c069a4 [ROCm] Defer tilelang import through its import from vllm.tilelang_utils import tilelang and relaxed has_tilelang (#51159) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 13:12:40 -05:00
Wentao Ye 9a276d6375 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[cC][dD]" (#52003) 2026-08-13 11:33:55 +08:00
Yongye ZhuandKimi Code 025d56a11e [Build] Update DeepGEMM pin to deepseek-ai nv_dev tip (#52035) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-08-12 13:49:04 -07:00
Tyler Michael Smith 34735aceda [Bugfix] Pin DeepEP by its full commit hash (#52028) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-08-12 20:01:32 +00:00
Louie Tsai 9b3382685e [Tools] vLLM Recipes conversion : support different data types variants and strategies (#51878) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-08-12 01:42:08 +00:00
Louie Tsai 5b184f775a connects vLLM Recipes with vLLM's native config-based deployment and benchmark (#51308) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-08-11 13:53:51 +00:00
Michael GoinOpenAI CodexOpenAI Codex
 		21c667aa64 [Docker] Cache test dependencies before vLLM install (#51184) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-10 10:34:59 -07:00
Wentao Ye adc3e03517 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[aA][bB]" (#48977) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-06 19:30:15 +00:00
Kevin H. Luu a1657a0235 [Bugfix][Build] Fix DeepGEMM CUDA 12.9 FP8 header visibility (#51003) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-08-04 03:06:04 -07:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Ruinan MaandNick Hill a07fac758f [Perf] Zero-copy torch.Tensor pickling in shm_broadcast MessageQueue (#48442) 
Signed-off-by: Ruinan Ma <r7ma3088@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-28 16:28:28 -07:00
Kevin H. LuuandOpenAI Codex d2bfc6fe20 [Build] Fix DeepEP CUDA driver stub linking (#50103) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-28 12:09:19 +00:00
Tyler Michael Smith ee1d996367 [Build] Fix for DeepEP manylinux pidfd sycall usage (#49814) 2026-07-25 15:29:38 -04:00
Harry Mellor 3e74c60b9c [Docs] Use gen-files for generated docs content (#49587) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-25 14:43:21 +00:00
Harry MellorandAndreas Karatzas ca0defa343 Make bare hugging_face imports forbidden (#49726) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-25 04:18:39 -07:00
Tyler Michael SmithClaudeIlya MarkovOpenAI CodexCodex
 		5d8e90a966 [WideEP] Update NCCL to 2.30.7 to enable DeepEPv2 in the vllm/vllm-openai image (#45321) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Tyler Michael Smith <tyler@vllm.ai>
Signed-off-by: Tyler Michael Smith <tyler@tylermsmith.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ilya Markov <ilmarkov@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-24 13:00:02 -07:00
Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> bd091079cb [Attention] FlashAttention 4 SM100 FP8 kv cache support (#42569) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-20 10:53:27 -07:00
Aaron HaoandSumanth R Hegde fb1d8ccaf5 [rl] Stateful Trainer Send: New Abstractions [1/N] (#48042) 
Signed-off-by: haoaaron <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: Sumanth R Hegde <39546518+SumanthRH@users.noreply.github.com>
 2026-07-17 15:11:06 +08:00
Wentao Ye 626c90b2d5 [Refactor] Move fla to third party (#48500) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-16 19:22:36 +01:00
Michael GoinandClaude Fable 5 ecf4aa5ce2 [Bugfix] Fix FlashInfer non-causal draft attention (DFlash/DSpark) on Blackwell (#48167) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-15 12:44:01 -07:00
Matthew Bonanni 313d01f507 [CI][Bugfix] Fix FlashAttention reported MLA dimension support (#48631) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-14 21:33:02 +00:00
Wentao Ye b3cfca996c [Mypy Fix] Split mypy work (#48490) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-13 12:42:42 +00:00
Bugen Zhao 487dfb3418 [CI] Add SPDX license header to Rust/Protobuf sources (#48472) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-13 10:22:47 +01:00
Kunshang JiandHarry Mellor 07f9baf756 Revert "[Platform] Replace torch.cuda.Event with torch.Event (#47140)" (#47668) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-06 14:18:33 +01:00
Dan BlanaruandMatthew Bonanni d29125c085 Xqa decode kernels (#43232) 
Signed-off-by: Dan Blanaru <48605845+DanBlanaru@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-02 12:32:05 -07:00
Michael Goin 84b9c2762f Update DeepGEMM tag to point to latest nv-dev branch for sm120 support (#47304) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-02 18:33:44 +08:00
Kunshang JiandHarry Mellor e840f0d3f5 [Platform] Replace torch.cuda.Event with torch.Event (#47140) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 08:39:59 -07:00
Kunshang Ji fb42e5219e [Platform] Replace torch.cuda.mem_get_info with torch.accelerator.get_memory_info (#44825) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-06-30 14:39:52 +08:00
Martin Hickey a4e3cb40d0 [mypy] Enable mypy for tests directory (#47018) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-06-29 09:29:09 +00:00
Bugen Zhao 76c3c4ff63 [Rust Frontend] Introduce unified parser interface & combined parser (#46583) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-25 03:17:31 +00:00
Martin HickeyandHarry Mellor 275b43183c [MyPy] Fix mypy for vllm/benchmarks (#39896) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-23 15:22:29 +00:00
Gabriel WuClaude Opus 4.8Yongye Zhu
 		44d95069e9 Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-06-22 11:54:14 -07:00
Martin Hickey ccd49f6821 [MyPy] Fix mypy for vllm/lora (#41722) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-06-22 10:57:09 -04:00
Mohammad Miadh AngkadandMatthew Bonanni 9d4dc4ca2f [Kernel] Support GLM-5 dimensions for TRT-LLM ragged MLA prefill (#43525) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-06-16 20:49:47 +00:00
youkaichaoOpenAI CodexIsotr0pyThien TranBugen ZhaoJee Jee LiRoger WangfunctionstackxYongye ZhuJee Jee Li
 		0a1c5034f5 [Model] Add MiniMax M3 support (#45381) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-16 01:01:25 +08:00
Bugen Zhao 43914dd743 [Rust Frontend] Add Python bridge for Rust tool parsers (#44624) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-11 04:51:06 +00:00