iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,220 Commits 3 Branches 0 Tags
00f7f258282ad3ea400a760ba3bc4679ef4e40fe
Commit Graph
247 Commits
Author SHA1 Message Date
Turner JabbourandClaude Sonnet 5 1eab6fef01 [CI] replace shellcheck script with shellcheck-py hook (#52572) 
Signed-off-by: Turner <doubleujabbour@gmail.com>
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-08-20 10:09:59 +00:00
jcotant-inferactClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c205726108 [CI] Fix and extend PR/issue auto-labeling (#51459) 
Signed-off-by: Joe Cotant <joe@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 21:45:44 +00:00
Bugen ZhaoNick HillAndreas Karatzas
 		c89d692cb5 [Build] Propagate vLLM version to Rust binaries (#52593) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-17 23:37:37 -07:00
fxmarty-amdAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c5b7c069a4 [ROCm] Defer tilelang import through its import from vllm.tilelang_utils import tilelang and relaxed has_tilelang (#51159) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 13:12:40 -05:00
Wentao Ye 9a276d6375 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[cC][dD]" (#52003) 2026-08-13 11:33:55 +08:00
Yongye ZhuandKimi Code 025d56a11e [Build] Update DeepGEMM pin to deepseek-ai nv_dev tip (#52035) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-08-12 13:49:04 -07:00
Tyler Michael Smith 34735aceda [Bugfix] Pin DeepEP by its full commit hash (#52028) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-08-12 20:01:32 +00:00
Louie Tsai 9b3382685e [Tools] vLLM Recipes conversion : support different data types variants and strategies (#51878) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-08-12 01:42:08 +00:00
Louie Tsai 5b184f775a connects vLLM Recipes with vLLM's native config-based deployment and benchmark (#51308) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-08-11 13:53:51 +00:00
Michael GoinOpenAI CodexOpenAI Codex
 		21c667aa64 [Docker] Cache test dependencies before vLLM install (#51184) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-10 10:34:59 -07:00
Wentao Ye adc3e03517 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[aA][bB]" (#48977) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-06 19:30:15 +00:00
Kevin H. Luu a1657a0235 [Bugfix][Build] Fix DeepGEMM CUDA 12.9 FP8 header visibility (#51003) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-08-04 03:06:04 -07:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Ruinan MaandNick Hill a07fac758f [Perf] Zero-copy torch.Tensor pickling in shm_broadcast MessageQueue (#48442) 
Signed-off-by: Ruinan Ma <r7ma3088@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-28 16:28:28 -07:00
Kevin H. LuuandOpenAI Codex d2bfc6fe20 [Build] Fix DeepEP CUDA driver stub linking (#50103) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-28 12:09:19 +00:00
Tyler Michael Smith ee1d996367 [Build] Fix for DeepEP manylinux pidfd sycall usage (#49814) 2026-07-25 15:29:38 -04:00
Harry Mellor 3e74c60b9c [Docs] Use gen-files for generated docs content (#49587) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-25 14:43:21 +00:00
Harry MellorandAndreas Karatzas ca0defa343 Make bare hugging_face imports forbidden (#49726) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-25 04:18:39 -07:00
Tyler Michael SmithClaudeIlya MarkovOpenAI CodexCodex
 		5d8e90a966 [WideEP] Update NCCL to 2.30.7 to enable DeepEPv2 in the vllm/vllm-openai image (#45321) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Tyler Michael Smith <tyler@vllm.ai>
Signed-off-by: Tyler Michael Smith <tyler@tylermsmith.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ilya Markov <ilmarkov@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-24 13:00:02 -07:00
Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> bd091079cb [Attention] FlashAttention 4 SM100 FP8 kv cache support (#42569) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-20 10:53:27 -07:00
Aaron HaoandSumanth R Hegde fb1d8ccaf5 [rl] Stateful Trainer Send: New Abstractions [1/N] (#48042) 
Signed-off-by: haoaaron <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: Sumanth R Hegde <39546518+SumanthRH@users.noreply.github.com>
 2026-07-17 15:11:06 +08:00
Wentao Ye 626c90b2d5 [Refactor] Move fla to third party (#48500) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-16 19:22:36 +01:00
Michael GoinandClaude Fable 5 ecf4aa5ce2 [Bugfix] Fix FlashInfer non-causal draft attention (DFlash/DSpark) on Blackwell (#48167) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-15 12:44:01 -07:00
Matthew Bonanni 313d01f507 [CI][Bugfix] Fix FlashAttention reported MLA dimension support (#48631) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-14 21:33:02 +00:00
Wentao Ye b3cfca996c [Mypy Fix] Split mypy work (#48490) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-13 12:42:42 +00:00
Bugen Zhao 487dfb3418 [CI] Add SPDX license header to Rust/Protobuf sources (#48472) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-13 10:22:47 +01:00
Kunshang JiandHarry Mellor 07f9baf756 Revert "[Platform] Replace torch.cuda.Event with torch.Event (#47140)" (#47668) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-06 14:18:33 +01:00
Dan BlanaruandMatthew Bonanni d29125c085 Xqa decode kernels (#43232) 
Signed-off-by: Dan Blanaru <48605845+DanBlanaru@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-02 12:32:05 -07:00
Michael Goin 84b9c2762f Update DeepGEMM tag to point to latest nv-dev branch for sm120 support (#47304) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-02 18:33:44 +08:00
Kunshang JiandHarry Mellor e840f0d3f5 [Platform] Replace torch.cuda.Event with torch.Event (#47140) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 08:39:59 -07:00
Kunshang Ji fb42e5219e [Platform] Replace torch.cuda.mem_get_info with torch.accelerator.get_memory_info (#44825) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-06-30 14:39:52 +08:00
Martin Hickey a4e3cb40d0 [mypy] Enable mypy for tests directory (#47018) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-06-29 09:29:09 +00:00
Bugen Zhao 76c3c4ff63 [Rust Frontend] Introduce unified parser interface & combined parser (#46583) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-25 03:17:31 +00:00
Martin HickeyandHarry Mellor 275b43183c [MyPy] Fix mypy for vllm/benchmarks (#39896) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-23 15:22:29 +00:00
Gabriel WuClaude Opus 4.8Yongye Zhu
 		44d95069e9 Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-06-22 11:54:14 -07:00
Martin Hickey ccd49f6821 [MyPy] Fix mypy for vllm/lora (#41722) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-06-22 10:57:09 -04:00
Mohammad Miadh AngkadandMatthew Bonanni 9d4dc4ca2f [Kernel] Support GLM-5 dimensions for TRT-LLM ragged MLA prefill (#43525) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-06-16 20:49:47 +00:00
youkaichaoOpenAI CodexIsotr0pyThien TranBugen ZhaoJee Jee LiRoger WangfunctionstackxYongye ZhuJee Jee Li
 		0a1c5034f5 [Model] Add MiniMax M3 support (#45381) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-16 01:01:25 +08:00
Bugen Zhao 43914dd743 [Rust Frontend] Add Python bridge for Rust tool parsers (#44624) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-11 04:51:06 +00:00
Bugen Zhao fdfb2566c0 [Rust Frontend] [CI] Unify Rust artifact builds with setuptools-rust (#44981) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-10 17:48:34 +08:00
Tyler Michael SmithandClaude Opus 4.6 e2f993dc41 [WideEP] Integrate DeepEP v2 (#41183) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-06-08 18:07:29 -07:00
Harry Mellor ef0df7dbd6 [CI] Bump mypy version 1.19.1 -> 1.20.2 (#44647) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-05 14:56:27 +00:00
Tianyu ZhangNick HillOpenAI Codex
 		7fe7800fa4 [BUG] Fix FP64 Gumbel precision coverage (#43150) 
Signed-off-by: tianyu-z <zhangtianyupro@gmail.com>
Signed-off-by: Tianyu Zhang <53099276+tianyu-z@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-06-05 19:04:14 +08:00
Yongye Zhu b5235fca2e [DSv4] Adding TRTLLM gen attention kernel (#43827) 2026-06-04 07:35:09 -07:00
Andreas Karatzas 22c2e87555 [CI] Reverted gitignore changes (#44497) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-04 00:37:44 -07:00
Andreas Karatzas 5e2af28838 [CI] Resolve release V2 docker build after ROCm CI wheels change (#44463) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-03 21:35:40 -07:00
Andreas Karatzas 87954eb50e [ROCm][CI] Optimize ROCm Docker build: registry cache, DeepEP, and ci-bake script (#36949) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-02 23:43:07 -07:00
Mohammad Miadh Angkad 158289e0fc [Docs] Fix MLA prefill backend default docs (#43697) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-05-27 10:13:22 +00:00
Bugen ZhaoNick HillEric CurtinDev-X25874Will.houWill.hou
 		39910f2b25 [Rust Frontend] Move code from vllm-frontend-rs (#43283) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
Signed-off-by: Dev-X25874 <283057883+Dev-X25874@users.noreply.github.com>
Signed-off-by: Will.hou <1205157517@qq.com>
Signed-off-by: Will.hou <willamhou@ceresman.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Eric Curtin <eric.curtin@docker.com>
Co-authored-by: Dev-X25874 <283057883+Dev-X25874@users.noreply.github.com>
Co-authored-by: Will.hou <1205157517@qq.com>
Co-authored-by: Will.hou <willamhou@ceresman.com>

Please see https://github.com/Inferact/vllm-frontend-rs for full original commit history.
 2026-05-21 17:21:48 -07:00