iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
21,556 Commits 3 Branches 0 Tags
ac78445bc9ee3bb615b8b21db22fb7dac6698ece
Commit Graph
124 Commits
Author SHA1 Message Date
Taneem IbrahimandHarry Mellor e960ead0aa [Mypy] Fix mypy typing for Transformers models (#54320) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-17 12:11:31 -04:00
Misha Goin 1085b64425 [Benchmark] Retire stale benchmarks and consolidate RMSNorm (#57083) 
Signed-off-by: Misha Goin <mgoin64@gmail.com>
 2026-09-16 13:21:43 -04:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Tahsin Tunan 80650454dc [CI] Give every Buildkite step an explicit key, and a hook to enforce it (#56329) 
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
 2026-09-11 18:08:05 -07:00
Taneem Ibrahim f9083cb83a [Mypy] Fix typing for R/S models (#54157) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-10 09:43:08 -04:00
Taneem IbrahimandHarry Mellor 784cac7c42 [Mypy] Fix mypy typing for P models (#54169) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-04 20:47:11 +00:00
siyuliusy58Isotr0py
 		a69e75b9b6 Fast Start (#54921) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: liusy58 <liusy58@smail.nju.edu.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-04 06:32:06 -07:00
Taneem Ibrahim c615b1fd67 [Mypy] Fix mypy typing for L models (#54177) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-04 07:50:43 -04:00
Taneem Ibrahim 76f3249fbd [Mypy] Fix typing for M models (#54262) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-01 11:26:00 -04:00
Taneem Ibrahim fd5d3aea94 [Mypy] Fix typing for J models (#54130) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-29 08:50:15 -04:00
Taneem IbrahimandWentao Ye 21fa2c5a2a [Mypy] Fix mypy typing for model interfaces and H/I models (#54079) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-08-28 16:49:19 -04:00
Jensen ChenClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c47ca4aaeb [Mypy Fix] Mypy fix for "vllm/model_executor/models/[tT]" (#53466) 
Signed-off-by: Jensen Chen <a1043904820@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-26 14:12:11 +00:00
Wentao Ye 70021a0081 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[gG]" (#53616) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-25 09:47:16 -07:00
Aaron Hao 6cd9713463 [RL] P2P RDT weight sync (#43375) 2026-08-23 09:42:22 -07:00
Wentao Ye 811d12ee26 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[eE][fF]" (#53381) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-22 15:41:27 -04:00
Turner JabbourandClaude Sonnet 5 1eab6fef01 [CI] replace shellcheck script with shellcheck-py hook (#52572) 
Signed-off-by: Turner <doubleujabbour@gmail.com>
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-08-20 10:09:59 +00:00
jcotant-inferactClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c205726108 [CI] Fix and extend PR/issue auto-labeling (#51459) 
Signed-off-by: Joe Cotant <joe@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 21:45:44 +00:00
fxmarty-amdAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c5b7c069a4 [ROCm] Defer tilelang import through its import from vllm.tilelang_utils import tilelang and relaxed has_tilelang (#51159) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 13:12:40 -05:00
Wentao Ye 9a276d6375 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[cC][dD]" (#52003) 2026-08-13 11:33:55 +08:00
Michael GoinOpenAI CodexOpenAI Codex
 		21c667aa64 [Docker] Cache test dependencies before vLLM install (#51184) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-10 10:34:59 -07:00
Wentao Ye adc3e03517 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[aA][bB]" (#48977) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-06 19:30:15 +00:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
Ruinan MaandNick Hill a07fac758f [Perf] Zero-copy torch.Tensor pickling in shm_broadcast MessageQueue (#48442) 
Signed-off-by: Ruinan Ma <r7ma3088@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-28 16:28:28 -07:00
Harry Mellor 3e74c60b9c [Docs] Use gen-files for generated docs content (#49587) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-25 14:43:21 +00:00
Harry MellorandAndreas Karatzas ca0defa343 Make bare hugging_face imports forbidden (#49726) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-25 04:18:39 -07:00
Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> bd091079cb [Attention] FlashAttention 4 SM100 FP8 kv cache support (#42569) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-20 10:53:27 -07:00
Aaron HaoandSumanth R Hegde fb1d8ccaf5 [rl] Stateful Trainer Send: New Abstractions [1/N] (#48042) 
Signed-off-by: haoaaron <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: Sumanth R Hegde <39546518+SumanthRH@users.noreply.github.com>
 2026-07-17 15:11:06 +08:00
Wentao Ye 626c90b2d5 [Refactor] Move fla to third party (#48500) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-16 19:22:36 +01:00
Michael GoinandClaude Fable 5 ecf4aa5ce2 [Bugfix] Fix FlashInfer non-causal draft attention (DFlash/DSpark) on Blackwell (#48167) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-15 12:44:01 -07:00
Matthew Bonanni 313d01f507 [CI][Bugfix] Fix FlashAttention reported MLA dimension support (#48631) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-14 21:33:02 +00:00
Wentao Ye b3cfca996c [Mypy Fix] Split mypy work (#48490) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-13 12:42:42 +00:00
Bugen Zhao 487dfb3418 [CI] Add SPDX license header to Rust/Protobuf sources (#48472) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-13 10:22:47 +01:00
Kunshang JiandHarry Mellor 07f9baf756 Revert "[Platform] Replace torch.cuda.Event with torch.Event (#47140)" (#47668) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-06 14:18:33 +01:00
Dan BlanaruandMatthew Bonanni d29125c085 Xqa decode kernels (#43232) 
Signed-off-by: Dan Blanaru <48605845+DanBlanaru@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-02 12:32:05 -07:00
Kunshang JiandHarry Mellor e840f0d3f5 [Platform] Replace torch.cuda.Event with torch.Event (#47140) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 08:39:59 -07:00
Kunshang Ji fb42e5219e [Platform] Replace torch.cuda.mem_get_info with torch.accelerator.get_memory_info (#44825) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-06-30 14:39:52 +08:00
Martin Hickey a4e3cb40d0 [mypy] Enable mypy for tests directory (#47018) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-06-29 09:29:09 +00:00
Martin HickeyandHarry Mellor 275b43183c [MyPy] Fix mypy for vllm/benchmarks (#39896) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-23 15:22:29 +00:00
Gabriel WuClaude Opus 4.8Yongye Zhu
 		44d95069e9 Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-06-22 11:54:14 -07:00
Martin Hickey ccd49f6821 [MyPy] Fix mypy for vllm/lora (#41722) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-06-22 10:57:09 -04:00
Mohammad Miadh AngkadandMatthew Bonanni 9d4dc4ca2f [Kernel] Support GLM-5 dimensions for TRT-LLM ragged MLA prefill (#43525) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-06-16 20:49:47 +00:00
youkaichaoOpenAI CodexIsotr0pyThien TranBugen ZhaoJee Jee LiRoger WangfunctionstackxYongye ZhuJee Jee Li
 		0a1c5034f5 [Model] Add MiniMax M3 support (#45381) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-16 01:01:25 +08:00
Harry Mellor ef0df7dbd6 [CI] Bump mypy version 1.19.1 -> 1.20.2 (#44647) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-05 14:56:27 +00:00
Yongye Zhu b5235fca2e [DSv4] Adding TRTLLM gen attention kernel (#43827) 2026-06-04 07:35:09 -07:00
Mohammad Miadh Angkad 158289e0fc [Docs] Fix MLA prefill backend default docs (#43697) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-05-27 10:13:22 +00:00
Bugen ZhaoNick HillEric CurtinDev-X25874Will.houWill.hou
 		39910f2b25 [Rust Frontend] Move code from vllm-frontend-rs (#43283) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
Signed-off-by: Dev-X25874 <283057883+Dev-X25874@users.noreply.github.com>
Signed-off-by: Will.hou <1205157517@qq.com>
Signed-off-by: Will.hou <willamhou@ceresman.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Eric Curtin <eric.curtin@docker.com>
Co-authored-by: Dev-X25874 <283057883+Dev-X25874@users.noreply.github.com>
Co-authored-by: Will.hou <1205157517@qq.com>
Co-authored-by: Will.hou <willamhou@ceresman.com>

Please see https://github.com/Inferact/vllm-frontend-rs for full original commit history.
 2026-05-21 17:21:48 -07:00
Lanze LiuandMatthew Bonanni b2c58ee942 [FlashAttn] Fix supports_kv_cache_dtype() accepting unhandled fp8 kv-cache dtype variants (#42685) 
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-15 15:34:59 -04:00
Aaron Hao e0a45f1455 [Feat][RL] IPC weight sync optimizations: multigpu support and chunked packed tensors (#37476) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: hao-aaron <ahao@anyscale.com>
 2026-05-15 22:53:06 +08:00
CynicDora 256dbcaabf [Feature] Support custom callable proposer backend for speculative decoding (#39487) 
Signed-off-by: 524031910363 <hyzhyzsh@sjtu.edu.cn>
Signed-off-by: CynicDora <hyzhyzsh@sjtu.edu.cn>
 2026-05-13 16:53:01 +00:00
Matthew Bonanni be5983b874 [Docs] Add non-causal support to attention backend docs (#41643) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-04 20:35:15 +00:00