iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,991 Commits 3 Branches 0 Tags
c19c263ac8ea3fd6e1d2d689f0e4e725855d89cf
Commit Graph
157 Commits
Author SHA1 Message Date
TaimysTaimysHarry Mellor
 		c068db0e1d [Bugfix][Compile] Replace Transformers AOT caching with graph caching (#58918) 
Signed-off-by: fangyao fan <2281693949@qq.com>
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-13 19:17:29 +00:00
Mikko TukiainenandCursor 81c993658e [ROCm][Perf] Fuse main QK-norm/RoPE/gate and KV-cache write into the AMD QSA prepare launch (#59732) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-09 21:14:49 +08:00
Woosuk Kwon 194bd72be7 [Compile] Remove torch.compile-based sequence parallelism and AsyncTP (#60517) 2026-10-08 08:28:23 -07:00
Cheese CakeOpenAI CodexClaude Opus 5.5
 		b9767299db [Bugfix] Respect Inductor deterministic mode in combo-kernel defaults (#59074) 
Signed-off-by: cheesecake <farzanaman99@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 20:08:11 +01:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		eb2901b735 [Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs (#59532) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 22:11:45 -07:00
vorapolsiloaiSantosh HiremathDouglas Lehr
 		b715699df4 [ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE (#50212) 
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com>
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-09-23 11:11:57 -05:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Andreas KaratzasOpenAI CodexLucas Wilkinson
 		d0dfe587d5 [Bugfix] Fall back to full decode graphs for noncompiled models (#55095) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-11 01:56:05 -07:00
Thien Tran d9105ea800 [Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation (#55272) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-06 21:57:56 -07:00
+5
Harry Huangzjy0516KungYorkThien TranYifan Qiaoandyluo7Isotr0pyandyluo7OpenAI Codexlishunyang12Vadim GimpelsonIsotr0pyLucas WilkinsonClaude Opus 5OpenAI Codex
 		e126687a9a [Model] Support Qwen3.8-Flash-Next (#53896) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-31 05:57:56 +00:00
Kevin H. Luu b2dc864bb6 [Bugfix][Spec Decode] Keep default CUDA graph sizes memory-safe (#54418) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-30 18:02:31 +00:00
Rahul ChalamalaJanelle CaiDevin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>Devin
 		d3d79ffc1e [Bugfix][Spec Decode] Capture the widest uniform decode batch by default (#50488) 
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Rahul Chalamala <rahul@modal.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin <devin@cognition.ai>
 2026-08-28 22:38:13 -07:00
Itay Alroy 6d11122607 [Elastic EP] Preserve AOT cache reuse during scaling (#53378) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-08-27 14:55:44 +00:00
Jie FangOpenAI CodexClaude Fable 5
 		1be3628367 [Kernel][Perf] Add fused CUDA post-conv MTP decode kernel for Qwen3.5 GDN (#51674) 
Signed-off-by: Jie Fang <jief@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-14 11:44:45 +08:00
Lucas Wilkinson 021b7d985b [Perf] Raise Blackwell CUDA graph capture default to 1024 (#49390) 2026-08-07 15:18:14 -04:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Taneem Ibrahim 0da6e7f3d6 [Bugfix] Reject contradictory custom-op directives (#49134) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-07-26 08:42:14 -04:00
Harry Mellor 84d26b9ee3 [Model] Remove Plamo2 (#49729) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-24 13:54:24 -07:00
Jack HuandDouglas Lehr 2cab53ddee [Model][Hardware][AMD]: Part 1/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_AITER_FA, and ROCM_AITER_UNIFIED_ATTN (#42749) 
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-07-16 17:39:04 -05:00
Ekagra RanjanBenjamin ChislettNick Hill
 		07516fda67 [MRV2][SD] Make Dynamic SD comatible with Full Cuda Graphs (#45953) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-03 23:58:27 -04:00
Cheng JiangchengvjiangAndreas Karatzas
 		8cf7c4d8ad [Attention Backend] add HPC-Ops Attention backend (#46020) 
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-30 18:17:43 +08:00
Yejing LaiandKunshang Ji ae2c4f3db7 [XPU][UT]Fix xpu pass_config.fuse_norm_quant assert issue (#46804) 
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-06-29 21:13:44 -07:00
Jeff (Junze) Maandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> fc61c6fc26 [Perf] Enable + tune FlashInfer fused allreduce at world_size=16 on SM 10.3 (GB300) (#46392) 
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-24 23:04:17 -07:00
Itay Alroy 3afe659b6b [EP] Enable DBO with NIXL EP (#45275) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-06-15 23:37:22 +00:00
Harry Mellor 03878d1c22 Deprecations for v0.23 and v0.24 (#44992) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-11 14:35:38 +00:00
Jee Jee Li 6e503868ca [Kernel] Porting fuse_minimax_qk_norm to manual fusion (#43410) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-26 13:16:03 -07:00
wangxiyuan 7e1b45a092 [Attention] Mamba attention module refactor (#41126) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-05-22 17:13:12 +08:00
Baorun (Lauren) Mu a7737cb4f3 [Fix] Misc Fixes in ViT CUDA Graph (#38040) 
Signed-off-by: Baorun Mu <bmu@nvidia.com>
 2026-05-14 23:49:06 +08:00
Rohan PotdarandElizaWszola a51376b3f0 [Performance][DSR1]: Fused RoPE+KVCache+q_concat for MLA (#40392) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
 2026-05-11 14:10:50 +00:00
+7
Yifan QiaoYongye ZhuYongye ZhuSimon MoBugen ZhaoGiancarlo DelfinJee Jee LiNick HillRoger WangRoy WangWoosuk KwonyoukaichaoZhewen LiZijing LiukhluuqizixiZhewen Li
 		4d51588e23 [Feat] DeepSeek V4 Rebased (#40860) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
 2026-04-26 18:31:08 -07:00
Xinan MiaoSouthWest7gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>OpenAI CodexWang Xingranmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		32e45636e3 [torch.compile]: Disable Sequence Parallelism (SP) for piecewise compilation (#38373) 
Signed-off-by: SouthWest7 <am1ao@qq.com>
Signed-off-by: Xinan Miao <1403572259@qq.com>
Co-authored-by: SouthWest7 <am1ao@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Wang Xingran <72983099+wangxingran222@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-26 17:44:42 +00:00
Yuwen ZhouandKunshang Ji 4c34b2f6fc [XPU] Enable torch.compile for XPU GDN attention (#39466) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Yuwen Zhou <yuwen.zhou@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-24 16:26:16 +08:00
Shanshan Shen 936e0b79aa [MM][CG] Optimize default max_frames_per_batch auto-infer for ViT CUDA graph video inference (#40445) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-04-21 14:47:53 +00:00
Rita Brugarolas fb5635d3f9 [ROCm] Add MLA dual RMS norm fusion (Q, KV) pass for DeepSeek/Kimi-K2 (#39242) 
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
 2026-04-20 14:56:27 +00:00
Lucas Kabela c08f3b2a62 Measure encoder compile time seperate from llm backbone (#39240) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-04-14 10:52:49 -04:00
Shanshan ShenandRoger Wang 80118853f4 [MM][Perf][CG] Support ViT full CUDA graph for Qwen3-VL video inference (#38061) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-14 16:49:32 +08:00
Jee Jee Li ecd1ea1363 [Kernel] Porting the TRTLLM minimax_allreduce_rms kernels (#37045) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-11 00:20:20 +08:00
zhrrr 8f121f7879 [Model Runner V2] support auto resolve cudagraph mode/sizes based on attn backend (#32936) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
 2026-04-10 08:27:15 -07:00
Lucas Wilkinson 70406eb1dc [Attention][V0 Deprecation] Deprecate accept output buffer (#39125) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-04-07 17:14:58 -04:00
Carl YandClaude Opus 4.6 1f5ec2889c [mla] Support fused FP8/NVFP4 output quantization in MLA attention (#35792) (#36205) 
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
Signed-off-by: Carl Y <4531192+carlyou@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-02 21:16:11 -04:00
Harry Mellor dfe5e31689 Don't compile vision encoder for Transformers backend (#30518) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-04-02 12:42:29 +00:00
Luka GovedičXinyu ChenChaojun ZhangLuka Govedič
 		40bb175027 [vLLM IR] 1/N Implement IR skeleton and rms_norm op (#33825) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Signed-off-by: chzhang <chaojun.zhang@intel.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Luka Govedič <ProExpertProg@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
 2026-03-31 22:15:05 -04:00
Nicolò Lucchesi cc06b4e86b [Mamba][Bugfix] Raise on insufficient cache blocks instead of silently capping cudagraph sizes (#38270) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-30 09:41:50 +00:00
Rohan Potdar a0e8c74005 [ROCm]: Update rope+kvcache fusion conditions and disable custom op by default (#36716) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-03-25 20:58:44 +00:00
Harry Mellor d215d1efca [Mypy] Better fixes for the mypy issues in vllm/config (#37902) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-25 06:14:43 -07:00
Terry GaoandTianren Gao 82580b10ac [Perf] Disable inductor runtime asserts by default for serving perfor… (#37485) 
Signed-off-by: tianrengao <terrygao87@gmail.com>
Co-authored-by: Tianren Gao <tianren@fb.com>
 2026-03-24 19:37:51 -04:00
Wentao Ye 45bd5c8e75 [Mypy] Fix mypy for vllm/config (#37808) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-23 14:33:59 +00:00
Baorun (Lauren) Mu f85e479e66 [Feature] ViT Full CUDA Graph (#35963) 
Signed-off-by: Baorun Mu <bmu@nvidia.com>
 2026-03-23 13:01:10 +08:00
Lucas Kabela 7769b58307 [torch.compile][BE][Multimodal] Remove requirement to set_model_tag to avoid cache conflict (#37345) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-03-19 17:26:12 +00:00