iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
21,722 Commits 3 Branches 0 Tags
382970ee6ca490aeaaaf4e32c53695b581ff61ba
Commit Graph
110 Commits
Author SHA1 Message Date
Jiahan Chang (Cyrus)andOpenAI Codex 4e5ffda19d [Bugfix] Isolate supplemental FlashInfer BF16 autotuning (#57285) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-17 07:38:58 +00:00
Lucas Wilkinson bd2d7ae7c1 [Perf] Reuse MoE workspace for DeepGEMM warmup (#57268) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-17 03:01:19 -04:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Woosuk Kwon 3b533197a9 [Perf] Parallelize mHC pre-norm JIT warmup (#56683) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-09-13 18:58:20 -07:00
Kevin H. Luu dec0b5d63f [Bugfix] Skip Triton autotune inspection without Triton (#56676) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-09-13 17:05:29 +00:00
Roberto L. CastroandCodex e52be1a62d [9/N][warmup][DSv4] Migrate MHC TileLang kernels (#50178) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-12 17:25:23 -07:00
Roberto L. CastroLucas WilkinsonOpenAI Codex
 		aed894c190 [6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-12 17:24:24 -07:00
Raphaël RiallandOpenAI CodexSimon VeitnerTomas Ruiz
 		7ee8a6dd01 [watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
 2026-09-12 18:42:01 +00:00
Ayushman Singhandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 7bcca16729 [Bugfix] Fix DeepGEMM FP8 warmup coverage (#56452) 
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-12 14:20:38 -04:00
Itay Alroy 658c8131c7 [Elastic EP] Reuse CUDA graphs across reconfiguration (#54985) 2026-09-12 10:26:39 -04:00
Matthew BonanniOpenAI CodexLucas WilkinsonLucas WilkinsonNicolò LucchesiOpenAI CodexClaude Opus 5Lucas Wilkinson
 		d43bb2f37f [3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-12 00:36:47 -04:00
Roberto L. CastroandCodex 120ec4ebd2 [5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels (#53566) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-12 00:05:59 -04:00
Yongye ZhuJee Jee Liandyluo7OpenAI Codexzjy0516Claude Opus 5
 		e77daef89e [Model] Support DeepSeek-V4.1-Flash (#56214) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-11 02:11:19 -07:00
Thien TranOpenAI CodexJee Jee Li
 		cc4210f671 [Perf] Improve BF16x3 router GEMM accuracy and make it default on sm100 (#55899) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-09 07:33:12 -07:00
Roberto L. CastroandCodex 6ddbab03de [4/N][warmup][DSv4] Migrate common attention kernels (#50176) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-08 10:25:48 -04:00
mingg26Kimi Codemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7fa2c63796 [Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture (#55377) 
Signed-off-by: ming <ming@inferact.ai>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-08 04:02:23 +00:00
Roberto L. CastroCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		42801b3a6b [3/N][warmup][DSv4] Migrate FA4 MLA and shared CuTeDSL kernels (#53565) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 09:36:35 -04:00
Roberto L. CastroCodexOpenAI CodexLucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1713b9866a [2/N][warmup][DSv4] Migrate sequence and DCP kernels (#53564) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 09:35:54 -04:00
vhagorandIsotr0py 3dc7a68ce4 [Perf] Extend Qwen Triton warmup to avoid first-request latency spikes (#54797) 
Signed-off-by: Juqi Li <2223621784@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-07 11:46:00 +08:00
Thien TranandKimi 31a8a26662 [Qwen3.8-Flash-Next] Improve QSA sparse GQA for prefill and short-ctx decode (#54873) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
 2026-09-04 06:08:02 -07:00
Kunshang JiandGitHub Copilot a8693df504 [SpecDecode]Fix spec decode warmup device selection (#55245) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: GitHub Copilot <copilot@github.com>
 2026-09-04 00:48:45 -07:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
Zupeng WangOpenAImergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c21751c90b [Kernel] Warm up Qwen GDN gated RMSNorm (#54251) 
Signed-off-by: Zupeng Wang <71580390+zupengwang@users.noreply.github.com>
Co-authored-by: OpenAI <support@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-03 18:19:48 +08:00
Gabriel WuandKimi Code 396c5a5632 [Bugfix] Lazy-import FlashInfer PCIe IPC all-reduce in kernel_warmup (#54869) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-02 04:31:03 +00:00
Yongye ZhuandOpenAI Codex 3ba9907a1d [Kimi-K3] Overlap low-M TP8 KDA projections (#54697) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-02 03:35:28 +00:00
Thien Tran 003e34341a [Qwen3.8-Flash-Next] Separate prefill and decode paths for QSA indexer (#54513) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-02 09:08:10 +08:00
Wentao Ye 6c58595c2d [Feature] Avoid flashinfer autotune each time when vllm source change (#54794) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-01 15:03:30 -04:00
Andrii SkliarandAndrii Skliar adebc41b7e [Mamba] Add FlashInfer ReplaySSM backend (#52506) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-09-01 10:26:43 -04:00
Gabriel WuYongye Zhumergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		446c769482 [Distributed] Add opt-in FlashInfer PCIe IPC all-reduce backend (#53576) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-01 03:34:56 +00:00
Roberto L. CastroandCodex e16b5e518d [1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels (#50175) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-31 23:26:05 -04:00
Jiahan Chang (Cyrus)OpenAI CodexJee Jee Li
 		7c877062ac [kernel] Integrate FlashInfer BF16 CuTeDSL Low Latency GEMM (#50572) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-27 22:29:12 +00:00
Ning Xie 6648eb118d [Core] drop duplicate VLLM_USE_DEEP_GEMM check (#48687) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-08-24 16:30:50 +00:00
Luke Alonsoandmgoin 2740c817ff [Kernel] Add b12x FP4 MoE backend (#52018) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-08-21 08:04:36 -07:00
Thien TranOpenAI Codexwzhao18
 		2785c72a14 [Kimi-K3] Extend GEMM-RS to GEMM-AR (#53053) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-08-21 03:44:15 +00:00
Roberto L. CastroandCodex 49905ad94d [3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup (#50174) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-17 10:11:14 -07:00
Misha Goin 4ab5e5012a [Refactor] Simplify B12X linear kernels and warmup (#52368) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-17 09:45:45 -07:00
Luke Alonso 3c79b1a8bf [Kernel] Add B12X dense linear backends (#52016) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
 2026-08-13 19:48:33 -07:00
Roberto L. CastroLucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6c95a641e9 [2/N][Feat][Perf] Add new warmup infrastructure for JITs. Add predicate filtering for JIT warmup, and migrate Inkling FA4 (#49315) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 07:46:07 -04:00
Wei ZhaojiahancKevin H. Luu
 		f18e10a7e1 Bump Flashinfer version to 0.6.16.post3 (#50892) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-08-09 00:54:07 -07:00
Michael GoinandOpenAI Codex 90329913e9 [UX] Reduce startup log noise (#50590) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 13:04:34 -07:00
bnellnmandHarry Mellor 6e311c6e20 [MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-31 11:28:18 +01:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Nick Hill 0d0504b54c [Core] Warm up runner-owned Triton kernels before the first request (#49903) 2026-07-28 07:58:02 -07:00
Itay Alroy 35efdf6b34 [Elastic EP] Async preparation (#47288) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-07-28 05:19:13 -07:00
Roberto L. Castro 56c96b0d91 [Perf] Tune LL BF16 Router GEMM (#48774) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
 2026-07-27 10:25:37 -04:00
neweyes 96fa3f42c9 [Perf] Skip ll_bf16 router GEMM warmup for non-MoE models (#49659) 
Signed-off-by: neweyes <328719365@qq.com>
 2026-07-27 05:16:42 -07:00
Elvir CrnčevićandClaude Opus 4.6 972848f276 [Bugfix] Support non-uniform page sizes in KVBlockZeroer (#49704) 
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-07-24 13:38:59 -07:00
Michael Goin 917fdb5bf7 [Bugfix] Fix DeepGEMM warmup when using FlashInferFp8DeepGEMMDynamicBlockScaledKernel (#49467) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-22 16:28:49 -07:00
Roberto L. Castro 7bb49be4d1 [Bugfix] Handle MLA fallback during FA4 JIT warmup (#49306) 2026-07-21 13:58:05 +00:00