iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,991 Commits 3 Branches 0 Tags
c19c263ac8ea3fd6e1d2d689f0e4e725855d89cf
Commit Graph
126 Commits
Author SHA1 Message Date
simon-veitner-redhatClaudeRoberto L. Castro
 		253e02b95e [Perf] Warm up DeepGEMM MegaMoE kernels before serving (#60468) 
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
 2026-10-13 20:17:48 +08:00
Nick Hill 3fe917abb1 [Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 (#61013) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 16:04:41 -07:00
siyuClaude Opus 5Isotr0py
 		f2376cdc4d [Fast Start] Preload the FlashInfer autotune table on the weight cache daemon (#60085) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-10 15:48:06 +00:00
Wentao Ye 3709632ff2 [Refactor] Remove dead code multiple places (#60120) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-10 15:41:10 +00:00
SeongJun LeeandIsotr0py 699c892dd5 [Perf] Skip the M-RoPE warmup when every holder of the RoPE fuses it (#60246) 
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-10 06:47:39 +00:00
Jiahan Chang (Cyrus) 816173ca9d [Kernel] Remove WAR for LL CuTeDSL BF16 GEMM (#60434) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
 2026-10-08 05:38:07 +00:00
Eduardo FonsecaYongye ZhuClaude Opus 5.5
 		fba462e008 [Bugfix]Fix FlashInfer warmup crash: autotune(tuning_buckets=...) must round M up, not down (Invalid MXFP8 split-K tactic with spec-decode) (#58165) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 14:01:21 -04:00
Madhav JivrajaniNickLuccheWei Zhao
 		3403e0f176 [Bugfix] Persist FlashInfer autotune cache per rank to fix the rank-0-only cache-hit deadlock (#57635) 
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-10-06 17:51:01 +00:00
Chris LeonardandShengqi Chen cc73cca9c8 [Build] Migrate vendored DeepGEMM from pybind to TORCH_LIBRARY (abi3) (#48962) 
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-10-06 08:39:04 -07:00
big_yellow_duckTan Pin SiangvllmellmCodex
 		edfccc9469 [ROCM][Perf] Add ROCM_SEGMENTED_ATTN attention-backend works across RDNA3/3.5/4 (#59132) 
Signed-off-by: root <jeffaw99@hotmail.com>
Signed-off-by: big-yellow-duck <jeffaw99@hotmail.com>
Co-authored-by: Tan Pin Siang <1716735+tanpinsiang@users.noreply.github.com>
Co-authored-by: vllmellm <190700713+vllmellm@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-06 07:53:47 +00:00
SeongJun Leelesj0610Claude Opus 5Thien Tran
 		24df2d3fc7 [Model] Add Qwen4Exp to the Qwen GDN Triton warmup (#56742) 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-02 21:38:15 +08:00
YueshenZYueshenZClaude CodeCodexzjy0516
 		08d77cad7b [Bugfix] Gate Kimi-K3 KDA warmup on sys.modules to skip Kimi import for non-Kimi models (#59257) 
Signed-off-by: YueshenZ <YueshenZ@users.noreply.github.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: YueshenZ <YueshenZ@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-10-01 09:25:45 +00:00
Andrii SkliarandAndrii Skliar deb148e730 [Mamba] Add FlashInfer ReplaySSM support for MTP (#52928) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-09-30 21:41:17 +00:00
VenkyandClaude Opus 5.5 42f0c17ea7 [Bugfix][PP][Spec Decode] MiniMax-M3 EAGLE3 aux-state relay at PP > 1 and per-stage FlashInfer autotune (#57197) 
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-01 01:14:24 +04:00
Misha GoinandCodex 71891bdb94 [Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-24 19:52:00 -04:00
Canlin GuoThien TranOpenAI CodexClaude Fable 5.1Jee Jee Li
 		f92b78f6ef [Kernel][DSV4.1] Fuse MXFP8 wo_b GEMM with sequence-parallel reduce-scatter (#57428) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-22 01:07:17 -07:00
Jiahan Chang (Cyrus)andOpenAI Codex 4e5ffda19d [Bugfix] Isolate supplemental FlashInfer BF16 autotuning (#57285) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-17 07:38:58 +00:00
Lucas Wilkinson bd2d7ae7c1 [Perf] Reuse MoE workspace for DeepGEMM warmup (#57268) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-17 03:01:19 -04:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Woosuk Kwon 3b533197a9 [Perf] Parallelize mHC pre-norm JIT warmup (#56683) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-09-13 18:58:20 -07:00
Kevin H. Luu dec0b5d63f [Bugfix] Skip Triton autotune inspection without Triton (#56676) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-09-13 17:05:29 +00:00
Roberto L. CastroandCodex e52be1a62d [9/N][warmup][DSv4] Migrate MHC TileLang kernels (#50178) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-12 17:25:23 -07:00
Roberto L. CastroLucas WilkinsonOpenAI Codex
 		aed894c190 [6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-12 17:24:24 -07:00
Raphaël RiallandOpenAI CodexSimon VeitnerTomas Ruiz
 		7ee8a6dd01 [watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
 2026-09-12 18:42:01 +00:00
Ayushman Singhandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 7bcca16729 [Bugfix] Fix DeepGEMM FP8 warmup coverage (#56452) 
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-12 14:20:38 -04:00
Itay Alroy 658c8131c7 [Elastic EP] Reuse CUDA graphs across reconfiguration (#54985) 2026-09-12 10:26:39 -04:00
Matthew BonanniOpenAI CodexLucas WilkinsonLucas WilkinsonNicolò LucchesiOpenAI CodexClaude Opus 5Lucas Wilkinson
 		d43bb2f37f [3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-12 00:36:47 -04:00
Roberto L. CastroandCodex 120ec4ebd2 [5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels (#53566) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-12 00:05:59 -04:00
Yongye ZhuJee Jee Liandyluo7OpenAI Codexzjy0516Claude Opus 5
 		e77daef89e [Model] Support DeepSeek-V4.1-Flash (#56214) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-11 02:11:19 -07:00
Thien TranOpenAI CodexJee Jee Li
 		cc4210f671 [Perf] Improve BF16x3 router GEMM accuracy and make it default on sm100 (#55899) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-09 07:33:12 -07:00
Roberto L. CastroandCodex 6ddbab03de [4/N][warmup][DSv4] Migrate common attention kernels (#50176) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-08 10:25:48 -04:00
mingg26Kimi Codemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7fa2c63796 [Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture (#55377) 
Signed-off-by: ming <ming@inferact.ai>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-08 04:02:23 +00:00
Roberto L. CastroCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		42801b3a6b [3/N][warmup][DSv4] Migrate FA4 MLA and shared CuTeDSL kernels (#53565) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 09:36:35 -04:00
Roberto L. CastroCodexOpenAI CodexLucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1713b9866a [2/N][warmup][DSv4] Migrate sequence and DCP kernels (#53564) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 09:35:54 -04:00
vhagorandIsotr0py 3dc7a68ce4 [Perf] Extend Qwen Triton warmup to avoid first-request latency spikes (#54797) 
Signed-off-by: Juqi Li <2223621784@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-07 11:46:00 +08:00
Thien TranandKimi 31a8a26662 [Qwen3.8-Flash-Next] Improve QSA sparse GQA for prefill and short-ctx decode (#54873) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
 2026-09-04 06:08:02 -07:00
Kunshang JiandGitHub Copilot a8693df504 [SpecDecode]Fix spec decode warmup device selection (#55245) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: GitHub Copilot <copilot@github.com>
 2026-09-04 00:48:45 -07:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
Zupeng WangOpenAImergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c21751c90b [Kernel] Warm up Qwen GDN gated RMSNorm (#54251) 
Signed-off-by: Zupeng Wang <71580390+zupengwang@users.noreply.github.com>
Co-authored-by: OpenAI <support@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-03 18:19:48 +08:00
Gabriel WuandKimi Code 396c5a5632 [Bugfix] Lazy-import FlashInfer PCIe IPC all-reduce in kernel_warmup (#54869) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-02 04:31:03 +00:00
Yongye ZhuandOpenAI Codex 3ba9907a1d [Kimi-K3] Overlap low-M TP8 KDA projections (#54697) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-02 03:35:28 +00:00
Thien Tran 003e34341a [Qwen3.8-Flash-Next] Separate prefill and decode paths for QSA indexer (#54513) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-02 09:08:10 +08:00
Wentao Ye 6c58595c2d [Feature] Avoid flashinfer autotune each time when vllm source change (#54794) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-01 15:03:30 -04:00
Andrii SkliarandAndrii Skliar adebc41b7e [Mamba] Add FlashInfer ReplaySSM backend (#52506) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-09-01 10:26:43 -04:00
Gabriel WuYongye Zhumergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		446c769482 [Distributed] Add opt-in FlashInfer PCIe IPC all-reduce backend (#53576) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-01 03:34:56 +00:00
Roberto L. CastroandCodex e16b5e518d [1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels (#50175) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-31 23:26:05 -04:00
Jiahan Chang (Cyrus)OpenAI CodexJee Jee Li
 		7c877062ac [kernel] Integrate FlashInfer BF16 CuTeDSL Low Latency GEMM (#50572) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-27 22:29:12 +00:00
Ning Xie 6648eb118d [Core] drop duplicate VLLM_USE_DEEP_GEMM check (#48687) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-08-24 16:30:50 +00:00
Luke Alonsoandmgoin 2740c817ff [Kernel] Add b12x FP4 MoE backend (#52018) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-08-21 08:04:36 -07:00