iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,722 Commits 3 Branches 0 Tags
382970ee6ca490aeaaaf4e32c53695b581ff61ba
Commit Graph
165 Commits
Author SHA1 Message Date
Cyrus Leung c8824a1e42 [MM] Move supports_multimodal_inputs and cache out of registry (#57913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-21 07:43:07 -07:00
Yifan QiaoandClaude Fable 5.1 d2983f2f16 [Bugfix][Spec Decode] Stop dummy draft decode steps from writing KV through stale block-table rows (#56734) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-21 01:06:25 +00:00
David CheungClaudeNick Hill
 		5302d1fe40 [Model Runner V2] Support custom logits processors (#56497) 
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-19 17:46:49 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Nick Hill f2aad6aa70 [MRV2] Buffer util simplifications (#56888) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-15 20:36:51 +00:00
zack 0136df94b0 [Bugfix][Spec Decode][MoE] Avoid uninitialized EPLB state in DeepSeek V4.1 DSpark drafter (#56387) 
Signed-off-by: zack <51604064+luoyuctl@users.noreply.github.com>
 2026-09-15 14:35:47 +00:00
Giancarlo Delfin 3bb0a03f35 [Model Runner V2] Acceptance estimation for adaptive verification (#52228) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-09-14 15:58:48 -07:00
Woosuk KwonandOpenAI Codex 2671fedfc7 [MRV2] Revert explicit Triton JIT warmup migration (#56654) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-13 04:07:56 +00:00
Roberto L. CastroLucas WilkinsonOpenAI Codex
 		aed894c190 [6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-12 17:24:24 -07:00
Raphaël RiallandOpenAI CodexSimon VeitnerTomas Ruiz
 		7ee8a6dd01 [watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
 2026-09-12 18:42:01 +00:00
Canlin GuoandCodex dca96bf97b [Bugfix][Spec Decode] Avoid fastsafetensors deadlock for PP draft models (#54416) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-12 14:22:30 +08:00
Giancarlo Delfin 9dcf6bf344 [BugFix] Fix DP token padding in dflash attention metadata (#56181) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-09-11 14:33:53 -07:00
Yongye ZhuJee Jee Liandyluo7OpenAI Codexzjy0516Claude Opus 5
 		e77daef89e [Model] Support DeepSeek-V4.1-Flash (#56214) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-11 02:11:19 -07:00
Andreas KaratzasOpenAI CodexLucas Wilkinson
 		d0dfe587d5 [Bugfix] Fall back to full decode graphs for noncompiled models (#55095) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-11 01:56:05 -07:00
Lucas WilkinsonSummer YangOpenAI CodexOpenAI Codex
 		980c16c8e4 [PCP][Spec Decode] Adds PCP support for single-module MTP and replicated DSpark. (#56107) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-10 22:07:53 -07:00
Sy03Raphael RiallandZiyang ZhangCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		83990f5bcc [Bugfix][MRV2] Initialize DCP metadata after batch partitioning (#55212) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Ziyang Zhang <hafuhafu@qq.com>
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Ziyang Zhang <hafuhafu@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-10 06:50:09 +00:00
Kevin H. LuuCodexWentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		5acd95906b [Bugfix] Exclude DP token padding from draft attention metadata (#55458) 
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-09 01:47:45 -07:00
Jing Wang c268198715 [Bugfix][Spec Decode] Preserve target parallel config (DCP) for DSpark (#55472) 
Signed-off-by: Jing Wang <jingwang96@qq.com>
 2026-09-08 16:59:10 -07:00
Stefano CastagnettaNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		9c2d21046b [Bugfix][Spec Decode] Honour the draft's moe_backend on Model Runner V2 (#54788) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-08 16:50:07 -07:00
yongqinwang-cmdCursorWentao Ye
 		d87a440f88 [Core][MRV2] Support eagle3 spec decode with pipeline parallel (#50514) 
Signed-off-by: Yongqin Wang <yongqinwang@roblox.com>
Co-authored-by: Cursor <noreply@cursor.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-05 11:45:34 +08:00
Zihan Zhang 5093e4844a [Bugfix][Spec Decode] Drop FlashAttention's AOT schedule for a sliding-window DFlash drafter (#54374) 2026-09-04 16:42:01 -04:00
Stefano Castagnetta 2524051385 [Bugfix][Spec Decode] Honour the draft's attention_backend on Model Runner V2 (#54826) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-09-04 12:23:19 -07:00
Giancarlo Delfin facd9a74a1 [Model Runner V2][Spec Decode] Skip DP sync for all speculator uniform decodes (#54856) 2026-09-03 10:32:33 +00:00
Zihan ZhangandClaude Opus 5 d61b6e1878 [Bugfix][Spec Decode] Take the DFlash draft's RoPE layout from its own config (#54373) 
Signed-off-by: SubSir <tiancaizhangdaxian@sjtu.edu.cn>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-31 18:13:20 -04:00
Giancarlo Delfin fe755c8899 [Bugfix][Model Runner V2][Spec Decode] Decouple the draft's gumbel noise stream from the target's (#54282) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-08-29 09:58:29 -07:00
Summer Yang 6d4562c59b [Attention][DCP] Enable FlashInfer MLA for DSpark drafting (#54277) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
 2026-08-28 21:11:28 -07:00
Cheng JiangchengvjiangJee Jee LiBugen ZhaorussellfengJee Jee Li
 		b2f685834a [Hy4] support Hy4-preview model (#54160) 
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: russellfeng <russellfeng@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-29 12:07:28 +08:00
Giancarlo Delfin 5acc1c4e4b [Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft prefill (#53694) 2026-08-26 20:59:35 -07:00
Misha Goin 7de96050c5 [Spec Decode] Enable adaptive DSpark on SM100 sparse MLA (#52783) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-25 13:31:15 -04:00
Naveenraj KamalakannanandBenjamin Chislett d5cadcee86 [Feature][DSpark]: Logprobs adaptive verification (#52242) 
Signed-off-by: Naveenraj Kamalakannan <therealnaveenkamal@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-08-25 10:22:33 -04:00
Misha Goin f620499ee3 [Bugfix][Spec Decode] Reapply group geometry for FlashAttention metadata (#53336) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-24 08:08:43 -07:00
Zhou248 2f55ef254c [Model] Add Qwen3-Omni DSpark support (#52560) 
Signed-off-by: Zhou248 <630563665@qq.com>
 2026-08-22 15:35:22 -07:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Misha Goin 9ff7041b55 [Bugfix][Spec Decode] Use group geometry for FlashAttention metadata (#53002) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-21 17:07:19 -07:00
Zihan Zhang b389ac2946 [Spec Decode] DFlash2: local convolution + candidate selector (#52816) 2026-08-20 22:27:22 -07:00
Misha GoinCodexOpenAI Codex
 		91a893de64 [Bugfix][Spec Decode] Scope DSpark backend inheritance to DeepSeek V4 (#52809) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-20 18:42:08 -07:00
Zupeng WangOpenAIMisha GoinOpenAI Codex
 		01af92e175 [Feature][Model Runner V2] Support extract_hidden_states speculation (#49811) 
Signed-off-by: Zupeng Wang <71580390+zupengwang@users.noreply.github.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI <support@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-20 11:13:43 -04:00
Giancarlo Delfin d4f4d3f40f [Model Runner V2][Spec Decode] Fix draft logits cache column stride in gumbel_sample (#53017) 2026-08-19 17:39:30 -07:00
Roger Wang 755492e37d Revert "[Kernel] Gemma-4 FA4 FP8 Kernel" (#52987) 2026-08-19 10:48:46 -07:00
Wei Zhao d1e3eee6fb [Spec decode] Support Kimi-K3 DCP with DSpark (#52188) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-08-17 13:08:45 -07:00
Gabriel Wu 97388c44f9 [Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538) 2026-08-15 07:01:32 -07:00
Yongye Zhu acb0f1dcdb [Bugfix][Spec Decode] DSpark: inherit the target's attention backend when the speculative config names none (#52288) 2026-08-15 09:32:59 +08:00
Tiezhen WANGBeto de PaolaGiancarlo DelfinJee Jee LiClaude Opus 5Jee Jee Lizixi-qi
 		6adad08767 Add Muse Glimmer model support (#51655) 
Signed-off-by: Tiezhen Wang <tiezhen@inferact.ai>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Beto de Paola <betodepaola@meta.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: zixi-qi <zixi@inferact.ai>
 2026-08-14 12:28:56 +08:00
Jhao-Ting ChenQixiang LinClaude Opus 4.8Yongye Zhu
 		827a2af806 [Kernel] Gemma-4 FA4 FP8 Kernel (#48666) 
Signed-off-by: Qixiang Lin <qixiangl@nvidia.com>
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Qixiang Lin <qixiangl@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-13 17:42:00 -07:00
Giancarlo DelfinandClaude Fable 5 f80b66f548 [Model Runner V2][Spec Decode] Add KV cache support for multi-layer MTP (#50062) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-13 16:22:29 -07:00
Lucas WilkinsonOpenAI CodexClaude Opus 5Benjamin Chislettmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick HillOpenAI Codex
 		7f7a32cfec [Spec Decode] DSpark confidence-scheduled verification (#47808) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-12 12:27:48 -07:00
Nick HillRahul ChalamalaJanelle Cai
 		0a94d85a66 [Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch (#51865) 
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
 2026-08-11 17:15:06 -07:00
Yizhou a311916a29 [MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph (#46849) 
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
 2026-08-11 15:06:22 +08:00
Giancarlo DelfinandWoosuk Kwon 3e174bb73c [Model Runner V2][MTP] Share topk index buffer between draft steps (#47352) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-10 15:46:13 -07:00