iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,722 Commits 3 Branches 0 Tags
382970ee6ca490aeaaaf4e32c53695b581ff61ba
Commit Graph
37 Commits
Author SHA1 Message Date
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Woosuk KwonandOpenAI Codex 2671fedfc7 [MRV2] Revert explicit Triton JIT warmup migration (#56654) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-13 04:07:56 +00:00
Roberto L. CastroLucas WilkinsonOpenAI Codex
 		aed894c190 [6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-12 17:24:24 -07:00
Canlin GuoandCodex dca96bf97b [Bugfix][Spec Decode] Avoid fastsafetensors deadlock for PP draft models (#54416) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-12 14:22:30 +08:00
Giancarlo Delfin 9dcf6bf344 [BugFix] Fix DP token padding in dflash attention metadata (#56181) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-09-11 14:33:53 -07:00
Lucas WilkinsonSummer YangOpenAI CodexOpenAI Codex
 		980c16c8e4 [PCP][Spec Decode] Adds PCP support for single-module MTP and replicated DSpark. (#56107) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-10 22:07:53 -07:00
Sy03Raphael RiallandZiyang ZhangCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		83990f5bcc [Bugfix][MRV2] Initialize DCP metadata after batch partitioning (#55212) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Ziyang Zhang <hafuhafu@qq.com>
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Ziyang Zhang <hafuhafu@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-10 06:50:09 +00:00
yongqinwang-cmdCursorWentao Ye
 		d87a440f88 [Core][MRV2] Support eagle3 spec decode with pipeline parallel (#50514) 
Signed-off-by: Yongqin Wang <yongqinwang@roblox.com>
Co-authored-by: Cursor <noreply@cursor.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-05 11:45:34 +08:00
Zihan Zhang 5093e4844a [Bugfix][Spec Decode] Drop FlashAttention's AOT schedule for a sliding-window DFlash drafter (#54374) 2026-09-04 16:42:01 -04:00
Giancarlo Delfin facd9a74a1 [Model Runner V2][Spec Decode] Skip DP sync for all speculator uniform decodes (#54856) 2026-09-03 10:32:33 +00:00
Zihan ZhangandClaude Opus 5 d61b6e1878 [Bugfix][Spec Decode] Take the DFlash draft's RoPE layout from its own config (#54373) 
Signed-off-by: SubSir <tiancaizhangdaxian@sjtu.edu.cn>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-31 18:13:20 -04:00
Giancarlo Delfin fe755c8899 [Bugfix][Model Runner V2][Spec Decode] Decouple the draft's gumbel noise stream from the target's (#54282) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-08-29 09:58:29 -07:00
Summer Yang 6d4562c59b [Attention][DCP] Enable FlashInfer MLA for DSpark drafting (#54277) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
 2026-08-28 21:11:28 -07:00
Giancarlo Delfin 5acc1c4e4b [Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft prefill (#53694) 2026-08-26 20:59:35 -07:00
Misha Goin f620499ee3 [Bugfix][Spec Decode] Reapply group geometry for FlashAttention metadata (#53336) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-24 08:08:43 -07:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Misha Goin 9ff7041b55 [Bugfix][Spec Decode] Use group geometry for FlashAttention metadata (#53002) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-21 17:07:19 -07:00
Wei Zhao d1e3eee6fb [Spec decode] Support Kimi-K3 DCP with DSpark (#52188) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-08-17 13:08:45 -07:00
Gabriel Wu 97388c44f9 [Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538) 2026-08-15 07:01:32 -07:00
Tiezhen WANGBeto de PaolaGiancarlo DelfinJee Jee LiClaude Opus 5Jee Jee Lizixi-qi
 		6adad08767 Add Muse Glimmer model support (#51655) 
Signed-off-by: Tiezhen Wang <tiezhen@inferact.ai>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Beto de Paola <betodepaola@meta.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: zixi-qi <zixi@inferact.ai>
 2026-08-14 12:28:56 +08:00
Lucas WilkinsonOpenAI CodexClaude Opus 5Benjamin Chislettmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick HillOpenAI Codex
 		7f7a32cfec [Spec Decode] DSpark confidence-scheduled verification (#47808) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-12 12:27:48 -07:00
Giancarlo Delfin dec13a33b7 [Model Runner V2][Spec Decode] Add multi-layer MTP speculator (#48892) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-07-30 15:54:15 -07:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Fynn Schmitt-UlmsandMichael Goin 642076d26c Support loading sample_from_anchor flag from speculators config (#48639) 
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-07-20 14:52:43 -07:00
Oxana KorzhCursorokorzh-amdMatthew Bonanni
 		4ec199b66a [Bugfix][Spec-Decode] Populate draft seq_lens_cpu_upper_bound for spec-decode attention metadata (#44492) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Signed-off-by: okorzh-amd <okorzh-amd@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: okorzh-amd <okorzh-amd@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-20 20:50:58 +00:00
Michael GoinandClaude Opus 4.8 8bfd683901 [Spec Decode] Add kv_cache_dtype to speculative_config to control separately from target (#48787) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-15 23:49:23 -06:00
Michael GoinandClaude Fable 5 ecf4aa5ce2 [Bugfix] Fix FlashInfer non-causal draft attention (DFlash/DSpark) on Blackwell (#48167) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-15 12:44:01 -07:00
Nick Hill 26587f9519 [BugFix][ModelRunner V2] Fix stale attn metadata in speculator prefill cudagraph capture (#48261) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-13 09:39:15 -07:00
Michael GoinandClaude Opus 4.8 0d12618e98 [Spec Decode] Support hybrid (SWA + full attention) DFlash drafters (#47914) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-08 11:12:45 -04:00
Benjamin Chislett 7bd154375d [Bugfix] Fix mamba+dflash for MRV2 (#47698) 2026-07-07 15:59:13 -04:00
Benjamin ChislettGiancarlo Delfinmgoin
 		f5a8d73377 [Spec Decode] DSpark (#46995) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-07-01 08:30:24 -07:00
Giancarlo Delfin 3cecee40f3 [Model Runner V2][Spec Decode] Fix stale values in idx_mapping from CG num reqs padding (#47066) 2026-06-30 11:25:32 -07:00
Ilya MarkovandMarkov Ilya a2abce646f [EPLB] Mask padding in EPLB load recording (#38128) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
 2026-06-28 19:43:58 -07:00
Giancarlo Delfin 5314665bad [Model Runner V2][DFlash] Enable dflash attention backend selection (#46770) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-06-25 19:29:25 -07:00
Giancarlo Delfin ef361de916 [Model Runer V2][DFlash] Fix lm head sharing for dflash (#46435) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-06-23 19:09:06 +00:00
Benjamin ChislettandGiancarlo Delfin 0bae1d3848 [MRV2][Spec Decode] DFlash (#44586) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-06-10 08:47:46 -07:00