iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,722 Commits 3 Branches 0 Tags
382970ee6ca490aeaaaf4e32c53695b581ff61ba
Commit Graph
99 Commits
Author SHA1 Message Date
Zoe923 6a7de3ba3a [Bugfix][GDN] Fix CuteDSL BF16 KKT inversion divergence (#53864) 
Signed-off-by: Zoe923 <871844693@qq.com>
 2026-09-17 08:49:56 -07:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 18f8aa0465 [Refactor] Remove dead kernel code (#56845) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-15 13:17:53 -07:00
Roberto L. CastroLucas WilkinsonOpenAI Codex
 		aed894c190 [6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-12 17:24:24 -07:00
Jee Jee Li 199cb9b964 [Kernel] Remove unused fake implementation (#55535) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-09-06 17:54:32 -07:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
Andrii SkliarandAndrii Skliar adebc41b7e [Mamba] Add FlashInfer ReplaySSM backend (#52506) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-09-01 10:26:43 -04:00
Dinesh Chitlangia 525b7bbb3a [Bugfix][CPU] Enable C++ causal_conv1d GDN path and float32 SSM cache on non-AMX AVX-512BF16 CPUs (#49688) 
Signed-off-by: Dinesh Chitlangia <dineshchitlangia@gmail.com>
 2026-08-19 10:50:59 -05:00
pmanczak cc7cf71fc8 [XPU] Enable Kimi K3 KDA kernel tests on XPU (#51809) 
Signed-off-by: pmanczak <pawel.manczak@intel.com>
 2026-08-17 17:21:34 +08:00
pmanczak c4e969294e [XPU] Add tuned Mamba SSU configs for Intel Arc Pro B70 (#50534) 
Signed-off-by: pmanczak <pawel.manczak@intel.com>
 2026-08-13 12:36:55 +00:00
vanshbhatia-amd e229fdbc87 [ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI325X (#50007) 
Signed-off-by: vanshbhatia-amd <vansh.bhatia@amd.com>
 2026-08-07 12:58:36 -04:00
amitz-nvandOpenAI Codex 52c0e3cb08 [Kernel] Add support for Flashinfer Mamba SSU algorithm selection (#50157) 
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-04 09:40:23 -07:00
vanshbhatia-amd 4689c7dd61 [ROCm] Add tuned selective_state_update float16 config for AMD Instinct MI325X (#50006) 
Signed-off-by: vanshbhatia-amd <vansh.bhatia@amd.com>
 2026-07-31 02:43:53 +00:00
Li, JiangandClaude 072a472782 [CPU] Bump up CPU kernels to latest version (#50387) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-30 19:18:43 +08:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
+12
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodex
 		7c6729b769 [Model] Add Kimi K3 support: model files and kernels [1/N] (#50089) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-07-29 14:10:58 +08:00
Andreas Karatzas 30fbd05537 [ROCm] Use backend-default dot precision for ReplaySSM (#49909) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-27 18:11:06 +08:00
Tianmu LiCodexLi, Jiang <jiang1.li@intel.com>
 		544cb724c8 [CPU][Spec Decode] Optimize GDN conv path for speculative decoding (#48577) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-07-27 06:20:14 +00:00
Johnny-LiouClaude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>tomeras91Cyrus Leung
 		866fea2b99 [Kernel] ReplaySSM: cache SSM inputs for faster Mamba2 standard decode (#48018) 
Signed-off-by: Johnny-Liou <a897111@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-07-24 09:39:49 -07:00
Akash kaothalkarAkash KaothalkarAkash KaothalkarAkash KaothalkarAkash kaothalkarLi, Jiang <jiang1.li@intel.com>
 		5c9f6557d7 [Hardware][CPU] Enable granite-4 model on cpu (#47641) 
Signed-off-by: Akash Kaothalkar <akashkaothalkar@akashs-mbp.bl1-in.ibm.com>
Signed-off-by: Akash Kaothalkar <akashkaothalkar@dhcp-9-123-5-76.bl1-in.ibm.com>
Signed-off-by: Akash Kaothalkar <akashkaothalkar@Akashs-MBP.lan>
Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akashkaothalkar@dhcp-9-123-5-76.bl1-in.ibm.com>
Co-authored-by: Akash Kaothalkar <akashkaothalkar@Akashs-MBP.lan>
Co-authored-by: Akash Kaothalkar <akashkaothalkar@akashs-mbp.bl1-in.ibm.com>
Co-authored-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-07-20 06:15:16 +00:00
Thien Tran fe784ff22e [M3] Improve indexer for long-context decode (sm100) (#48582) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-07-16 18:12:15 -07:00
Giuseppe Grossi d119beb1b9 [ROCm] Add tuned selective_state_update config for AMD MI350 (#48159) 
Signed-off-by: Giuseppe Grossi <ggrossi@amd.com>
 2026-07-15 10:18:09 +03:00
vanshbhatia-amdandvanshbhatia-amd 7ffb98e248 [ROCm] Retune MI355 selective_state_update float32 config on the unified effective_batch grid (#48373) 
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
 2026-07-14 18:26:35 +00:00
vanshbhatia-amdandvanshbhatia-amd 31be872f55 [ROCm] Retune MI355 selective_state_update float16 config on the unified effective_batch grid (#48372) 
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
 2026-07-14 05:16:29 +00:00
guybdCursorLi, Jiang <jiang1.li@intel.com>
 		05fa8183a6 [CPU][Spec Decode] Support DFlash speculative decoding for GDN models on CPU (#46090) 
Signed-off-by: guybd <guy.boudoukh@intel.com>
Signed-off-by: Guy Boudoukh <guy.boudoukh@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-07-13 04:16:18 +00:00
rasmith 934eeaecfb [CI/Build][BugFix][The Rock] Fix get_ssm_device_name to return sanitized, usable filename (#47781) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-07-08 12:12:54 +00:00
vanshbhatia-amdandvanshbhatia-amd db39d60010 Add tuned selective_state_update float32 config for AMD Instinct MI355 (#47943) 
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
 2026-07-08 11:41:26 +00:00
vanshbhatia-amdandvanshbhatia-amd eeaf23107f [ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI300X (#47947) 
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
 2026-07-08 11:09:02 +00:00
vanshbhatia-amdandvanshbhatia-amd 1f4ad059d1 [ROCm] Add tuned selective_state_update float16 config for AMD Instinct MI300X (#47945) 
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
 2026-07-08 11:03:58 +00:00
vanshbhatia-amdandvanshbhatia-amd c8c2f838e7 Add tuned selective_state_update config for AMD Instinct MI355 (#47767) 
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
 2026-07-07 22:19:08 +00:00
Thien Tran 43916891b2 [GDN] Improve kkt kernel of CuteDSL prefill backend (#46346) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-06-29 18:34:18 -07:00
MajidMajid Taheri Andanitomeras91
 		9af53a3c13 [Perf] Add tuned selective_state_update configs for H200 and RTX PRO … (#44251) 
Signed-off-by: Majid Taheri Andani <tahemaji@amazon.com>
Co-authored-by: Majid Taheri Andani <tahemaji@amazon.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
 2026-06-02 23:59:01 -07:00
Marceli FylcekandKunshang Ji f69ede495b [XPU][Mamba] Triton-based selective scan forward op for XPU (#43421) 
Signed-off-by: Marceli Fylcek <marceli.fylcek@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-06-02 03:50:26 -07:00
Fadi ArafehandLi, Jiang <jiang1.li@intel.com> 0b25cf4419 [CPU][Perf] Enable fused kernels for GDN's gated delta rules (#43534) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-06-02 08:00:48 +00:00
MajidandMajid Taheri Andani c08ebebf30 [Perf] Add do_not_specialize to Mamba SSD chunk kernels (#43803) 
Signed-off-by: Majid Taheri Andani <tahemaji@amazon.com>
Co-authored-by: Majid Taheri Andani <tahemaji@amazon.com>
 2026-05-28 15:40:02 +00:00
Thien Tran d56612c621 [GDN] GDN Prefill kernel for SM100 (#43273) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-05-26 14:02:11 +08:00
daniserebandBanani Ghosh d56285c747 Tuning script and configs for Triton Mamba SSU kernel (#43083) 
Signed-off-by: Banani Ghosh <bg2502@nyu.edu>
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
Co-authored-by: Banani Ghosh <bg2502@nyu.edu>
 2026-05-24 20:12:44 +03:00
Li, Jiang b4601ad43f [CPU] Add fused GDN support for AMX CPU platform (#42707) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-05-18 03:04:36 -07:00
wangxiyuan 5536fc0c01 [Misc] Replace mamba_type string literals with MambaAttentionBackendEnum (#41188) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-05-11 03:59:36 +00:00
Vensenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 215e2f7990 [Bugfix][Mamba] IMA in causal_conv1d kernel for long sequences (#41617) 
Signed-off-by: vensen <vensenmu@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-10 12:38:28 +00:00
Fadi ArafehandVadim Gimpelson 51f22dcfd0 [Feat][CPU] Enable Gated DeltaNet Attention (Qwen 3.5 / 3.6) (#41025) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-05-07 12:57:09 +08:00
roikoren755 b897f00c9c Gate SSU dispatch setup (#40039) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-04-16 13:06:01 -07:00
roikoren755 ecf5ff7ce3 [Mamba] Flashinfer selective_state_update (#36162) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-04-14 15:10:58 -04:00
Nicolò Lucchesi 66e86f1dbd [Kernel] Mamba support different layout for Conv state (#37416) 2026-04-03 01:50:09 +02:00
Jingu Kang f1ff50c86c [Bugfix] clamp dA_cumsum differences to prevent Inf in Mamba2 SSD kernels (#37501) 
Signed-off-by: Jingu Kang <jg.k@navercorp.com>
 2026-03-31 17:35:51 +02:00
SandishKumarHNLucas WilkinsonMatthew Bonanni
 		bcc6f67447 [Bugfix] Use null block (0) for padded block table entries (#35431) 
Signed-off-by: SandishKumarHN <sandish@fb.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-30 14:02:51 -07:00
roikoren755 8e6293e838 [Mamba] Add stochastic rounding support (#35753) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-03-30 12:33:49 -04:00
Benjamin Chislett 8a680463fa [Bugfix] Fix NemotronH MTP + Chunked Prefill (#35447) 2026-03-17 07:07:33 +01:00
Yan Ma 894843eb25 replace with torch.cuda.device with with torch.accelerator.device_index (#36144) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-03-11 23:12:57 -07:00