iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,220 Commits 3 Branches 0 Tags
00f7f258282ad3ea400a760ba3bc4679ef4e40fe
Commit Graph
664 Commits
Author SHA1 Message Date
vanshbhatia-amdandVansh Bhatia d591d1d511 [Bugfix] Add Kimi K3 MoE support to benchmark_moe.py (#50082) 
Signed-off-by: Vansh Bhatia <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: Vansh Bhatia <210711135+vanshbhatia-amd@users.noreply.github.com>
 2026-08-19 19:42:30 +00:00
Do_it_now_!andDo_it_now_! 101c4477dd [Bugfix] Handle DeepseekV4ForCausalLM in benchmark_moe get_model_params (#52044) 
Co-authored-by: Do_it_now_! <23432123@users.noreply.github.com>
 2026-08-18 04:46:08 +00:00
Thien Tran 6014f9e672 [Kimi-K3] Add GEMM-RS for sequence parallelism (#52079) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-08-13 09:02:28 -07:00
pmanczak c4e969294e [XPU] Add tuned Mamba SSU configs for Intel Arc Pro B70 (#50534) 
Signed-off-by: pmanczak <pawel.manczak@intel.com>
 2026-08-13 12:36:55 +00:00
kliuaeClaudeTJian
 		7f9173dfa2 [ROCm][Perf] Kimi-K3 Fused kernel for KDA decode (#50654) 
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-08-11 20:39:34 -07:00
Yongye ZhuOpenAI CodexClaude Opus 5
 		c76a425278 [Kernel] Optimize long-context MLA cache gathers (#51739) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-10 22:23:43 -07:00
ChaunceyandNick Hill 72c0d67657 [Feat] Support thinking_token_budget in Model Runner V2 (#46727) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 03:36:18 +00:00
zexplorerhj d4da0c55af [Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support (#51045) 
Signed-off-by: zexplorerhj <zhjoneson@163.com>
 2026-08-06 00:38:28 +08:00
xiaozhoupyandClaude 3756bf1e2b [Kernel][SM100] Add a CuTeDSL fused query kernel (#49792) 
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-04 15:23:06 -07:00
Matthew BonanniandOpenAI Codex 82ae4164ee [2/N][Attention] Enable masked MHA for sparse MLA prefills (#48770) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 06:01:24 -07:00
yinfengLiu f1e8fd27ae [ROCm] Add AITER FP8 ViT encoder attention (#49937) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
 2026-07-30 06:22:10 +00:00
+12
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodex
 		7c6729b769 [Model] Add Kimi K3 support: model files and kernels [1/N] (#50089) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-07-29 14:10:58 +08:00
Matthew Bonanniandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> bf2b45b5d6 [Attention] Integrate FlashAttention 4 SM100 headdim 256 support (#42669) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-07-27 18:25:50 +00:00
TJianandOpenAI Codex 0906123953 [ROCm] [Model] Enable TML inkling (#48841) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-27 10:05:46 +00:00
Akash kaothalkarandAkash kaothalkar f19ee27e39 [Hardware][Power] Add FAST_EXP for Power (#49571) 
Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
 2026-07-27 05:47:29 +00:00
Johnny-LiouClaude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>tomeras91Cyrus Leung
 		866fea2b99 [Kernel] ReplaySSM: cache SSM inputs for faster Mamba2 standard decode (#48018) 
Signed-off-by: Johnny-Liou <a897111@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-07-24 09:39:49 -07:00
Matthew BonanniOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		67f9046e4a [Bugfix] Sparse MLA: enable fp8_ds_mla dense prefill (#48642) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-16 22:44:04 +00:00
Matthias GehreandAndreas Karatzas 382bbd5144 [ROCm][Kernel] Add HybridW4A16LinearKernel: Triton prefill + HIP skinny decode (#40977) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-13 20:22:00 -07:00
Matthew BonanniandClaude Opus 4.8 c4f5cd60da [1/N] Add dense MHA path for sparse MLA short sequences (#47327) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-14 00:29:56 +00:00
Tanish MalekarandClaude Sonnet 4.6 e26264f3ef [Kernel] Implement CUDA kernel for ReLUSquaredActivation (relu^2) (#39058) 
Signed-off-by: Tanish Malekar <tanishmalekar32@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-07-12 19:18:03 -07:00
rasmith 934eeaecfb [CI/Build][BugFix][The Rock] Fix get_ssm_device_name to return sanitized, usable filename (#47781) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-07-08 12:12:54 +00:00
Harry Mellor dd94484577 Bump Transformers version to 5.10.4 (#41359) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-07 05:13:28 -07:00
Kunshang JiandHarry Mellor 07f9baf756 Revert "[Platform] Replace torch.cuda.Event with torch.Event (#47140)" (#47668) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-06 14:18:33 +01:00
Michael Goin d715b3aa1e Delete PagedAttention (#47361) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-02 12:31:26 -07:00
Kunshang JiandHarry Mellor e840f0d3f5 [Platform] Replace torch.cuda.Event with torch.Event (#47140) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 08:39:59 -07:00
Michael Goin e8e7b592d1 [Kernel][MoE] Tune block-FP8 fused MoE for low-batch decode (#46642) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-06-25 12:38:28 -07:00
Jeff (Junze) Maandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> fc61c6fc26 [Perf] Enable + tune FlashInfer fused allreduce at world_size=16 on SM 10.3 (GB300) (#46392) 
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-24 23:04:17 -07:00
Fadi Arafeh 061043eaca [CPU][Perf] Accelerate unquantized MoE for AArch64 (#46353) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-06-24 14:14:35 +00:00
Carl Y eb04c769d3 feat: MLA prefill enable FA4 fp8 output (#43050) 
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
 2026-06-16 07:10:59 -07:00
Jimmy Lee 8bf374955f [Bug Fix] Allow pinned memory for WSL2 (#41496) 
Signed-off-by: Jimmy Lee <hirejimmylee@gmail.com>
 2026-06-16 05:56:26 +00:00
Amanzhol Salykov 725c3bc808 [ROCm][Perf] Enable W4A16 FlyDSL MoE (#44400) 
Signed-off-by: amd-asalykov <asalykov@amd.com>
Signed-off-by: Amanzhol Salykov <asalykov@amd.com>
 2026-06-14 00:14:39 -07:00
Matthew Bonanni cf567cbc71 [Attention] Improve attention benchmarks: configs and profiling (#39336) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-06-12 16:24:25 -04:00
Lucas WilkinsonMartin KuklaMatthew BonanniDipika SikkaNickLucchejiahancAlec KohlhoffPorras HuangClaudescoootscooob
 		eb28452b10 [Model] Add DiffusionGemma Support (#45163) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Martin Kukla <martin.kukla@cantab.net>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Dipika Sikka <dsikka@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com>
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com>
 2026-06-11 22:17:35 -07:00
Nick HillandClaude Opus 4.8 4085ff7cb4 [Core] Add kvcache watermark to reduce preemptions (#44594) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-11 08:27:31 -07:00
Fynn Schmitt-Ulmsandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ebc6ef971a Hidden states extraction improvements (#43805) 
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-11 09:44:45 -04:00
Change72 320c52b134 [Bench] benchmark_serving_multi_turn: make non-standard conversation_id payload opt-in (#43756) 
Signed-off-by: Change72 <cguo51@asu.edu>
 2026-06-09 19:41:56 -07:00
Jimmymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>simon-mo
 		1c2ffc6f88 feat(multi-turn-bench): add api_key and custom headers for multi turn benchmark (#44516) 
Signed-off-by: Jimmy <jinmingyi1998@sina.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
 2026-06-09 14:00:07 -07:00
Tyko Niemi cad4ca12b8 [Bugfix] Add X-Session-ID from conversation_id in multi-turn benchmark (#44663) 
Signed-off-by: Tyko Niemi <tyko.niemi@amd.com>
 2026-06-09 08:57:00 -07:00
bnellnm dc68bd8c41 [MoE Refactor] FusedMoE/MoERunner inversion refactor (#41184) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-06-08 10:42:58 -04:00
Nicolò Lucchesi 5add018beb [Connector] Remove P2pNcclConnector (#44854) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-06-08 18:58:29 +08:00
Qiuyang Yue 9c7f7741d4 [Bugfix] Fix benchmark_moe.py after inplace mechanism removal (#44041) 
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>
 2026-06-07 00:32:00 -04:00
Jee Jee LiandYiliu Dong 559d6710bf [PERF]MiniMax-M2 gate kernel (#38445) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: qianlihuang <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
 2026-05-29 18:28:34 -07:00
Wentao Ye 64e1218673 [Perf] Optimize moe permute by pre-allocate buffer, 9~14% kernel performance improvement (#43014) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-28 06:18:26 -07:00
Jee Jee Li ec5de7fa7d [LoRA] Add one shot triton kernel For MoE LoRA (#42290) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-05-25 19:47:04 -07:00
Jee Jee Li d4004455d2 [Kernel] Remove NormGateLinear (#43554) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-25 09:49:19 +00:00
daniserebandBanani Ghosh d56285c747 Tuning script and configs for Triton Mamba SSU kernel (#43083) 
Signed-off-by: Banani Ghosh <bg2502@nyu.edu>
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
Co-authored-by: Banani Ghosh <bg2502@nyu.edu>
 2026-05-24 20:12:44 +03:00
Benjamin Chislettandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 4e2eba28be [Perf] Optimize hidden state extraction logic (#37374) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-22 18:23:08 -04:00
Viktor Pusandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 87a2adcb43 [Misc] Add common random prefix option to structured-output serving benchmark (#41632) 
Signed-off-by: Viktor Pus <viktorpus@tenstorrent.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-16 00:44:48 +00:00
lyd1992andClaude f351455f0f [CPU][RISC-V] Add RVV-optimized attention kernels for RISC-V Vector Extension (#40119) 
Signed-off-by: liuyudong <liuyudong@iscas.ac.cn>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-15 12:08:23 +08:00
Matthew Bonanni 9898f94abe [Attention] Remove deprecated MLA prefill arguments (#42555) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-14 10:34:06 -07:00