iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
97 Commits
Author SHA1 Message Date
Andreas Karatzas 096d8e8ce6 [ROCm][CI] Add MiniMax reduce RMS kernel coverage (#55057) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-03 04:07:59 -05:00
Divakar VermaandAndreas Karatzas c0adee9231 [ROCm][CI] Add ROCm misc ops and env tests (#53279) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-01 19:03:37 +00:00
Misha GoinandOpenAI Codex 82b7d49a6e [MoE] Generalize masked activation for padded layouts (#51217) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-01 11:34:13 -04:00
Jee Jee Li e2c8eeac40 [kernel] Fused embedding kernel (#53677) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-31 11:34:14 +00:00
canlahlahOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Yongye Zhu
 		06cccf8730 [Bugfix] Fix int32 token offset overflow in fused SiLU block quant (#53409) 
Signed-off-by: Anni <dranox2222@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-28 16:23:36 -04:00
Roy WangandClaude Fable 5 b74ac0bc61 [Bugfix] Allocate packed outputs in fused_q_kv_rmsnorm so q_b_proj keeps its low-latency GEMM path at decode (#53109) 
Signed-off-by: yasong.wang <yasong.wang@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-28 02:08:27 +00:00
Misha GoinandOpenAI Codex 5bbc58c0ad [Bugfix] Remove race in fused groupwise RMSNorm quantization (#54111) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-27 19:01:22 -07:00
Chauncey 7d5769b2d9 [Perf][DSv4] Use native CUDA SwiGLU clamp kernel for Humming MoE (throughput +1.40%) (#53685) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-08-27 13:52:45 -07:00
Shanshan Shen 32ad1400d7 [ROCm][Perf] Fuse SWA q/kv RMSNorm and q FP8 group quant for DeepSeek-V4 (#53540) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-08-27 09:37:54 -07:00
Divakar Verma 29b7c2f7d4 [ROCm][CI] aiter kernel ops - enable rope test (#52854) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-08-21 15:17:34 -05:00
Jee Jee Li 2687fec6ef Replicated embedding and norm fusion for DSV3 flat model (#48484) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-18 14:53:21 +08:00
Divakar Verma 49fb2ee348 [ROCm][CI] add Aiter ops tests (#52208) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-08-17 18:33:22 -05:00
bastefaniak 71b0da7c4e [Bugfix] Fix .../mrope.py::apply_interleaved_rope() when torch.compile is used in torch==2.13 (#52005) 
Signed-off-by: Bartosz Stefaniak <bstefaniak@nvidia.com>
 2026-08-13 22:08:40 +00:00
pmanczakandpmanczak 6b5bec7bed [Misc] Add and enable Triton kernel unit tests on XPU (#45694) 
Signed-off-by: pmanczak <pmanczak@users.noreply.github.com>
Signed-off-by: pmanczak <pawel.manczak@intel.com>
Co-authored-by: pmanczak <pmanczak@users.noreply.github.com>
 2026-08-07 02:53:03 -07:00
Rohan PotdarandClaude Opus 4.8 38ebd97bca [ROCm] Relax MLA rope+cache test tolerances for bf16 (#51083) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-05 17:50:52 -05:00
Andreas Karatzas bdc98bf506 [CI] Initialize fused gated RMSNorm weights (#50377) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 15:34:08 -04:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
Andreas KaratzasandOpenAI Codex 0c64be8873 [Test][ROCm] Account for gfx950 FP8 RMSNorm rounding (#49839) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-30 18:46:37 +08:00
yinfengLiu f1e8fd27ae [ROCm] Add AITER FP8 ViT encoder attention (#49937) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
 2026-07-30 06:22:10 +00:00
Wentao Ye 82642d7d6c [Perf] RMSNorm uncontiguous support, 1.2~3.1x kernel performance improvement (#49750) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-07-29 13:45:38 -07:00
+12
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodex
 		7c6729b769 [Model] Add Kimi K3 support: model files and kernels [1/N] (#50089) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-07-29 14:10:58 +08:00
Andreas Karatzas 05a0814863 [ROCm] Fix and optimize GPT-J-style MRoPE (#49906) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-28 10:50:10 -06:00
Andreas Karatzas e09900436c [CI][ROCm] Reduce kernel test runtime (#49915) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-27 06:44:21 +00:00
Wentao Ye 626c90b2d5 [Refactor] Move fla to third party (#48500) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-16 19:22:36 +01:00
Tanish MalekarandClaude Sonnet 4.6 e26264f3ef [Kernel] Implement CUDA kernel for ReLUSquaredActivation (relu^2) (#39058) 
Signed-off-by: Tanish Malekar <tanishmalekar32@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-07-12 19:18:03 -07:00
almayneandLi, Jiang <jiang1.li@intel.com> 89e99202f2 [CPU][Perf]Added tanh AOR for faster gelu activations. (#44639) 
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
Signed-off-by: almayne <anna.mayne@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-06-30 23:24:40 -07:00
Divakar Verma 68ee8300a0 [ROCm][CI]Fix test_concat_and_cache_mla_rope_fused on ROCm (#46409) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-06-27 12:38:13 +08:00
Divakar Verma 17a71d8702 [ROCm][CI] Relax fused layernorm quant test tolerances for one-ULP outliers (#46658) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-06-27 10:44:29 +08:00
Giancarlo Delfinandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 02a1f23711 [DFlash] Fuse precompute kv per-layer rmsnorms (#46761) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-25 19:32:07 -07:00
Divakar VermaAndreas KaratzasWentao Ye
 		84586c9acc [ROCm][CI] fix fp8 range in vit_fp8_quant (#46410) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Signed-off-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-06-23 11:34:21 -05:00
Jee Jee Li 745bba5ea8 [Model]Fix MiniMaxM2ForCausalLM perf regression (#45935) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-22 00:28:52 +08:00
arghyadeep sarkar 93bbe94d3a [Kernel] Add weightless RMSNorm CUDA kernels for has_weight=False (#41430) (#44109) 
Signed-off-by: hello-args <args.sarkar@gmail.com>
 2026-06-16 23:45:55 -07:00
youkaichaoOpenAI CodexIsotr0pyThien TranBugen ZhaoJee Jee LiRoger WangfunctionstackxYongye ZhuJee Jee Li
 		0a1c5034f5 [Model] Add MiniMax M3 support (#45381) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-16 01:01:25 +08:00
Jee Jee Li 6e503868ca [Kernel] Porting fuse_minimax_qk_norm to manual fusion (#43410) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-26 13:16:03 -07:00
Woosuk Kwon b14be81c1f [Model Refactoring] Move deepseek_v4_ops to models/deepseek_v4 [3/N] (#43073) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-05-19 00:52:54 -07:00
Chauncey 92879e12ba [CI] fix test_rotary_embedding_opcheck format error (#41202) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-29 00:32:37 -07:00
Jee Jee Li a269744e9f [Bugfix] Fix rope (#41113) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-28 22:42:35 -07:00
Yongye Zhu 706a04d34b [DSV4] Add silu clamp limit to shared expert (#40950) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-04-27 00:37:43 -07:00
Zhanda ZhuandYubo Gao 5d5c776444 [Perf] FP8 FlashInfer Attn for ViT (#38065) 
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com>
Co-authored-by: Yubo Gao <ybgao-nvidia@users.noreply.github.com>
 2026-04-27 13:44:15 +08:00
+7
Yifan QiaoYongye ZhuYongye ZhuSimon MoBugen ZhaoGiancarlo DelfinJee Jee LiNick HillRoger WangRoy WangWoosuk KwonyoukaichaoZhewen LiZijing LiukhluuqizixiZhewen Li
 		4d51588e23 [Feat] DeepSeek V4 Rebased (#40860) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
 2026-04-26 18:31:08 -07:00
Fadi Arafehandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 445b7093fd [perf][cpu] Accelerate BF16 GELU with LUT impl on Arm CPUs (#37469) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-15 22:26:17 -07:00
Jee Jee Li ecd1ea1363 [Kernel] Porting the TRTLLM minimax_allreduce_rms kernels (#37045) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-11 00:20:20 +08:00
Yan Ma ec68d53b2b Add platform manual_seed_all API (#38468) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-04-10 13:43:50 +08:00
Gregory Shtrasberg 56c976c1b5 [ROCm] Enable fused_silu_mul_block_quant on ROCm (#38817) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2026-04-08 11:23:32 -05:00
Jiangyun ZhuandLuka Govedič 8060bb0333 [vLLM IR] rework gemma_rms_norm (#39014) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-04-07 01:37:00 -07:00
Micah Williamson 9570654c6d [ROCm][CI] Run Kernels Core Operation Test On MI325 and mitigate flakiness (#38184) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-04-06 09:42:02 +08:00
Monishver c09ad767cd Feature/silu block quant fusion v1 (#32996) 
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
 2026-04-01 18:50:43 +00:00
Zhanda Zhu c75a313824 [Perf] triton bilinear_pos_embed kernel for ViT (#37948) 
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com>
 2026-04-01 01:52:02 -07:00
Krish Gupta c0f011918d [Bugfix] opcheck false mutation error in rms_norm_per_block_quant (#36688) (#36779) 
Signed-off-by: Krish Gupta <krishom70@gmail.com>
 2026-03-16 21:11:33 +00:00
Kunshang Ji 53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00