iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
7,797 Commits 3 Branches 0 Tags
4adc66f64d56338489d00d94de6e13d95741c4be
Commit Graph
382 Commits
Author SHA1 Message Date
Woosuk Kwon 4de7146351 [V0 deprecation] Remove V0 HPU backend (#21131) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-17 16:37:36 -07:00
Peter Pan 1eb2b9c102 [CI] update typos config for CI pre-commit and fix some spells (#20919) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
 2025-07-15 21:12:40 -07:00
Li Wang 20149d84d9 [MISC] Add init files for python package (#20908) 
Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-07-15 12:16:33 +00:00
Cyrus Leung e8cc53af5e [Misc] Log the reason for falling back to FlexAttention (#20699) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-14 04:16:51 -07:00
Wentao Ye c1acd6d7d4 [Refactor] Change the way of import triton (#20774) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-12 19:39:55 -07:00
Congcong Chen 2c11a738b3 [Model] New model support for microsoft/Phi-4-mini-flash-reasoning (#20702) 
Signed-off-by: Congcong Chen <congcongchen@microsoft.com>
 2025-07-12 06:02:10 -07:00
Pavani Majetyshuwmgoin
 		7bd4c37ae7 [Core] Add Flashinfer TRTLLM Backend for Flashinfer decode path (SM100). (#19825) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: shuw <shuw@nvidia.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 09:23:23 +00:00
Luka Govedičandmgoin 31d5c1797f [Perf][fp8] Use CustomOp abstraction for fp8 quant for better perf (#19830) 
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 04:56:28 +00:00
Alexander Matveev 5b032352cc [Attention] MLA - Flashinfer Ragged Prefill (#20034) 2025-07-10 20:17:47 -07:00
Li, Jiang 7721ef1786 [CI/Build][CPU] Fix CPU CI and remove all CPU V0 files (#20560) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-07 22:13:44 -07:00
jvlunteren 22dd9c2730 [Kernel] Optimize Prefill Attention in Unified Triton Attention Kernel (#20308) 
Signed-off-by: Jan van Lunteren <jvl@zurich.ibm.com>
 2025-07-07 19:08:12 +00:00
Cyrus Leung 9fb52e523a [V1] Support any head size for FlexAttention backend (#20467) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-06 09:54:36 -07:00
Woosuk KwonandLi, Jiang <jiang1.li@intel.com> e202dd2736 [V0 deprecation] Remove V0 CPU/XPU/TPU backends (#20412) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2025-07-06 08:48:13 -07:00
Chengji Yao 7da296be04 [TPU] kv cache update kernel supports dynamic grid (#20235) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-07-02 06:33:37 +00:00
Chendi.Xue dec197e3e5 Quick Fix by adding conditional import for flash_attn_varlen_func in flash_attn (#20143) 
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
 2025-06-27 05:48:13 +00:00
Chengji Yao 04e1642e32 [TPU] add kv cache update kernel (#19928) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-06-26 10:01:37 -07:00
Kunshang Ji b69781f107 [Hardware][Intel GPU] Add v1 Intel GPU support with Flash attention backend. (#19560) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-06-26 09:27:18 -07:00
TJian 27c065df50 [Bugfix][V1][ROCm] Fix AITER Flash Attention Backend (Fix API Break and Local Attention Logic: affecting Llama4) (#19904) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-06-26 12:42:31 +00:00
Wentao Ye 879f69bed3 [Refactor] Remove duplicate ceil_div (#20023) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-25 05:19:09 +00:00
Ning Xie 71baf85ae1 [Kernel] mark TorchSDPABackend swap_blocks NotImplementedError (#19749) 2025-06-20 18:18:11 +00:00
Ning Xie 71d1219545 [Kernel] correct cpu worker function parameter type (#19745) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-20 10:50:13 +00:00
Woosuk Kwon f04d604567 [Minor] Zero-initialize attn output buffer (#19784) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-18 06:59:27 +00:00
Ning Xie c53711bd63 [MISC] correct copy_blocks src_to_dists param type (#19696) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-17 17:21:06 -07:00
Nicolò Lucchesi 4c8f64faa7 [V1][Kernel] Flashinfer HND KV cache layout (#19280) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-17 09:09:22 -04:00
jvlunteren ccd7c05089 [Kernel] Add Split-KV Support to Unified Triton Attention Kernel (#19152) 
Signed-off-by: Jan van Lunteren <jvl@zurich.ibm.com>
 2025-06-17 10:45:07 +00:00
22quinn 0b73736a0d [Kernel] Raise verbose error and consolidate num_heads/num_kv_heads divisibility check (#19339) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-15 13:43:48 +08:00
Luka GovedičandSage Moore f98548b9da [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
 2025-06-12 08:31:04 -07:00
Ning Xie 2f1c19b245 [CI] change spell checker from codespell to typos (#18711) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-11 19:57:10 -07:00
rasmith c7ea0b56cd [AMD] [Quantization] Add override flag for attention dtype instead of using kv_cache_dtype trigger (#17331) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-06-11 15:53:28 -04:00
Jee Jee Li 04a55612dd [Misc] Fix misleading ROCm warning (#19486) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-12 00:12:10 +08:00
Li, Jiang 4555143ea7 [CPU] V1 support for the CPU backend (#16441) 2025-06-03 18:43:01 -07:00
Yong Hoon Shin bdf13965ab [V1] Support cross-layer KV sharing (#18212) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-06-03 20:33:07 +00:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
vllmellm 77b6e74fe2 [ROCm] Remove unnecessary assertion of max_model_len in ROCM_AITER_MLA attention backend. (#18938) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-05-29 22:33:17 -07:00
Gregory Shtrasberg 1b7cfd5a36 [ROCm][V0][Attention] Revert to the previous FA triton kernel (#18226) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-05-29 12:13:18 -04:00
Gregory Shtrasberg da4b69d0b4 [Attention][V1] Toggle for v1 attention backend (#18275) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-05-29 10:48:24 -04:00
Hongxia Yangandtjtanaa 269d901734 [Bugfix][ROCm] fix the power of 2 exception from triton_unified_attention.py when running llama4 models and unit test fix (#18100) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-05-29 07:21:46 +08:00
Lucas Wilkinson ce75efeecb [BugFix] FA2 MLA Accuracy Issue (#18807) 
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>
 2025-05-28 08:59:39 +00:00
Hosang dd5fa7e04f [ROCm][Kernel][V1] Enable AMD Radeon GPU Custom Paged Attention on v1 (#17004) 
Signed-off-by: Hosang Yoon <hosang.yoon@amd.com>
 2025-05-21 08:35:00 -07:00
Percy 980a172474 [Kernel] update comment for KV shape in unified triton attn (#18099) 
Signed-off-by: haochengxia <xhc_1007@163.com>
 2025-05-20 11:19:34 -07:00
Thomas ParnellandLucas Wilkinson 01c22335ba [Kernel] [V1] Fix performance regression for triton unified attention (#18161) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-15 06:39:00 -07:00
Thomas ParnellandLucas Wilkinson e6b8e65d2d [Bugfix] Fix fp8 tests for triton_unified_attention for Triton 3.3 (#18013) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-15 13:26:34 +08:00
qli88 4f8b373225 [BugFix][AMD] Compatible patch for AITER lib after 04/20 (#17912) 
Signed-off-by: Qiang Li <qiang.li2@amd.com>
 2025-05-13 23:05:20 -07:00
Luka Govedič 176a95c670 [Fix] Support CUDAGraph capture for encoder-decoder on ROCm (#18104) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2025-05-13 19:31:42 -07:00
Tao He 60f7624334 Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844) 2025-05-12 19:52:47 -07:00
Gregory Shtrasberg 06c0922a69 [FP8][ROCm][Attention] Enable FP8 KV cache on ROCm for V1 (#17870) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-05-11 15:58:45 +08:00
Shiyan Deng eea22a56ab fix amd triton mla path (#17871) 2025-05-11 07:53:31 +00:00
Michael Goin 85b72cb7b1 Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910) 2025-05-09 08:58:18 -07:00
qli88 9f64e93415 [BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864) 
Signed-off-by: Qiang Li <qiang.li2@amd.com>
 2025-05-09 08:59:36 -06:00
Lucas Wilkinson 5e6f939484 [Attention] MLA move rotary embedding to cuda-graph region (#17668) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-09 11:14:42 +08:00