iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
12,946 Commits 3 Branches 0 Tags
eb28e8068dd870283f1a4fe8251ed9254d6f477e
Commit Graph
104 Commits
Author SHA1 Message Date
Andreas Karatzas 11b6af5280 [ROCm][Bugfix] Fix Mamba batched decode producing incorrect output (#32099) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-01-13 05:46:53 +00:00
Matthew Bonanni 2612ba9285 [1/N][Attention] Restructure attention: move files (#31916) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-09 13:10:24 -08:00
Shanshan Shen 08d954f036 [Doc] Add developer guide for CustomOp (#30886) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-01-09 16:21:11 +00:00
Cyrus Leung b665bbc2d4 [Chore] Migrate V0 attention utils (#31891) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-07 13:44:36 +00:00
Wang Kunpeng 5708297e4e [Misc][Model][Refactor] Pass the prefix into Linear layers (#31669) 
Signed-off-by: Wang Kunpeng <1289706727@qq.com>
 2026-01-05 20:03:18 +00:00
Roger Youngandxuebi 5bc664110f Optimize QKNorm for MiniMax-M2/M2.1 (#31493) 
Signed-off-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: xuebi <xuebi@minimaxi.com>
 2025-12-29 16:30:18 +00:00
Asaf Joseph Gardin 34916ae37f [Mamba] - Consolidate Mambas Attention Logic (#28133) 2025-12-23 21:57:00 +01:00
roikoren755 ae0f69b16a Add SpecDec support to selective_state_update (#29488) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2025-12-08 16:45:18 -05:00
Matthew Bonanni 1d93f11675 [Attention][CUDAGraph] Remove CG padding from attention backends (#29352) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-12-02 13:48:08 -05:00
Matthew Bonanni fc1d8be3dc [Attention] Update attention imports (#29540) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-27 11:19:09 -05:00
Matthew Bonanni 430dd4d9eb [Attention] Remove imports from vllm/attention/__init__.py (#29342) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-26 10:53:15 -07:00
Cyrus Leung aab0102a26 [V0 deprecation] Remove more V0 references (#29088) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 11:56:59 +00:00
Shanshan Shen d44e9df7d4 [Model][Mamba] Add selector for mamba attention backend and make it pluggable for other device (#26487) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2025-11-19 16:24:55 +00:00
tomeras91 1395461f5f [Hybrid][torch.compile] Refactor mamba2 forward to avoid obscuring linear projections under custom op (#28587) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
 2025-11-18 16:49:36 -08:00
Thomas Parnell e0c910bb89 [Hybrid] [Kernel] Fix chunk scan kernel when BLOCK_SIZE_DSTATE > 128 (#28295) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-11-14 22:55:42 +00:00
Asaf Joseph Gardin 00b31a36a2 [V1] [Hybrid] Mamba1 Automatic Prefix Caching (#26377) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-11-02 04:16:23 -08:00
Zhiyuan Li 4e68cc9b6a [Model] Introduce Kimi Linear to vLLM (#27809) 
Signed-off-by: lizhiyuan <lizhiyuan@moonshot.cn>
Signed-off-by: Zhiyuan Li <uniartisan2017@gmail.com>
 2025-10-30 21:02:27 +08:00
Roger Youngandxuebi d6704dd099 Fix MiniMax-M2 rmsnorm precision and remove useless code (#27627) 
Signed-off-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: xuebi <xuebi@minimaxi.com>
 2025-10-29 21:01:05 +08:00
Isotr0py 6ac5e06f7c [Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
Nicolò Lucchesi b26b70bec4 [Misc] Refactor get_kv_cache_spec into AttentionLayerBase (#26587) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-18 13:51:21 +00:00
Lukas Geiger 4d055ef465 Remove unused imports (#26972) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-10-16 19:51:17 -07:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor 2f99f2f506 Tidy vllm/config/__init__.py to only add classes and functions (#26405) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-08 07:10:00 -07:00
Thomas Parnell 778f554157 [V1] [Hybrid] Some additional clean-up in Mamba2 prefix caching (#26222) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-10-06 10:40:30 +08:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Stan WozniakThomas OrtnerThomas Parnell
 		ea507c3a93 [V1] [Hybrid] Mamba2 Automatic Prefix Caching (#25752) 
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Ortner <boh@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-10-04 06:34:22 +02:00
Paul Pak 5f42fc53b6 [backends][short_conv] CUDA graph piecewise edits (#24215) 
Signed-off-by: Paul Pak <paulpak58@gmail.com>
 2025-10-03 12:59:48 +00:00
Harry Mellor 61aedb5ffe MoveVllmConfig from config/__init__.py to config/vllm.py (#25271) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-29 19:49:49 -07:00
Thomas Parnell fea3e476aa [Kernel] Chunk-aligned mamba2 (#24683) 2025-09-29 23:18:25 +02:00
Chih-Chieh YangandRishiAstra 2b6b1d7809 [Model] Mamba2 varlen refactor (#21467) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: RishiAstra <40644327+RishiAstra@users.noreply.github.com>
 2025-09-26 11:31:14 +00:00
Tao He 99b3a504c5 [Qwen3-Next][GDN] fixes cuda graph capturing bug in GDN metadata and a stride bug in causal_conv_1d. (#25743) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-09-26 01:18:58 -07:00
Thomas ParnellandChih-Chieh-Yang 5e25b12236 [Kernel] [Mamba] Remove BLOCK_H=1 from list of tuneable configurations for _chunk_cumsum_fwd_kernel (#25197) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Chih-Chieh-Yang <chih.chieh.yang@ibm.com>
 2025-09-23 23:23:30 +00:00
Michael Goin 7361ab379f Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-23 22:48:40 +00:00
Thomas Parnell a903669e10 [V1] Remove V0 code paths for Hybrid models (#25400) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-09-23 08:26:13 -07:00
Woosuk Kwon bc6e542d9f Remove V0 attention backends (#25351) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 16:03:28 -07:00
Vadim Gimpelson 072d7e53e5 [PERF] Add conv1d metadata to GDN attn (#25105) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-09-18 14:27:49 +00:00
Asaf Joseph Gardin 66072b36db [Bugfix][Mamba] - Fix Conv State Kernel FP32 Support (#24883) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-09-18 12:21:17 +00:00
PunitvaraandHarry Mellor 05b044e698 [Doc] Fix cross-reference warnings (#25058) 
Signed-off-by: Punit Vara <punitvara@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 02:05:16 -07:00
Tao He dd6a910aac [Bugfix][Qwen3-Next] fixes the varlen issue in qwen3-next's MTP implementation. (#24957) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-09-17 21:59:09 +08:00
Chih-Chieh Yang 73cfb3c5ee [Model] Clean up and simplify Mamba2 Metadata Usage in both V0 and V1 (#24331) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
 2025-09-16 14:53:43 +00:00
tomeras91andgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 27fcfe7bcf [Mamba] Support TP>1 with quantization for mamba2 mixer in case n_groups % tp_size == 0 (#24593) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-16 10:51:01 +00:00
Didier Durand bcb06d7baf [Doc]: fix typos in various files (#24726) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-12 06:43:12 -07:00
Tao HeandCyrus Leung 880c741bb6 [Bugfix] fixes the causal_conv1d_update kernel update non-speculative decoding cases (#24680) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-11 18:16:43 -07:00
Tao HeandJee Jee Li e93f4cc9e3 Add the support for the qwen3 next model (a hybrid attention model). (#24526) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-11 15:32:09 +08:00
Didier Durand e2b1f863aa [Doc]: fixing doc typos (#24635) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-10 23:19:28 -07:00
tomeras91andCyrus Leung 08abfa78ec [Bugfix] fix modelopt exclude_modules name mapping (#24178) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-10 10:20:46 -07:00
tomeras91andgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> e041314184 [Bugfix] Fix mamba2 prefill chunking (#23279) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-08 11:42:41 +00:00
qscqesze 6997a25ac6 [Model] Remove useless code from MiniMax implementation (#23982) 
Signed-off-by: QscQ <qscqesze@gmail.com>
Signed-off-by: qingjun <qingjun@minimaxi.com>
 2025-09-03 11:27:04 +00:00
Asaf Joseph Gardin 2b41cbbf03 [V1][Mamba1] - FP32 SSM Kernel Support (#23506) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-09-01 20:53:00 -07:00
Thomas ParnellandCyrus Leung 4071c76cf3 [V1] [Hybrid] Move MiniMaxLinearAttention into layers/mamba (#23831) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-08-30 00:16:15 -07:00