iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
19,456 Commits 3 Branches 0 Tags
d83eb0b36bfbe26fc856ffdb60f05c8bd460f2fd
Commit Graph
161 Commits
Author SHA1 Message Date
Jee Jee Li 4635cc3e8f Shard the K3 Latent-MoE up-projection on large batches (#50383) 2026-08-03 00:49:39 -07:00
Jason Li 0055b8bfa3 [Attention][MiniMax-M3] Add MSA speculative decode verification (#50032) 2026-08-02 13:46:55 -07:00
Fangzhou AiandOpenAI Codex c666810676 [ROCm][Bugfix][Kimi-K3] Preserve MoE correction bias in FP32 (#50761) 
Signed-off-by: Fangzhou Ai <Fangzhou.Ai@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-02 20:30:54 +00:00
Woosuk Kwon 38a466e7b6 [DSV4] Implement Sequence Parallelism (#46789) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-01 10:22:33 -07:00
Andreas KaratzasandOpenAI Codex f7097a92fd [Bugfix][CI] Prevent common ops imports from initializing CUDA (#50639) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-01 00:13:29 -04:00
Kyle Sayers d87d2ca747 [Compressed-Tensors] Support Kimi-K3 quantized models (#50500) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-07-31 17:06:30 +00:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> df71917cf1 [DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 08:45:58 -07:00
Jee Jee Li 92643d68f5 K3 DSpark AR fusion (#50242) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-07-31 11:12:03 +00:00
bnellnmandHarry Mellor 6e311c6e20 [MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-31 11:28:18 +01:00
Thien TranandCodex 34bb795ff3 [CI] Add M3 MSA tests to CI (#49143) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-31 01:01:48 -07:00
Taneem IbrahimOpenAI CodexIsotr0py
 		f727951d3f [Bugfix] Re-land MiniMax M3 default video processor (#50305) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-31 07:46:18 +00:00
Jeff (Junze) MaClaude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Yongye Zhu
 		b49eaf205a [DSv4] Remove sparse-MLA q-head padding for FlashInfer >=0.6.14 (#48047) 
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-07-31 03:30:21 +00:00
Giancarlo Delfin dec13a33b7 [Model Runner V2][Spec Decode] Add multi-layer MTP speculator (#48892) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-07-30 15:54:15 -07:00
Mehmet CagriClaude Opus 4.8Andreas Karatzas
 		12a34a6bc7 [ROCm][DSV4] B-preshuffle the attention fp8 projections (#46720) 
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 13:59:52 -05:00
Wentao Ye 837eae6458 [DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement (#50298) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-30 11:39:07 -04:00
Wentao Ye 904fae8be1 [DSv4 Perf] Fix redundant memory allocation and copy for dsv4 pp buffer, 448 MiB GPU memory saved (#50312) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-30 11:38:51 -04:00
sroberts-amdElvir CrncevicAndreas Karatzas
 		e2efe79695 [ROCm]Migrating Deepseek V3.2 to vllm/models/deepseek_v32/ (#47207) 
Signed-off-by: sroberts-amd <sroberts@amd.com>
Signed-off-by: Stacy Roberts <sroberts@amd.com>
Co-authored-by: Elvir Crncevic <elvircrn@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 08:37:43 -05:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Nick Hill 2ecd8645d8 Revert "[Misc][Minimax-M3]add default video_processor (#50092)" (#50313) 2026-07-29 13:46:07 -07:00
stefankoncarevic 381b691620 [ROCm][CI] Fix Kimi K3 KDA on ROCm (#50262) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
 2026-07-29 15:47:50 +00:00
rongfu.lengandCyrus Leung df2735ea2e [Misc][Minimax-M3]add default video_processor (#50092) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-07-29 01:24:51 -07:00
+12
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodex
 		7c6729b769 [Model] Add Kimi K3 support: model files and kernels [1/N] (#50089) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-07-29 14:10:58 +08:00
Krishna Teja Chitty-VenkatamgoinOpenAI Codex
 		17a74b745b [Model] Add Inkling compressed-tensors dynamic FP8 support (#48876) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-28 19:27:01 -07:00
Yiliu DongandOpenAI Codex ba702e978e [Attention] Skip sparse indexer scoring for dense short prefills (#48407) 
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-28 16:17:22 +00:00
Harry Mellor 62d8db7c05 [Bugfix] Add missing vllm/models/kimi_k3/__init__.py (#50131) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-28 13:47:04 +00:00
Thien Tran 61ac368021 [Kimi-K3] Add AttnRes kernels (#50090) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-07-28 00:22:09 -07:00
Colin ZandAndreas Karatzas fbb1ef6803 [Bugfix] Fix DeepseekV4FP8 Quark MXFP4 crash on list-valued weight (#49634) 
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-27 22:32:42 -05:00
Wentao Ye b2f9e4caa4 [DSv4 Perf] Adaptive topk width, 1.0% E2E throughput improvement (#50004) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-27 15:56:52 -04:00
TJianandOpenAI Codex 0906123953 [ROCm] [Model] Enable TML inkling (#48841) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-27 10:05:46 +00:00
Wentao Ye 7fe6d3c76b [Perf] Fix moe reduce_scatter perf regression by removing additional comm, 5% E2E throughput gain back. (#48763) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-25 16:36:19 +00:00
rongfu.leng d1a8ba63d9 [Bugfix][MiniMax-M3] Fix token-major top-k buffer handling in Triton … (#49149) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-07-25 06:45:27 -07:00
Woosuk Kwon 213f681f81 Revert "[Perf][GLM-5.2] Blackwell decode optimizations" (#49768) 2026-07-24 16:04:59 -07:00
xiaozhoupyClaudeGiancarlo DelfinThien TranOpenAI Codexzhoustefankoncarevic
 		dd72658e7d [Perf][GLM-5.2] Blackwell decode optimizations (#48597) 
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Signed-off-by: zhou <zhou@zhoudeMacBook-Pro.local>
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zhou <zhou@zhoudeMacBook-Pro.local>
Co-authored-by: stefankoncarevic <Stefan.Koncarevic@amd.com>
 2026-07-23 21:36:27 -07:00
Kyle Sayers bf27e34ebb [CompressedTensors] DeepSeek4 CT Quantization Support (#41276) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-07-23 20:07:31 -07:00
Michael Goin d65acd83d8 [Model] Support llm-compressor Inkling NVFP4 weights (#49258) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-23 20:06:01 -07:00
Wentao Ye b0cb1da1bd [DSv4 Perf] Skip topk and router when not needed, 3.4% E2E TTFT improvement for Decode case (#49486) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-23 13:08:08 -04:00
Mike G 76bf55240c [Bugfix] Fix DeepSeek-V4 DSpark draft shared-expert padding for TP > 8 (#49415) 
Signed-off-by: Mike G <180722391+mikekg@users.noreply.github.com>
 2026-07-23 05:06:21 +00:00
Mike GMichael GschwindOpenAI Codex
 		9a698f3255 [Performance][Model] Avoid transient Inkling result allocations (performance, and OOM prevention on smaller memory configurations) (#49487) 
Signed-off-by: Michael Gschwind <mgschwind@nvidia.com>
Co-authored-by: Michael Gschwind <mgschwind@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-23 05:03:34 +00:00
Mike GMichael GschwindOpenAI Codex
 		4080263bb2 [Bugfix][Model] Remove SciPy dependency from Inkling scale planning (#49485) 
Signed-off-by: Michael Gschwind <mgschwind@nvidia.com>
Co-authored-by: Michael Gschwind <mgschwind@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-23 04:34:57 +00:00
Colin Zandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 27ffbfde8d Fused Shared Expert Support for AMD Quark DeepSeek-V4 Model Checkpoints (#48044) 
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-23 00:34:17 +00:00
Summer Yang f3a920a076 [Core][DSV4] Compact MXFP4 indexer KV cache and packed group overlays (#48993) 2026-07-22 16:58:44 -07:00
Wentao Ye 37e370fe93 [DSv4 Perf] Skip empty c128 kernel launch, around 2x kernel performance improvement. (#48957) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-22 10:55:20 -04:00
aoshen02andaoshen02 ba18929079 [Bugfix][SpecDecode] Scope MTP completeness checks outside bucketed updates (#49178) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-07-21 18:56:39 -07:00
Avery MiaoandKunshang Ji c67650f04b [XPU][DeepSeekV4]Add DeepSeek-V4 fuse_index_q SYCL kernel path (#45991) 
Signed-off-by: Avery Miao <avery.miao@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-07-21 21:22:34 +08:00
Roberto L. Castroandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 7ca017778f [Feat][Perf] Add new warmup infrastructure for JITs (#47451) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-20 13:21:55 -07:00
Xiaochang WuKunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		823eaf667d [XPU] FP8 o_proj with fp8_bmm and load-time scale transpose (#48334) 
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-20 16:32:03 +08:00
Michael Goin 02c01f442b [Model] Use standard ModelOpt config for Inkling NVFP4 (#48990) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-17 18:13:14 -07:00
Woosuk KwonBugen ZhaoGiancarlo DelfinIsotr0pyIsotr0pyJee Jee LiRoger WangYifan QiaoClaude Fable 5OpenAI Codex
 		f3e9497e92 [Model] Add Inkling LoRA support [4/N] (#48884) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-17 09:52:15 +08:00
Thien Tran fe784ff22e [M3] Improve indexer for long-context decode (sm100) (#48582) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-07-16 18:12:15 -07:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f17be06fbe [Perf] Optimize clamp to clamp_ (#48143) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-16 18:41:07 -04:00