iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
141 Commits
Author SHA1 Message Date
aoshen02andaoshen02 69cf055936 [Performance][DSv4] Size dequant gather launch grid by rows (#55061) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen524@gmail.com>
 2026-09-03 20:06:34 -07:00
elehayymelehayymKevin Luu
 		fc8f10792c Fix DeepSeek V4 FlashMLA auto KV cache dtype (#45091) 
Signed-off-by: elehayym <huangxh1618@gmail.com>
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: elehayym <huangxh1618@gmail.com>
Co-authored-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-09-03 19:29:14 +00:00
Isotr0py 1356635d83 [New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support (#54566) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-02 11:09:09 -07:00
Isotr0py 1d8d7a3965 [Bugfix] Fix RoPE construction for deepseek-v4 sparse SWA layers (#54815) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-01 20:48:50 -07:00
Ilya Markov 25efcfa788 [Attention] Enable adaptive verification for FLASHINFER_MLA_SPARSE_DSV4 (#52724) 
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
 2026-09-01 13:43:28 +00:00
Roberto L. CastroandCodex e16b5e518d [1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels (#50175) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-31 23:26:05 -04:00
Gabriel WuWentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		699e180df4 [Bugfix][SM120] DSv4: pass contiguous C128A decode topk indices on SM120 (#53574) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-31 09:44:42 +00:00
Shanshan Shen 32ad1400d7 [ROCm][Perf] Fuse SWA q/kv RMSNorm and q FP8 group quant for DeepSeek-V4 (#53540) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-08-27 09:37:54 -07:00
Woosuk Kwon 0a5ad6f0d4 [Model] Remove unused DeepSeek V4 top-k buffer helper (#53697) 2026-08-26 11:59:09 -07:00
Fangzhou Ai 657f9b9ce2 [ROCm][DSV4][Perf] Fuse DeepSeek V4 C4 compressor GEMMs (#53838) 
Signed-off-by: fai <fangzhouai@gmail.com>
 2026-08-26 23:36:14 +08:00
Md Saidul Hoque AnikClaude Fable 5Anerudhan Gopal
 		8fe9317f2e [Model][MoE] DeepSeek-V4: add opt-in FlashInfer moe_ep expert backend (#49636) 
Signed-off-by: Md Saidul Hoque Anik <mhoqueanik@nvidia.com>
Signed-off-by: Anerudhan Gopal <agopal@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Anerudhan Gopal <agopal@nvidia.com>
 2026-08-25 03:35:01 +00:00
Giancarlo Delfin d154d90d6c [Model Runner V2] batch-sharded sample (#50465) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-08-24 15:49:57 -07:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟andJee Jee Li 702e1d7186 [LoRA] feat: Support LoRA for DeepSeek V4 (#53361) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-24 03:57:57 +00:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Wentao Ye e6f35d3c69 [DSv4 Perf] Adaptive topk width for dsv4, making #50004 back (#52823) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-21 12:11:45 -07:00
Fangzhou Ai fe76112ff2 [ROCm][Perf] Optimize DeepSeek V4 C4A top-k with AITER (#52882) 2026-08-21 10:00:39 -07:00
Canlin Guo 4f6885fffc [DSV4][Kernel] Fuse shared experts into MegaMoE (#53040) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-08-20 12:07:02 -07:00
Harry Mellor 1fe3a1571a Reduce AutoWeightsLoader kwargs (#53106) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 15:40:11 +00:00
Andreas Karatzas 6b68db441e [ROCm] Fix DeepSeek V4 indexer numerics and coverage (#50803) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-20 09:34:00 +00:00
Shanshan Shen d626108b18 [ROCm][Perf] Fuse DeepSeek-V4 mHC post/pre and RMSNorm with AITER (#52737) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-08-20 04:51:04 +00:00
Woosuk KwonandOpenAI Codex f1178f3a06 Revert DSv4 eager workspace reuse (#52836) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-18 17:02:59 -07:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 a9f4afb66f [Bugfix] Fix DeepSeek V4 mHC broadcast buffer for dummy load (#51368) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
 2026-08-18 17:02:18 -07:00
fxmarty-amd 88b2bff2c6 [MOE] Standardize and abstract fused shared expert optimization selection (#51695) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-18 12:56:56 -04:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 d5f5de7a7d [Bugfix] Fix DeepSeek V4 mHC broadcast buffer for weight sync (#52626) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
 2026-08-18 10:55:37 +08:00
Lucas WilkinsonOpenAI CodexYongye Zhu
 		292187dd8c [Bugfix][DSv4] Keep indexer scoring in breakable graphs (#52492) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-17 03:01:13 +00:00
Yongye ZhuandClaude Opus 5 967e104fad [Config] Unify indexer cache dtype under attention_config.indexer_kv_dtype (#52550) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-17 10:20:04 +08:00
Fangzhou AiNick HillCursor Grok 4.6
 		ef43e3101b [ROCm][DSV4][Perf] Optimize Triton sparse-MLA decode on gfx950 (#52212) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Cursor Grok 4.6 <cursoragent@cursor.com>
 2026-08-16 12:16:46 -07:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 83f591d7f6 [Perf][DSV4] Optimize global top-k index kernel with compile-time constants (#51967) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-16 23:24:14 +08:00
Chauncey 836aac92ff [Perf][DSV4] Optimize sparse top-k metadata kernels for higher prefill throughput (#52084) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-08-16 21:15:47 +08:00
Nick Hill 8efa13b700 [Bugfix] Pick the DeepSeek V4 eager cudagraph region per model runner (#52401) 2026-08-15 21:04:08 -07:00
Toby MaoClaudeRoger WangYongye ZhuOpenAI Codex
 		edd4c8176c [Bugfix][DSv4] Revert adaptive C128A metadata packing (#51318) 
Signed-off-by: tobymao <toby.mao@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-15 18:39:23 -07:00
Gabriel Wu 97388c44f9 [Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538) 2026-08-15 07:01:32 -07:00
Lucas WilkinsonandClaude 57bd0ed441 [5/N][KV-Cache Layout Refactor] Backend-published KV packing via customize_spec (#51704) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-14 09:47:55 -04:00
fxmarty-amd 48825acc23 [Quantization] Remove dead QuantizationConfig.is_mxfp4_quant (#51793) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-13 13:58:31 -04:00
stefankoncarevicandAndreas Karatzas b369f10d5c [Bugfix][ROCm][CI] Restore the DeepSeek-V4 input GEMM override point (#51821) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-12 20:33:52 -05:00
Lucas WilkinsonOpenAI CodexClaude Opus 5Benjamin Chislettmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick HillOpenAI Codex
 		7f7a32cfec [Spec Decode] DSpark confidence-scheduled verification (#47808) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-12 12:27:48 -07:00
aoshen02 4eef91c03c [Model] Support R3 capture with DeepGEMM MegaMoE (#51831) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-08-12 01:07:07 -07:00
Wentao Ye fd04edef3e [Refactor] Delete dead code in models (#51838) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-11 15:44:26 -04:00
Tuukka Sarvi 12bea3eedc [Bugfix][ROCm] Fix DeepSeek V4 DSpark probabilistic startup (#51145) 
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
 2026-08-11 22:15:31 +08:00
Yizhou a311916a29 [MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph (#46849) 
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
 2026-08-11 15:06:22 +08:00
xijiaatWentao YeOpenAI Codex
 		d40c3e3c00 Fix DSpark warmup without sparse index buffer (#50693) 
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com>
Signed-off-by: xijiaat <52128022+xijiaat@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-10 15:12:19 -04:00
Woosuk KwonandOpenAI Codex 79c865b838 [Perf] Narrow DeepSeek V4 eager CUDA graph region (#51430) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-10 10:52:10 -07:00
jimmy-adamsAndreas KaratzashujiCursor
 		da788334bc Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel (#47972) 
Signed-off-by: Jimmy Adams <Adam_ji@outlook.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: huji <huidong.ji@amd.com>
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: huji <huidong.ji@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-07 01:19:31 -07:00
Woosuk Kwon 38a466e7b6 [DSV4] Implement Sequence Parallelism (#46789) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-01 10:22:33 -07:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> df71917cf1 [DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 08:45:58 -07:00
bnellnmandHarry Mellor 6e311c6e20 [MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-31 11:28:18 +01:00
Jeff (Junze) MaClaude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Yongye Zhu
 		b49eaf205a [DSv4] Remove sparse-MLA q-head padding for FlashInfer >=0.6.14 (#48047) 
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-07-31 03:30:21 +00:00
Mehmet CagriClaude Opus 4.8Andreas Karatzas
 		12a34a6bc7 [ROCm][DSV4] B-preshuffle the attention fp8 projections (#46720) 
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 13:59:52 -05:00
Wentao Ye 837eae6458 [DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement (#50298) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-30 11:39:07 -04:00
Wentao Ye 904fae8be1 [DSv4 Perf] Fix redundant memory allocation and copy for dsv4 pp buffer, 448 MiB GPU memory saved (#50312) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-30 11:38:51 -04:00