iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,968 Commits 3 Branches 0 Tags
f83ea61cfe77f0b200f2ded1fda9be5ff591af2c
Commit Graph
452 Commits
Author SHA1 Message Date
Yongye ZhuandClaude Opus 5.5 378504a544 [MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-25 15:38:29 +08:00
Yongye ZhuandClaude Opus 5.5 5ff3bbfb08 [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 23:26:50 -07:00
Mathew Oddenopencode+deepseek-v4-flash+vllmAndreas KaratzasOpenAI Codex
 		267eee54bf [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497) 
Signed-off-by: Mathew Odden <modden@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-24 23:28:20 -05:00
Juntian LiuKimi CodeClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6936e77ba4 [Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:52:02 +08:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		48d8880d09 [Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool (#58489) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:51:50 +08:00
Juntian Liu 609731e8f5 [Perf][DSv4.1] Restore the fused query RMSNorm + MXFP8 quantization path (#57679) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
 2026-09-25 01:39:03 +00:00
Thien TranKimiIsotr0py
 		16070ed7e9 [Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope (#58526) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-24 18:30:32 -07:00
Yifan QiaoandClaude Fable 5.1 39724758e7 [Bugfix] GLM-5.3-Flash: launch the kpool paged MQA logits in the varlen mode its schedule was built with (#55270) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-25 00:13:55 +00:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
Fangzhou AiandClaude Opus 5.5 721d0e5c11 [ROCm][DSv4.1][Perf] Emit MXFP8 from the sparse decode reduce and run wo_a as a grouped FP8 GEMM (#58456) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 11:58:14 +00:00
Thien TranandOpenAI Codex 00b7847c80 [Perf] Use Conv3dLayer for MiniMax M3 patch embedding (#58512) 
Signed-off-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-24 00:41:36 -07:00
Wei ZhaoCodexYongye ZhuKevin H. Luu
 		44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-09-23 22:45:23 -07:00
Jhao-Ting Chen 4fb767ff0a [Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection (#53175) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
 2026-09-23 20:29:06 -07:00
Andreas Karatzas 56b3acb55b [ROCm][Bugfix] Keep zero MiniMax MXFP8 activation blocks finite (#58089) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-23 16:42:33 +08:00
aoshen02andYongye Zhu 973a3be780 [Refactor][Quantization] Make FP8 and MLA weight transforms reusable pure functions (#57732) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-09-23 01:16:20 -07:00
Albert Chengmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Benjamin Chislett
 		88aa0d287d [Spec decode] Support variable-length decode for Kimi-K3 adaptive ver (#52988) 
Signed-off-by: Albert Cheng <albecheng@nvidia.com>
Signed-off-by: Albert Cheng (Engrg-Hardware 1) <albecheng@login-bia01.bia.clusters.nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-09-22 20:43:32 -07:00
Fangzhou Ai f9dce295c9 [ROCm][DSv4][Perf] Fuse the inverse RoPE into the sparse decode reduce (#57451) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
 2026-09-23 01:23:55 +00:00
Jack HuJames E T SmithDouglas LehrDouglas Lehr
 		1c0eee919d [ROCm][Model][Bugfix] Enable GLM-5.2-MXFP4 on the deepseek_v32 path and fix sparse attention correctness (#51915) 
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Signed-off-by: Jack Hu <jack.hu@amd.com>
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: James E T Smith <jamesETsmith@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com>
 2026-09-22 18:08:51 -05:00
lucamotz 276db94fb2 [Bugfix][Model][Spec Decode] Defer disposable GLM MTP head (#55442) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
 2026-09-22 20:08:27 +00:00
Fangzhou AiCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ca831d1c55 [ROCm][DSv4.1][Perf] Fuse the inverse RoPE into the sparse decode reduce (#57435) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-22 18:05:11 +00:00
Juntian LiuClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		a6131b0e65 [Bugfix][Engram] Fall back when /dev/shm is absent before sharing tables (#57914) 
Signed-off-by: Juntian Liu <juntianl@inferact.ai>
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-23 00:19:09 +08:00
Jared WenClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c9b34fdb2d [Bugfix][GLM-5.3-Flash] Run the dense MLP layers on the sequence-parallel shard (#58061) 
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-22 21:58:21 +08:00
Canlin GuoThien TranOpenAI CodexClaude Fable 5.1Jee Jee Li
 		f92b78f6ef [Kernel][DSV4.1] Fuse MXFP8 wo_b GEMM with sequence-parallel reduce-scatter (#57428) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-22 01:07:17 -07:00
Shanshan Shen 42a85a4976 [ROCm][Bugfix] Explicitly reject FSE=1 with DPA+ETP deployment for DeepSeek-V4 (#57919) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-22 04:45:53 +00:00
yinfengLiuLiuyinfeng01Andreas Karatzas
 		382970ee6c [Kimi-K3][AMD] Return KDA and MLA projection outputs directly (#50592) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-21 15:44:41 -05:00
Tuukka Sarviandlarryli2-amd e34685dfc0 [ROCm][DSv4] Enable DSpark adaptive verification (#52362) 
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Signed-off-by: larryli2-amd <larryli2@amd.com>
Co-authored-by: larryli2-amd <larryli2@amd.com>
 2026-09-21 10:03:41 -05:00
Fangzhou AiandCursor 04c1f4a407 [Bugfix][ROCm][DSv4.1] Disable SWA bounded replay on ROCm (#57906) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-21 10:29:14 +00:00
weitliaoCursorShanshan Shen
 		b8cf275382 [AMD][Minimax-M3][perf] Enable packed LBHNC AITER QK-norm fusion for MiniMax-M3 on ROCm (#54535) 
Signed-off-by: Wei-Ting Liao <wei-ting.liao@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-21 16:38:30 +08:00
Juntian LiuClaude Fable 5.1zjy0516
 		76ffb0d374 [Model][Engram] Share host tables across co-located DP replicas by default (#57651) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-09-21 08:26:56 +00:00
akii96 0ff0477ff9 [Perf][ROCm] Add a ROCm path for Hy4 and compile the backbone (#57526) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
 2026-09-21 08:20:43 +00:00
Woosuk KwonandOpenAI Codex 67513c8b67 [Bugfix][DSV4.1] Restrict mHC overlap to full CUDA graphs (#57874) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-21 01:16:59 -07:00
Chaunceyandzjy0516 3df4ae153e [Core][KDA] Generalize Mamba prefill checkpoint builder and exporter (#57783) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-09-21 04:46:11 +00:00
Woosuk KwonandCodex 9a70c233cd [Perf][DSV4.1] Fuse TP all-reduce with mHC input preparation (#57643) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-20 20:28:06 -07:00
Jee Jee Li 86ce4d10e2 [BUGFIX][HY4] Record indexer completion event for full CUDA graph capture (#57811) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-09-21 10:59:11 +08:00
JohnQinAMD 6dfd87f596 [ROCm][DSv4.1] Keep the Engram tables in host memory on ROCm (#57491) 2026-09-21 10:17:11 +08:00
Woosuk KwonandCodex 9b49f92344 [Perf][DSV4.1] Overlap mHC coefficients for small TP batches (#57603) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-20 17:33:17 -07:00
Isotr0py 27757dde02 [DSV4.1] Add encoder cuda graph support for deepseek-v4.1-flash (#56625) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-20 21:51:13 +08:00
Chauncey bf01fc4a31 [GLM-5.3-Flash] Route kpool indexer top-k through the shared SparseIndexerTopk dispatcher (#57546) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-09-20 11:17:59 +00:00
Jared WenClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		db1bfdd4fb [Bugfix][GLM-5.3-Flash] Address kpool tail blocks by the padded indexer stride in the NVIDIA prefill seed kernel (#57477) 
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-20 18:15:04 +08:00
yinfengLiuandShanshan Shen 0e110f696d [ROCm][DSV4][Perf] Use FP8 WO_A output projection (#54894) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-20 09:13:55 +00:00
Fangzhou AiandShanshan Shen 1596fa5f88 [ROCm][DSv4.1][Perf] Reuse the decode topk ragged metadata across layers (#57434) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-20 11:46:33 +08:00
Ariel BereslavskyAriel BereslavskyCodexMisha GoinYongye Zhumergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		a7fda4c88b [Bugfix] Fix SM100 fp8_ds_mla cache scales (#49435) 
Signed-off-by: Ariel Bereslavsky <arielb@getimpala.ai>
Signed-off-by: Ariel Bereslavsky <ariel@getimpala.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ariel Bereslavsky <arielb@getimpala.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-19 17:15:07 -07:00
Wentao Ye 36fa72d2d0 [GLM5.3 Perf] Size the GLM-5 sparse indexer decode workspace, 3072 MiB GPU memory saved (#57701) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-19 16:57:01 -04:00
Yifan QiaoClaude Fable 5.1Woosuk Kwon
 		1dc2d854c1 [Feat][Model] Support encoder-side SWA-bounded replay for DeepSeek-V4.1-Flash (#56227) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-09-18 15:01:24 -07:00
Lucas WilkinsonCodexRobert Shaw
 		62af3df734 [Bugfix][MLA] Reserve sparse prefill buffers before KV cache sizing (#57575) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-09-18 21:22:18 +00:00
Woosuk KwonandCodex 50d812b66a [Perf][DSV4.1] Optimize MegaMoE staging and NVFP4 cache gathers (#57604) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-18 12:57:19 -07:00
Harry MellorandClaude Opus 5 8a5cf54387 [Multimodal] Type dummy options per modality (#57576) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-18 17:29:11 +01:00
Itay AlroyandTyler Michael Smith 017dced6a6 [DeepSeek V4] Fix fused MoE expert distribution (#57465) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-09-18 15:17:07 +00:00
Julien DebacheandHarry Mellor 2bdae2a5a8 [fix] Mistral-Large-3 accuracy regression on main (#57563) 
Signed-off-by: jdebache <jdebache@nvidia.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-18 14:19:16 +00:00
Kyle SayersandMisha Goin 76d517fd12 [Quantization] Support kimi-k3 routed expert quant (#57430) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-18 08:03:59 -06:00