iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,982 Commits 3 Branches 0 Tags
96eccb8f49aff58aa2a11b431bf8331f4b368606
Commit Graph
6144 Commits
Author SHA1 Message Date
Taoyu ZhuandShanshan Shen ce6c241ddc [ROCm][Perf][M3] Fused allreduce+GemmaRMSNorm fast path (#54787) 
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-08 07:21:49 +00:00
Subhash Polisetti 782f36cd0c [Bugfix][InternVL] Stop the video parser consuming image_embeds (#55779) 
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
 2026-09-08 05:19:41 +00:00
weihan 9ea8f3ffc3 [Bugfix] Honor STEP token pooling in DispatchPooler.for_seq_cls (#55307) 
Signed-off-by: weihan <130119942+krsish@users.noreply.github.com>
 2026-09-08 04:13:09 +00:00
mingg26Kimi Codemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7fa2c63796 [Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture (#55377) 
Signed-off-by: ming <ming@inferact.ai>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-08 04:02:23 +00:00
Wei Zhao a69402aaa8 [Perf] Integrate FlashInfer KDA kernels (#55364) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-09-07 20:29:30 -07:00
leo 12579c7f01 [Kernel][Perf] Tune H20 block-FP8 MoE low-batch configs (+21%) (#54668) 
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com>
 2026-09-08 03:20:21 +00:00
Artur FierkaandClaude 9dbdf8e9af [Bugfix][Quantization][MoE] Normalise an unset group_size on the compressed-tensors WNA16 MoE path (#53163) 
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-08 09:42:46 +08:00
Mohit SinglaCursorCodex
 		54da70c1eb [Feature] Support EAGLE3 for Sarvam (#53052) 
Signed-off-by: mohit-sarvam <mohit@sarvam.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-08 00:08:35 +00:00
am-cohereCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		70584f69b1 [Model] Add Cohere Compass model (#54774) 
Signed-off-by: am-cohere <312513976+am-cohere@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 07:36:49 -07:00
Asaf Gardinminyichenaoshen02
 		f2d45f26bd [Bugfix][Gemma] Conditionally create KV projections/norms on KV-shared layers (#54917) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
Co-authored-by: minyichen <96753147+cyc00518@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-07 15:04:00 +01:00
Roberto L. CastroCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		42801b3a6b [3/N][warmup][DSv4] Migrate FA4 MLA and shared CuTeDSL kernels (#53565) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 09:36:35 -04:00
Roberto L. CastroCodexOpenAI CodexLucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1713b9866a [2/N][warmup][DSv4] Migrate sequence and DCP kernels (#53564) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 09:35:54 -04:00
Artur Fierka 5e6f6a8ed4 [Bugfix][Quantization][XPU] Fix moe_wna16 linear weight loading (#52651) 
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
 2026-09-07 20:20:17 +08:00
Marceli FylcekandClaude Opus 5 9b85112e13 [XPU] Route grouped_topk to the fused _moe_C kernel on XPU (#53580) 
Signed-off-by: Marceli Fylcek <marceli.fylcek@intel.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-07 17:43:38 +08:00
Gabriel Wumergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Jee Jee LiYongye Zhu
 		5893426b88 [Bugfix] DSv4 MXFP4 selector: stop narrowing explicit aliases to their BF16 variant (#53586) 
Signed-off-by: Zihua Wu <zihuaw@nvidia.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-09-07 07:19:13 +00:00
zofiaandKunshang Ji ed29dfae6e [XPU] Use fused_input_norm kernel in FusedInputNorm (#52945) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-07 13:44:31 +08:00
Thien Tran d9105ea800 [Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation (#55272) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-06 21:57:56 -07:00
vhagorandIsotr0py 3dc7a68ce4 [Perf] Extend Qwen Triton warmup to avoid first-request latency spikes (#54797) 
Signed-off-by: Juqi Li <2223621784@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-07 11:46:00 +08:00
Wei Zhao 6748217fb9 [Bugfix] Fix Kimi K3 NVFP4 MoE weight conversion OOM (#55407) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-09-07 11:45:25 +08:00
yulunandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 1f778486fc [Bugfix][Offloader] Preserve prefetch static-buffer slot ownership (#54975) 
Signed-off-by: Big2Wheel <1404207590@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 10:58:44 +08:00
Fangzhou AiOpenAI CodexCursor Agent
 		de69e821b7 [ROCm][Perf][DeepSeek V4] Fuse native FP8 shared expert with MXFP4 routed experts (#53161) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Cursor Agent <agent@cursor.com>
 2026-09-06 19:42:21 -07:00
Wenhua Cheng 294fbb4f59 add 2/3/5/6/7 CUDA support in AutoRound format (#52890) 
Signed-off-by: Wenhua Cheng <wenhua.cheng@intel.com>
 2026-09-07 09:59:19 +08:00
Jee Jee Li 199cb9b964 [Kernel] Remove unused fake implementation (#55535) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-09-06 17:54:32 -07:00
SunandIsotr0py a1541f5742 [Perf] Use SDPA for BLIP-2 Q-Former attention (#55285) 
Signed-off-by: levius <2114377220@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-06 07:04:19 +00:00
Baki Burak Öğün f2e2936f91 [Kernel] Add fused MoE tuned config for E=256,N=512 on NVIDIA A100 80GB PCIe (#55511) 
Signed-off-by: Baki Burak Ogun <63836730+bakiburakogun@users.noreply.github.com>
 2026-09-06 03:15:01 +00:00
waizuichougou 32601ef7a1 [Perf][Multimodal] Avoid duplicate text embedding in Qwen2.5-Omni (#55415) 
Signed-off-by: waizuichougou <2082431897@qq.com>
 2026-09-05 06:39:32 +00:00
yongqinwang-cmdCursorWentao Ye
 		d87a440f88 [Core][MRV2] Support eagle3 spec decode with pipeline parallel (#50514) 
Signed-off-by: Yongqin Wang <yongqinwang@roblox.com>
Co-authored-by: Cursor <noreply@cursor.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-05 11:45:34 +08:00
Wei ZhaoandOpenAI Codex 16328c7a77 [Perf] Kimi K3 nvfp4 Align in_proj weights by 128 to avoid elementwise copy (#55242) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-05 10:46:47 +08:00
jimmy-adamsfxmarty-amdShanshan Shen
 		385ba6b5b0 [Bugfix][Quantization] Register Quark per-block FP8 scales as weight_scale (#54770) 
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
Co-authored-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-05 10:44:24 +08:00
Nick Hill 8277c42e4c [Perf] Ensure async h2d copies are pinned in more places (#55202) 2026-09-04 17:17:01 -07:00
Misha Goin 8f269a93bb Revert "[CI] Remove deleted nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 and its arch aliases" (#55392) 2026-09-04 14:45:24 -07:00
Anjie Houjiangkuaixue123Lucas Wilkinson
 		3f41d102c5 [1/2][Model Runner V2] DBO support, eager mode (#50945) 
Signed-off-by: specture724 <specture724@gmail.com>
Signed-off-by: Anjie Hou <149605198+specture724@users.noreply.github.com>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-04 14:17:39 -07:00
Taneem IbrahimandHarry Mellor 784cac7c42 [Mypy] Fix mypy typing for P models (#54169) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-04 20:47:11 +00:00
fxmarty-amdCodexmgoin
 		5690b02c03 [Quantization] Support online quantization with partially pre-quantized checkpoints (#51392) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-04 11:46:06 -07:00
yzyyzyhhhandyaozhiyuan.666 1ff5edb023 [Bug-fix] Fix MoE fused sum row offsets (#50220) 
Signed-off-by: yaozhiyuan.666 <yaozhiyuan.666@bytedance.com>
Co-authored-by: yaozhiyuan.666 <yaozhiyuan.666@bytedance.com>
 2026-09-04 09:39:02 -07:00
Thomas OrtnerandHarry Mellor 8ad2076c4a [Transformers backend] Find attention with a fuser and attach vLLM's layer to it (#54941) 
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-04 16:26:01 +00:00
aoshen02aoshen02aoshen02Codex
 		f19431e5c8 [Bugfix][MoE] Allow TRTLLM FP8 block-scale MoE with SwiGLU clamp (#55069) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen524@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-04 08:34:21 -07:00
siyuliusy58Isotr0py
 		a69e75b9b6 Fast Start (#54921) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: liusy58 <liusy58@smail.nju.edu.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-04 06:32:06 -07:00
Thien TranandKimi 31a8a26662 [Qwen3.8-Flash-Next] Improve QSA sparse GQA for prefill and short-ctx decode (#54873) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
 2026-09-04 06:08:02 -07:00
Taneem Ibrahim c615b1fd67 [Mypy] Fix mypy typing for L models (#54177) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-04 07:50:43 -04:00
Kunshang JiandGitHub Copilot a8693df504 [SpecDecode]Fix spec decode warmup device selection (#55245) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: GitHub Copilot <copilot@github.com>
 2026-09-04 00:48:45 -07:00
Luyi Xiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8a728663c1 [Bugfix] Reject tokenizer-less Qwen VL processor init (#54886) 
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-03 21:25:35 -07:00
Rohan PotdarClaude Opus 4.8Andreas Karatzasfai
 		579aef4e8d [ROCm] Bump AITER to 0.1.21.post1 (#52826) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: fai <fangzhouai@gmail.com>
 2026-09-03 16:28:40 -07:00
Fangzhou Ai 8bf39632b8 [ROCm][Perf] Add low-M FP32 router GEMM for gfx950 (#54845) 2026-09-03 10:42:25 -07:00
Misha GoinClaude Opus 4.8ElizaWszola
 		b7624069ff [Fusion] Manual ActivationQuantFusionPass initial application (#51415) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
 2026-09-03 13:37:31 -04:00
Bolin SunandClaude Fable 5 d410fc12f3 [Kernel] Enable Kimi-K3 SiTU on the CuteDSL MoE backend and the SM107 low-latency GEMM plan (#54606) 
Signed-off-by: Bolin Sun <bolins@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-03 16:49:31 +00:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
SunandCodex 848ab131bc [Perf] Accumulate Conformer attention scores with baddbmm (#55062) 
Signed-off-by: levius <2114377220@qq.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-03 04:17:52 -07:00
Taneem Ibrahim 758c79e1c5 [Bugfix] Retain vocab embeddings during replacement (#55083) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-03 04:17:38 -07:00
djramic 0d3ede3e3b [Bugfix][Model] Enable torch.compile for StableLM (#54969) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-09-03 03:21:54 -07:00