iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,022 Commits 3 Branches 0 Tags
5af4cc33ec0837ded8a18b368ec58a9fd4e4d5cd
Commit Graph
6159 Commits
Author SHA1 Message Date
Taneem Ibrahim 5af4cc33ec [Bugfix] Fix OpenPangu multimodal embedding merge (#55941) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-08 23:53:59 +00:00
Jim BanClaudeHarry Mellor
 		c0d8d5413e [Transformers] Generalize merged-column linear fusion (#55301) 
Signed-off-by: BANANASJIM <bananasjim1@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-08 23:49:37 +00:00
sychen52mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Isotr0py
 		60ad959b6f Fix block FP8 MTP in ModelOpt mixed checkpoints (#55513) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-08 22:47:21 +00:00
bnellnmLucas WilkinsonOpenAI Codex
 		d29c88f162 [Core] Let SWA layers take the primary block size to avoid inflating the KV block LCM (#53007) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-08 16:44:43 -04:00
Roderick Wuandmgoin 2c9d68f80b [Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering (#54809) 
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com>
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-08 12:45:12 -07:00
Jiangyun ZhukhluujiahaoKimi Code
 		8ebc5b0a18 [Bugfix] Fix NVFP4 fused SiLU+mul scale allocation and global scale direction (#55643) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: jiahao <jxia77@terpmail.umd.edu>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-08 08:26:07 -07:00
Wentao Ye 6b15bea080 [Refactor] Remove utils dead code (#53941) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-08 11:22:09 -04:00
Canlin GuoandOpenAI Codex 8e1f97e709 [Qwen3.8-Flash-Next] Tune FP8 TP2/TP4 Triton MoE on B200 (#55890) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-08 22:45:18 +08:00
Roberto L. CastroandCodex 6ddbab03de [4/N][warmup][DSv4] Migrate common attention kernels (#50176) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-08 10:25:48 -04:00
Mohit SinglaandCodex 613ab20f7f [Model] Enable torch.compile for Sarvam MLA (#55817) 
Signed-off-by: mohit-sarvam <mohit@sarvam.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-08 22:01:30 +08:00
Thomas ParnellClaudeHarry Mellor
 		414057a3d3 [Bugfix] Apply attention sinks in the Transformers backend (#52156) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-08 13:54:07 +00:00
SunCodexIsotr0py
 		472d8c9125 [Perf] Fuse DeepEncoder relative bias in Triton attention (#55629) 
Signed-off-by: levius <2114377220@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-08 13:22:04 +00:00
Stefano Castagnetta f6326f53bd [Perf][GDN] Enable the FlashInfer GDN prefill kernel on SM12x (#55715) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-09-08 12:26:03 +00:00
Stefano Castagnetta 13cf9e05c1 [Perf][Quant][NVFP4] Prefer W4A4 linear kernels over weight-only ones on SM120/121 (#55170) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-09-08 15:06:45 +04:00
jimmy-adams e41a17e606 [ROCm][Quantization] Support AMD Quark per-block FP8 for fused MoE layers (#52263) 
Signed-off-by: huji <huidong.ji@amd.com>
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
 2026-09-08 10:30:26 +00:00
Taoyu ZhuandShanshan Shen ce6c241ddc [ROCm][Perf][M3] Fused allreduce+GemmaRMSNorm fast path (#54787) 
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-08 07:21:49 +00:00
Subhash Polisetti 782f36cd0c [Bugfix][InternVL] Stop the video parser consuming image_embeds (#55779) 
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
 2026-09-08 05:19:41 +00:00
weihan 9ea8f3ffc3 [Bugfix] Honor STEP token pooling in DispatchPooler.for_seq_cls (#55307) 
Signed-off-by: weihan <130119942+krsish@users.noreply.github.com>
 2026-09-08 04:13:09 +00:00
mingg26Kimi Codemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7fa2c63796 [Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture (#55377) 
Signed-off-by: ming <ming@inferact.ai>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-08 04:02:23 +00:00
Wei Zhao a69402aaa8 [Perf] Integrate FlashInfer KDA kernels (#55364) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-09-07 20:29:30 -07:00
leo 12579c7f01 [Kernel][Perf] Tune H20 block-FP8 MoE low-batch configs (+21%) (#54668) 
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com>
 2026-09-08 03:20:21 +00:00
Artur FierkaandClaude 9dbdf8e9af [Bugfix][Quantization][MoE] Normalise an unset group_size on the compressed-tensors WNA16 MoE path (#53163) 
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-08 09:42:46 +08:00
Mohit SinglaCursorCodex
 		54da70c1eb [Feature] Support EAGLE3 for Sarvam (#53052) 
Signed-off-by: mohit-sarvam <mohit@sarvam.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-08 00:08:35 +00:00
am-cohereCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		70584f69b1 [Model] Add Cohere Compass model (#54774) 
Signed-off-by: am-cohere <312513976+am-cohere@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 07:36:49 -07:00
Asaf Gardinminyichenaoshen02
 		f2d45f26bd [Bugfix][Gemma] Conditionally create KV projections/norms on KV-shared layers (#54917) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
Co-authored-by: minyichen <96753147+cyc00518@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-07 15:04:00 +01:00
Roberto L. CastroCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		42801b3a6b [3/N][warmup][DSv4] Migrate FA4 MLA and shared CuTeDSL kernels (#53565) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 09:36:35 -04:00
Roberto L. CastroCodexOpenAI CodexLucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1713b9866a [2/N][warmup][DSv4] Migrate sequence and DCP kernels (#53564) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 09:35:54 -04:00
Artur Fierka 5e6f6a8ed4 [Bugfix][Quantization][XPU] Fix moe_wna16 linear weight loading (#52651) 
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
 2026-09-07 20:20:17 +08:00
Marceli FylcekandClaude Opus 5 9b85112e13 [XPU] Route grouped_topk to the fused _moe_C kernel on XPU (#53580) 
Signed-off-by: Marceli Fylcek <marceli.fylcek@intel.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-07 17:43:38 +08:00
Gabriel Wumergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Jee Jee LiYongye Zhu
 		5893426b88 [Bugfix] DSv4 MXFP4 selector: stop narrowing explicit aliases to their BF16 variant (#53586) 
Signed-off-by: Zihua Wu <zihuaw@nvidia.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-09-07 07:19:13 +00:00
zofiaandKunshang Ji ed29dfae6e [XPU] Use fused_input_norm kernel in FusedInputNorm (#52945) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-07 13:44:31 +08:00
Thien Tran d9105ea800 [Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation (#55272) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-06 21:57:56 -07:00
vhagorandIsotr0py 3dc7a68ce4 [Perf] Extend Qwen Triton warmup to avoid first-request latency spikes (#54797) 
Signed-off-by: Juqi Li <2223621784@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-07 11:46:00 +08:00
Wei Zhao 6748217fb9 [Bugfix] Fix Kimi K3 NVFP4 MoE weight conversion OOM (#55407) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-09-07 11:45:25 +08:00
yulunandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 1f778486fc [Bugfix][Offloader] Preserve prefetch static-buffer slot ownership (#54975) 
Signed-off-by: Big2Wheel <1404207590@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 10:58:44 +08:00
Fangzhou AiOpenAI CodexCursor Agent
 		de69e821b7 [ROCm][Perf][DeepSeek V4] Fuse native FP8 shared expert with MXFP4 routed experts (#53161) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Cursor Agent <agent@cursor.com>
 2026-09-06 19:42:21 -07:00
Wenhua Cheng 294fbb4f59 add 2/3/5/6/7 CUDA support in AutoRound format (#52890) 
Signed-off-by: Wenhua Cheng <wenhua.cheng@intel.com>
 2026-09-07 09:59:19 +08:00
Jee Jee Li 199cb9b964 [Kernel] Remove unused fake implementation (#55535) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-09-06 17:54:32 -07:00
SunandIsotr0py a1541f5742 [Perf] Use SDPA for BLIP-2 Q-Former attention (#55285) 
Signed-off-by: levius <2114377220@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-06 07:04:19 +00:00
Baki Burak Öğün f2e2936f91 [Kernel] Add fused MoE tuned config for E=256,N=512 on NVIDIA A100 80GB PCIe (#55511) 
Signed-off-by: Baki Burak Ogun <63836730+bakiburakogun@users.noreply.github.com>
 2026-09-06 03:15:01 +00:00
waizuichougou 32601ef7a1 [Perf][Multimodal] Avoid duplicate text embedding in Qwen2.5-Omni (#55415) 
Signed-off-by: waizuichougou <2082431897@qq.com>
 2026-09-05 06:39:32 +00:00
yongqinwang-cmdCursorWentao Ye
 		d87a440f88 [Core][MRV2] Support eagle3 spec decode with pipeline parallel (#50514) 
Signed-off-by: Yongqin Wang <yongqinwang@roblox.com>
Co-authored-by: Cursor <noreply@cursor.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-05 11:45:34 +08:00
Wei ZhaoandOpenAI Codex 16328c7a77 [Perf] Kimi K3 nvfp4 Align in_proj weights by 128 to avoid elementwise copy (#55242) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-05 10:46:47 +08:00
jimmy-adamsfxmarty-amdShanshan Shen
 		385ba6b5b0 [Bugfix][Quantization] Register Quark per-block FP8 scales as weight_scale (#54770) 
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
Co-authored-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-05 10:44:24 +08:00
Nick Hill 8277c42e4c [Perf] Ensure async h2d copies are pinned in more places (#55202) 2026-09-04 17:17:01 -07:00
Misha Goin 8f269a93bb Revert "[CI] Remove deleted nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 and its arch aliases" (#55392) 2026-09-04 14:45:24 -07:00
Anjie Houjiangkuaixue123Lucas Wilkinson
 		3f41d102c5 [1/2][Model Runner V2] DBO support, eager mode (#50945) 
Signed-off-by: specture724 <specture724@gmail.com>
Signed-off-by: Anjie Hou <149605198+specture724@users.noreply.github.com>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-04 14:17:39 -07:00
Taneem IbrahimandHarry Mellor 784cac7c42 [Mypy] Fix mypy typing for P models (#54169) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-04 20:47:11 +00:00
fxmarty-amdCodexmgoin
 		5690b02c03 [Quantization] Support online quantization with partially pre-quantized checkpoints (#51392) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-04 11:46:06 -07:00
yzyyzyhhhandyaozhiyuan.666 1ff5edb023 [Bug-fix] Fix MoE fused sum row offsets (#50220) 
Signed-off-by: yaozhiyuan.666 <yaozhiyuan.666@bytedance.com>
Co-authored-by: yaozhiyuan.666 <yaozhiyuan.666@bytedance.com>
 2026-09-04 09:39:02 -07:00