iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,601 Commits 3 Branches 0 Tags
b976e980efdfb95d0ade2ff6658ca736c737f1bf
Commit Graph
2837 Commits
Author SHA1 Message Date
zofiaopencodeKunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b976e980ef [XPU] Route AWQ linear through choose_mp_linear_kernel (#50840) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: opencode <noreply@opencode.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 13:50:11 +08:00
Thien TranandOpenAI Codex 9f3169960a [MoE] Align TRTLLM MXFP4 autotune buckets (#50942) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-05 18:17:53 -07:00
aoshen02Matej SirovatkaRishi PuriHsiao-Yuan ChenClaude Opus 5
 		7c77868cdf [Bugfix] Size and iterate w13 by shard count for non-gated MoE (#51125) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com>
Co-authored-by: Rishi Puri <riship@nvidia.com>
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-05 17:24:47 -07:00
bnellnm 373fe8b83e [MoE Refactor] Remove MoE legacy code (#51078) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-08-05 14:35:36 -07:00
Netanel Haber 482c613423 [Bugfix][Humming] Preserve ModelOpt FP8 weight dimensions (#51093) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-08-05 14:08:55 -07:00
Michael GoinandOpenAI Codex 8f158d0ee2 [MoE] Share apply_moe_activation support metadata (#44359) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-05 12:05:47 -07:00
fxmarty-amd bc37fc970e Revert [Misc] Avoid importing nixl_ep on every vllm serve config (#50879) (#51176) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-05 11:04:40 -07:00
zexplorerhj d4da0c55af [Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support (#51045) 
Signed-off-by: zexplorerhj <zhjoneson@163.com>
 2026-08-06 00:38:28 +08:00
Gabriel WuandPavani Majety 08b8613b7b [Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness (#48929) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-08-05 07:25:46 -07:00
Jacob ZhangMatthew Bonannimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		cd930c8e78 [Bugfix] Fix MLA kv_b_proj activation dtype with Marlin FP8 (#38771) 
Signed-off-by: Jacob Zhang <jaco8123@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-05 07:25:41 -07:00
Jakub ZakrzewskiOpenAI CodexWentao Ye
 		c25093a305 [Kernel] Batch invariant NVFP4 MoE using cutlass (#40372) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-08-05 04:31:32 +00:00
aoshen02andyihengz 6153dbe036 [R3] Unify routed expert shape configuration (#50940) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: yihengz <yihengz@nvidia.com>
 2026-08-04 20:20:41 -07:00
Yejing Laiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2cb3ff88cc [BUGFIX][Quant]Fix test_kv_scale_reload failed (#50405) 
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-05 10:50:54 +08:00
Julian HuangOpenAI CodexTyler Michael Smith
 		05b7876f50 [MoE][Humming] Support SiTU activation for Kimi-K3 (#50510) 
Signed-off-by: 墨楼 <huangzhilin.hzl@antgroup.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-08-04 12:39:18 -07:00
Wentao Ye d31de3c421 [Kimi K3 Perf] option to shard the shared expert for non mega case, 16.98 GiB memory/GPU saved (#50912) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-04 14:00:44 -04:00
amitz-nvandOpenAI Codex 52c0e3cb08 [Kernel] Add support for Flashinfer Mamba SSU algorithm selection (#50157) 
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-04 09:40:23 -07:00
Nicolò Lucchesi 7b50d2c0bc [Misc] Avoid importing nixl_ep on every vllm serve config (#50879) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-04 07:56:54 -07:00
yimdevOpenAI CodexMatthew Bonanni
 		199644d410 [Bugfix][Attention] Guard sparse MLA masked MHA workspace (#50906) 
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-08-04 09:59:33 -04:00
fxmarty-amdandAndreas Karatzas 385d4c084e [ROCm][AITER] Hotfix for memory access fault errors in AITER triton MOE routing (#50859) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-03 21:07:46 -05:00
zzt 413e70d52c Perf/h20 moe config e256 n512 (#48825) 
Signed-off-by: zzt <zengzetang.zzt@antgroup.com>
 2026-08-03 17:47:58 -07:00
Chris FontesChris FontesChris Fontesmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0b37d8389f fix: NVFP4 quantization out_dtype should match model dtype, not torch default (#48861) 
Signed-off-by: Chris Fontes <2224082+fattchris@users.noreply.github.com>
Co-authored-by: Chris Fontes <chris@fontes.io>
Co-authored-by: Chris Fontes <chris@fontes.dev>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-03 22:47:37 +00:00
skysnow2001andDouglas Lehr f43e1d26e3 [ROCm] Enable AITER and FP8 inference on GFX120x (#43615) 
Signed-off-by: skysnow2001 <skysnow9285@gmail.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-08-03 16:45:37 -05:00
aoshen02andNick Hill 42ab184ea7 [MRV2] Enable routed-experts capture (#50721) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-03 21:00:13 +00:00
Hongxia Yang 8f50685c48 [ROCm][Kimi-K3] aiter moe environment variable cleanup (#50582) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
 2026-08-03 18:54:04 +00:00
wangxiyuan dd11df04f3 [Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation (#49389) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-08-03 11:15:13 -04:00
Wentao Ye 0cf49a5d15 [Refactor] Remove multiple dead codes (#50285) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-03 10:59:08 -04:00
Li, Jiang c8602c7906 [CPU] Refine CPU kernel dispatch (#50801) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-08-03 17:42:53 +08:00
Jee Jee Li b9d1e2437e K3: Move LatentMoERunner (#50678) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-03 09:36:17 +00:00
Jee Jee Li 4635cc3e8f Shard the K3 Latent-MoE up-projection on large batches (#50383) 2026-08-03 00:49:39 -07:00
Li, JiangClaude Opus 5Fadi Arafeh
 		0a6446005d [CPU] Migrate unquantized MoE to the modular-kernel experts structure (#50133) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-08-03 14:55:04 +08:00
zofiaandKunshang Ji e481da9508 [XPU] [Linear] add torch as xpu linear backend (#49664) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-03 14:30:21 +08:00
Qiming Zhang 2755489a26 [INC] fix w4a4 model (#50807) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-08-03 14:28:58 +08:00
Mike GMichael Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e3be89673d Enable ModelOpt FP8 emulation on SM80 (#50019) 
Signed-off-by: Mike G <180722391+mikekg@users.noreply.github.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 14:48:27 -07:00
bnellnmClaudeTaneem IbrahimAgata DobrzyniewiczTushar JainTushar JainWentao YeJiangyun Zhu
 		454ea5b526 [MoE Refactor] Combine CompressedTensorsWNA16MarlinMoEMethod with CompressedTensorsWNA16MoEMethod (#44570) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Dobrzyniewicz, Agata <agata.dobrzyniewicz@intel.com>
Signed-off-by: Tushar Jain <tushar00jain@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyn@users.noreply.github.com>
Co-authored-by: Tushar Jain <8455015+tushar00jain@users.noreply.github.com>
Co-authored-by: Tushar Jain <tushar00jain@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-07-31 14:13:03 -07:00
Soila KavulyaandClaude aef85aed5d [Bugfix][TurboQuant] Add KV quant mode for turboquant (#50533) 
Signed-off-by: Soila Kavulya <soila.p.kavulya@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-31 17:42:32 +00:00
fxmarty-amd e67a2e0a56 [ROCm][Quark][7/N] Use MXFP4 linear kernel abstraction for emulation backend (#48949) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-07-31 10:07:15 -07:00
Harry MellorClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		94e9ef0768 [Bugfix] Don't transpose fused MoE quantization scales in RoutedExperts.load_weights (#50137) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 08:46:52 -07:00
Andreas Karatzas 5233368dac [ROCm][CI] Fall back to lossless Kimi K3 MXFP4 emulation on gfx942 (#50516) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-31 11:33:16 -04:00
Matthew BonanniandOpenAI Codex 82ae4164ee [2/N][Attention] Enable masked MHA for sparse MLA prefills (#48770) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 06:01:24 -07:00
Martin Hickey b2fb83e7ff [Attention]: Use KVCacheSpec for AttentionMetadataBuilder type hints (#50148) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-31 11:20:18 +00:00
bnellnmandHarry Mellor 6e311c6e20 [MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-31 11:28:18 +01:00
Fadi Arafeh 0e9b50074e [chore] clean-up weight prepack for INT8 MoE (#50116) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-07-31 10:15:38 +00:00
Taneem IbrahimOpenAI CodexIsotr0py
 		f727951d3f [Bugfix] Re-land MiniMax M3 default video processor (#50305) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-31 07:46:18 +00:00
Ryan RockAndreas Karatzasdepthfirst-app[bot] <184448029+depthfirst-app[bot]@users.noreply.github.com>root
 		6724051c57 [CI/Build][AMD] Install triton_kernels via CMake (#50328) 
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: depthfirst-app[bot] <184448029+depthfirst-app[bot]@users.noreply.github.com>
Co-authored-by: root <root@smci355-ccs-aus-m11-05.cs-aus.dcgpu>
 2026-07-30 22:05:19 -05:00
vanshbhatia-amd 4689c7dd61 [ROCm] Add tuned selective_state_update float16 config for AMD Instinct MI325X (#50006) 
Signed-off-by: vanshbhatia-amd <vansh.bhatia@amd.com>
 2026-07-31 02:43:53 +00:00
+1
Juan VillamizarDouglas LehrClaude Opus 4.7DanielJaden MathiasjpvillamrootDouglas LehrdllehrDaniel ChanAndreas Karatzas
 		4f1da84eb5 Enable gfx1250 ROCm architecture (#46516) 
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com>
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: dllehr <douglehr@amd.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-31 01:41:12 +00:00
Wentao Ye 0bff0ce5d3 [Kimi K3 Bug] Fix deepgemm support for kimi k3 (#50458) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-30 20:10:16 -04:00
Ryan ClarkandClaude Fable 5 68fb303a41 [Bugfix] Preserve Marlin runtime tensor storage across weight reload (#48438) 
Signed-off-by: Ryan Clark <ryanclark2k@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 14:50:03 -07:00
Netanel HaberandOpenAI Codex 70bd10930b [Quantization] Honor --linear-backend for ModelOpt W4A16 (#50273) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-30 13:19:56 -07:00
Richard ZouandCodex c27b080d26 [compile] Fix fake kernel return dtype (#50444) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-30 19:50:02 +00:00