iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,830 Commits 3 Branches 0 Tags
ca9c8cbd1bfb6a3fdcd3abf64b7204d967f539f0
Commit Graph
5828 Commits
Author SHA1 Message Date
Harry Mellor c65aa2ee6c Bump Transformers version to 5.15.0 (#51668) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-11 09:00:51 -07:00
Jiangyun ZhuandOpenAI Codex 5af7c8dad7 [Bugfix] Align Qwen GDN gates with speculative tokens (#51812) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 23:35:30 +08:00
pmanczakandKunshang Ji 5b5eae2394 [Misc] Enable test_silu_mul_fp8_quant_deep_gemm on XPU (#49444) 
Signed-off-by: pmanczak <pawel.manczak@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-11 23:19:14 +08:00
Andrii SkliarAndrii SkliarOpenAI Codex
 		0fb9897df1 [Bugfix][MoE] Support GELU tanh in FlashInfer B12x MoE (#51819) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 15:07:58 +00:00
Jee Jee Li dd7cc85f17 Add MoE output contract for MoE tail fusion (#51407) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-11 22:23:59 +08:00
Roberto L. CastroLucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6c95a641e9 [2/N][Feat][Perf] Add new warmup infrastructure for JITs. Add predicate filtering for JIT warmup, and migrate Inkling FA4 (#49315) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 07:46:07 -04:00
liushujia122 d8f840071e [Model] Enable tower and connector LoRA for Keye (#51780) 
Signed-off-by: shujialiu <liushujia0122@163.com>
 2026-08-11 10:35:59 +00:00
zofiaandKunshang Ji 52c70b210c [XPU] [Linear] enable torch linear backend for blockwise gemm on xpu (#50826) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-11 16:43:33 +08:00
Fangzhou Ai 1044d19998 [ROCm][DSV4] Preserve native MXFP4 TP8 shard allocation (#51473) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
 2026-08-10 23:57:27 -07:00
Harry Mellor be2e274ef5 Fix docs on main (#51773) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 23:52:04 -07:00
Priyansh JainandAndreas Karatzas 2acb055ed7 [CPU][Zen] Route BF16 MoE inference through zentorch on AMD (#44201) 
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-10 23:20:05 -07:00
Wei Zhao 608c12473f [Attention] Fix MLA prefill workspace allocation size (#51733) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-08-11 05:34:30 +00:00
Yongye ZhuOpenAI CodexClaude Opus 5
 		c76a425278 [Kernel] Optimize long-context MLA cache gathers (#51739) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-10 22:23:43 -07:00
Khushali Desaiandwang.yuqi dc5101fb1b replace batch_norm to numerically identical without cudnn (#51734) 
Signed-off-by: khushali9 <khushali.desai9@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-11 12:43:20 +08:00
Clinton ThomasandLucas Bourtoule 90fd4a333f Preserve revision pins in secondary artifact loaders (#51446) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
 2026-08-11 04:33:10 +00:00
aoshen02andRoger Wang 99e62b802c [Bugfix][Model Loader] Defer post-load attention weight processing (#49519) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-10 19:16:57 -07:00
Qiuyang Yue b2506d62ae [MM][CG][BugFix] Fix Ernie-4.5-VL encoder CG postprocess for multi-path outputs (#51461) 
Signed-off-by: qyYue1389 <yueqiuyang1389@gmail.com>
 2026-08-11 00:39:04 +00:00
Almog Tavor c3cac8c63d [Bugfix][MiMo] Apply vision attention sinks in the window attention path (#49815) 
Signed-off-by: almogtavor <almogtavor@gmail.com>
 2026-08-10 22:36:42 +00:00
Summer YangClaude Fable 5云挚foraxeWoosuk Kwon
 		63ac04a61e [Kimi-K3] DCP support (#50484) 
Signed-off-by: girasoley <girasoley@inferact.ai>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: foraxe <73625538+foraxe@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-10 12:54:26 -07:00
Elvir CrnčevićClaude Opus 4.6gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Tyler Michael Smith
 		8fea1d306f Fix uniform_random routing simulation to sample without replacement (#43680) 
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Elvir Crnčević <elvircrn@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-08-10 15:28:57 +00:00
Harry Mellor cf8f3a3bb2 [2/N] Harden Transformers modelling backend multi-modal path (#51657) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 12:57:12 +00:00
Harry Mellor 70b84f0bcb Fix Gemma 4 for upcoming Transformers version (#49797) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 07:48:26 +00:00
zexplorerhjIsotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ba1cdcfcf0 [Model][Quantization] Add Ling-3.0-flash-fp8 support (#51265) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-09 23:09:08 -07:00
Isotr0py 3b4c86e489 [Bugfix][Quantization] Fix fp32 weight scale for mxfp4 quantization and per-expert checkpoint mapping (#51419) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-09 23:08:52 -07:00
Li, Jiang 7ce84b99ce [CPU] Restore linear dispatch for small unquantized GEMMs (#51379) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-08-10 11:29:59 +08:00
Lena OnyshchenkoandKunshang Ji 751f2ccdd3 [Kernel][XPU] Tensor-descriptor operand loads for Triton W8A8 scaled_mm (#47205) 
Signed-off-by: oonyshch <xonyshch@gmail.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-10 00:56:11 +00:00
Wei ZhaojiahancKevin H. Luu
 		f18e10a7e1 Bump Flashinfer version to 0.6.16.post3 (#50892) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-08-09 00:54:07 -07:00
Isotr0py 073c510c91 [Migration] Migrate bitsandbytes support to OOT plugin (#43529) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-09 10:27:31 +08:00
Nick Hill 9b0afeb4f6 [Perf] Avoid some more unnecessary GPU<->CPU syncs (#51458) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-08 17:13:26 -07:00
ylsunandClaude Opus 5 0fe9a916e8 [Bugfix] Fix LFM2 ShortConv prefix breaking quant ignore list (#51495) 
Signed-off-by: ylsun <1654366371@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-08 11:27:13 +00:00
Tianyu Guo 653ebb52df Add torch compile for qwen3_vl encoder (#40116) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-08 08:23:18 +00:00
Andreas Karatzas 6e18901195 [Bugfix][MM] Avoid device sync in FusedInputNorm initialization (#51435) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-08 13:32:23 +08:00
djramic 5eaa70dac5 [Bugfix][Quantization] Fix INT8 W8A8 MoE crash in TritonExperts (#51411) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-08-07 22:06:58 -05:00
Linkun 7f58e8294a [Kimi][MM] disable kimi_vit's dynamic torch.compile for TPU (#51196) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2026-08-08 00:42:51 +00:00
Andrii SkliarandAleksandr Skliar a801e71cb8 [Perf] Improve --linear-backend filtering (#48735) 
Signed-off-by: Aleksandr Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Aleksandr Skliar <askliar@nvidia.com>
 2026-08-07 15:15:54 -07:00
Nick Hill 9823714346 [Bugfix] Drop stale layer kwarg from online MXFP4 kernel creation (fix precommit) (#51442) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:49:00 -07:00
Wei-Ming Chen 1c94e8dc7d Add NVFP4 KV 4-over-6 scale search (#45187) 
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
 2026-08-07 13:22:27 -07:00
Walter Beller-MoralesandHarry Mellor 70456e5e6f [Bugfix] when loading weights skip empty expert bias if model does not support them (#50937) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-07 12:45:52 -07:00
fxmarty-amd 3518110f2f [Online quantization] Add online MXFP4 quantization support (#49347) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-07 12:39:46 -07:00
wangxiyuan 4eccf906ca [Attention] Mamba attention module refactor - Final part (#44857) 2026-08-07 11:27:10 -07:00
Harry Mellor 45273b8dcb [1/N] Harden Transformers modelling backend multi-modal path (#51408) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-07 17:30:43 +00:00
Jinzhen LinandOpenAI Codex fcde8e1460 [Refactor] refactor humming linear and moe backends to use explicit layer configs (#49610) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-07 10:03:19 -07:00
vanshbhatia-amd e229fdbc87 [ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI325X (#50007) 
Signed-off-by: vanshbhatia-amd <vansh.bhatia@amd.com>
 2026-08-07 12:58:36 -04:00
haic0 f2bfad9167 [Model] Enable Qwen3.8 for AMD Rocm (#50068) 
Signed-off-by: haic0 <haic0@users.noreply.github.com>
Signed-off-by: haic0 <haichzha@amd.com>
 2026-08-07 21:19:07 +08:00
Xiaochang WuandKunshang Ji d5aae2b464 Fix ROCm architecture import on non-ROCm platforms (#51357) 
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-07 05:32:56 -07:00
Julien Debache ae934ba8a5 feat: extended EPLB support for Mistral Large 3 and additional MoE backends (#48355) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-08-07 05:32:52 -07:00
Hank_andCyrus Leung b8db7f4abd [Bugfix][Quantization] Fix dynamic INT8 W8A8 MoE config being built as W8A16 (#50833) 
Signed-off-by: Hank <hcc.mayday@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-07 01:21:13 -07:00
Wentao Ye c84789c40b [Refactor] Remove kernel dead code (#51051) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-07 01:20:19 -07:00
jimmy-adamsAndreas KaratzashujiCursor
 		da788334bc Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel (#47972) 
Signed-off-by: Jimmy Adams <Adam_ji@outlook.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: huji <huidong.ji@amd.com>
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: huji <huidong.ji@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-07 01:19:31 -07:00
stefankoncarevicAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0de0362ea1 [ROCm][CI] Loosen block-FP8 fused MoE test tolerance for large-K shapes (#48847) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-07 01:19:26 -07:00