iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
19,695 Commits 3 Branches 0 Tags
e644c8cd8c734bf3b5e662a4bc363cfa8524d821
Commit Graph
5794 Commits
Author SHA1 Message Date
Andrii SkliarandAleksandr Skliar a801e71cb8 [Perf] Improve --linear-backend filtering (#48735) 
Signed-off-by: Aleksandr Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Aleksandr Skliar <askliar@nvidia.com>
 2026-08-07 15:15:54 -07:00
Nick Hill 9823714346 [Bugfix] Drop stale layer kwarg from online MXFP4 kernel creation (fix precommit) (#51442) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:49:00 -07:00
Wei-Ming Chen 1c94e8dc7d Add NVFP4 KV 4-over-6 scale search (#45187) 
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
 2026-08-07 13:22:27 -07:00
Walter Beller-MoralesandHarry Mellor 70456e5e6f [Bugfix] when loading weights skip empty expert bias if model does not support them (#50937) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-07 12:45:52 -07:00
fxmarty-amd 3518110f2f [Online quantization] Add online MXFP4 quantization support (#49347) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-07 12:39:46 -07:00
wangxiyuan 4eccf906ca [Attention] Mamba attention module refactor - Final part (#44857) 2026-08-07 11:27:10 -07:00
Harry Mellor 45273b8dcb [1/N] Harden Transformers modelling backend multi-modal path (#51408) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-07 17:30:43 +00:00
Jinzhen LinandOpenAI Codex fcde8e1460 [Refactor] refactor humming linear and moe backends to use explicit layer configs (#49610) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-07 10:03:19 -07:00
vanshbhatia-amd e229fdbc87 [ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI325X (#50007) 
Signed-off-by: vanshbhatia-amd <vansh.bhatia@amd.com>
 2026-08-07 12:58:36 -04:00
haic0 f2bfad9167 [Model] Enable Qwen3.8 for AMD Rocm (#50068) 
Signed-off-by: haic0 <haic0@users.noreply.github.com>
Signed-off-by: haic0 <haichzha@amd.com>
 2026-08-07 21:19:07 +08:00
Xiaochang WuandKunshang Ji d5aae2b464 Fix ROCm architecture import on non-ROCm platforms (#51357) 
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-07 05:32:56 -07:00
Julien Debache ae934ba8a5 feat: extended EPLB support for Mistral Large 3 and additional MoE backends (#48355) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-08-07 05:32:52 -07:00
Hank_andCyrus Leung b8db7f4abd [Bugfix][Quantization] Fix dynamic INT8 W8A8 MoE config being built as W8A16 (#50833) 
Signed-off-by: Hank <hcc.mayday@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-07 01:21:13 -07:00
Wentao Ye c84789c40b [Refactor] Remove kernel dead code (#51051) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-07 01:20:19 -07:00
jimmy-adamsAndreas KaratzashujiCursor
 		da788334bc Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel (#47972) 
Signed-off-by: Jimmy Adams <Adam_ji@outlook.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: huji <huidong.ji@amd.com>
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: huji <huidong.ji@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-07 01:19:31 -07:00
stefankoncarevicAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0de0362ea1 [ROCm][CI] Loosen block-FP8 fused MoE test tolerance for large-K shapes (#48847) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-07 01:19:26 -07:00
Bi TiekaiClaude Opus 5Yongye Zhu
 		e08111211b [Kernel] Support Nvfp4 Cutedsl Moe Swiglu-oai and Relu2(non-gated) Activation (#47106) 
Signed-off-by: Tiekai Bi <tiekaib@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-06 17:42:45 -07:00
Matej SirovatkaOpenAI Codexaoshen02Claude Opus 5Roger Wang
 		8170c23c4f [Quantization] Share online weight scales across TP (#49764) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-06 16:34:33 -07:00
Taneem Ibrahim 9464529612 [ModelRunner v2] Enable decoder token-wise pooling (#50931) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-06 22:21:52 +00:00
Wentao Ye adc3e03517 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[aA][bB]" (#48977) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-06 19:30:15 +00:00
fxmarty-amdClaudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		81be2e09ae [Weight processing] Copy over new_data attributes in replace_parameter (#49601) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 13:03:37 -04:00
Matthew BonanniClaude Opus 5OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b38e111d3e [Attention][MLA] Per-request scheduling for MLA chunked context (#50613) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 16:45:44 +00:00
Harry Mellor e7b8d59460 Fully generalise input embedding handling in Transformers modelling backend (#51247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-06 16:21:24 +00:00
Ning Xie 62a86318de [VocabParallelEmbedding] fix extra_repr fields concat (#51224) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-08-06 05:22:18 -07:00
Liangqiusong 1e05b21d61 Remove the XPU branch of topk_softplus_sqrt (#51242) 
Signed-off-by: xiaolong <xiaolong.guo@intel.com>
 2026-08-06 12:18:45 +00:00
Lyu Han 22013f74ff Interns2mobius support (#51149) 
Signed-off-by: Lyu Han <lvhan_028@163.com>
 2026-08-06 10:39:35 +00:00
anhtra3889andKhang Ngo 872fd5973e [Bugfix][LoRA] Guard TrtLlm BF16 MoE LoRA gate on activation type (#51002) 
Signed-off-by: Anh Tran <anhtra@nvidia.com>
Co-authored-by: Khang Ngo <khngo@nvidia.com>
 2026-08-06 18:12:54 +08:00
wang.yuqiandCyrus Leung 2fa490470d [Model] Fused mm preprocess normalisation on the Device (#50411) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-08-06 03:11:14 -07:00
Matej Sirovatka 9c22668436 [Quantization] Preserve precision in online NVFP4 expert packing (#50029) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
 2026-08-06 02:23:54 -07:00
zofiaandopencode febea17f6a [Model] Fix weight prefix mapping for native Qwen3.5 text-only checkp… (#50355) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: opencode <opencode@anomaly.co>
 2026-08-06 07:40:44 +00:00
Cheng JiangchengvjiangVAthree
 		470297c143 [HPC Attention Backend] hpc attention backend support bf16 kv cache with fp8 weight (#50980) 
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: VAthree <56064364+VAthree@users.noreply.github.com>
 2026-08-06 15:40:14 +08:00
zofiaandKunshang Ji 8060260e1d [Linear] [Kernel] add block-wise scaled_mm (#49932) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 14:40:26 +08:00
Gabriel WuandYongye Zhu 2e35c529b8 [Bugfix][Quantization] Fix MXFP4 conversion for FlashInfer CUTLASS (#51038) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-05 23:19:22 -07:00
zofiaopencodeKunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b976e980ef [XPU] Route AWQ linear through choose_mp_linear_kernel (#50840) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: opencode <noreply@opencode.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 13:50:11 +08:00
Xiaochang WuCopilotKunshang Ji
 		d779835a11 [XPU] Support MXFP8 linear weights for INC DeepSeek V4 model (#48476) 
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 13:17:48 +08:00
Qiuyang Yue 777b01d1a8 [MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference (#45254) 
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>
 2026-08-06 05:04:19 +00:00
Li JuqiandCyrus Leung 821717118f [BugFix][Pooling] Skip weight-prefix probe when model has WeightsMapper (#50890) 
Signed-off-by: Juqi Li <2223621784@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-05 21:12:33 -07:00
Yunzhu LuandCyrus Leung f84df12c6a [Bugfix][MM] Fix MiniCPM-V placeholder replacement and image processor loading on Transformers v5 (#48413) 
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com>
Signed-off-by: Yunzhu Lu <lucia.yunzhu@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-05 21:12:00 -07:00
Thien TranandOpenAI Codex 9f3169960a [MoE] Align TRTLLM MXFP4 autotune buckets (#50942) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-05 18:17:53 -07:00
aoshen02Matej SirovatkaRishi PuriHsiao-Yuan ChenClaude Opus 5
 		7c77868cdf [Bugfix] Size and iterate w13 by shard count for non-gated MoE (#51125) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com>
Co-authored-by: Rishi Puri <riship@nvidia.com>
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-05 17:24:47 -07:00
Tianyu Guo b30291cf0c [EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU (#50390) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-05 15:59:54 -07:00
bnellnm 373fe8b83e [MoE Refactor] Remove MoE legacy code (#51078) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-08-05 14:35:36 -07:00
Netanel Haber 482c613423 [Bugfix][Humming] Preserve ModelOpt FP8 weight dimensions (#51093) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-08-05 14:08:55 -07:00
Michael GoinandOpenAI Codex 8f158d0ee2 [MoE] Share apply_moe_activation support metadata (#44359) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-05 12:05:47 -07:00
fxmarty-amd bc37fc970e Revert [Misc] Avoid importing nixl_ep on every vllm serve config (#50879) (#51176) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-05 11:04:40 -07:00
zexplorerhj d4da0c55af [Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support (#51045) 
Signed-off-by: zexplorerhj <zhjoneson@163.com>
 2026-08-06 00:38:28 +08:00
Li JuqiMatthew BonanniClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		66b3c0e61f [Kernel][Model] Optimize FA4 mm_prefix range lookup (#50294) 
Signed-off-by: Juqi Li <2223621784@qq.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-05 07:33:49 -07:00
Gabriel WuandPavani Majety 08b8613b7b [Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness (#48929) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-08-05 07:25:46 -07:00
Jacob ZhangMatthew Bonannimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		cd930c8e78 [Bugfix] Fix MLA kv_b_proj activation dtype with Marlin FP8 (#38771) 
Signed-off-by: Jacob Zhang <jaco8123@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-05 07:25:41 -07:00
LucainOpenAI CodexClaude Opus 5Harry Mellor
 		397094da17 Resolve revision to commit_hash once per model load, via huggingface_hub's resolve_revision (#49990) 
Signed-off-by: Wauplin <lucainp@gmail.com>
Signed-off-by: Lucain <lucainp@gmail.com>
Signed-off-by: Lucain Pouget <lucain@huggingface.co>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-05 13:49:56 +00:00