iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,756 Commits 3 Branches 0 Tags
bd6536071cec4dcd8cf91c0e2aa04aec83fc1c37
Commit Graph
5808 Commits
Author SHA1 Message Date
Harry Mellor cf8f3a3bb2 [2/N] Harden Transformers modelling backend multi-modal path (#51657) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 12:57:12 +00:00
Harry Mellor 70b84f0bcb Fix Gemma 4 for upcoming Transformers version (#49797) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 07:48:26 +00:00
zexplorerhjIsotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ba1cdcfcf0 [Model][Quantization] Add Ling-3.0-flash-fp8 support (#51265) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-09 23:09:08 -07:00
Isotr0py 3b4c86e489 [Bugfix][Quantization] Fix fp32 weight scale for mxfp4 quantization and per-expert checkpoint mapping (#51419) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-09 23:08:52 -07:00
Li, Jiang 7ce84b99ce [CPU] Restore linear dispatch for small unquantized GEMMs (#51379) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-08-10 11:29:59 +08:00
Lena OnyshchenkoandKunshang Ji 751f2ccdd3 [Kernel][XPU] Tensor-descriptor operand loads for Triton W8A8 scaled_mm (#47205) 
Signed-off-by: oonyshch <xonyshch@gmail.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-10 00:56:11 +00:00
Wei ZhaojiahancKevin H. Luu
 		f18e10a7e1 Bump Flashinfer version to 0.6.16.post3 (#50892) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-08-09 00:54:07 -07:00
Isotr0py 073c510c91 [Migration] Migrate bitsandbytes support to OOT plugin (#43529) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-09 10:27:31 +08:00
Nick Hill 9b0afeb4f6 [Perf] Avoid some more unnecessary GPU<->CPU syncs (#51458) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-08 17:13:26 -07:00
ylsunandClaude Opus 5 0fe9a916e8 [Bugfix] Fix LFM2 ShortConv prefix breaking quant ignore list (#51495) 
Signed-off-by: ylsun <1654366371@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-08 11:27:13 +00:00
Tianyu Guo 653ebb52df Add torch compile for qwen3_vl encoder (#40116) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-08 08:23:18 +00:00
Andreas Karatzas 6e18901195 [Bugfix][MM] Avoid device sync in FusedInputNorm initialization (#51435) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-08 13:32:23 +08:00
djramic 5eaa70dac5 [Bugfix][Quantization] Fix INT8 W8A8 MoE crash in TritonExperts (#51411) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-08-07 22:06:58 -05:00
Linkun 7f58e8294a [Kimi][MM] disable kimi_vit's dynamic torch.compile for TPU (#51196) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2026-08-08 00:42:51 +00:00
Andrii SkliarandAleksandr Skliar a801e71cb8 [Perf] Improve --linear-backend filtering (#48735) 
Signed-off-by: Aleksandr Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Aleksandr Skliar <askliar@nvidia.com>
 2026-08-07 15:15:54 -07:00
Nick Hill 9823714346 [Bugfix] Drop stale layer kwarg from online MXFP4 kernel creation (fix precommit) (#51442) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:49:00 -07:00
Wei-Ming Chen 1c94e8dc7d Add NVFP4 KV 4-over-6 scale search (#45187) 
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
 2026-08-07 13:22:27 -07:00
Walter Beller-MoralesandHarry Mellor 70456e5e6f [Bugfix] when loading weights skip empty expert bias if model does not support them (#50937) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-07 12:45:52 -07:00
fxmarty-amd 3518110f2f [Online quantization] Add online MXFP4 quantization support (#49347) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-07 12:39:46 -07:00
wangxiyuan 4eccf906ca [Attention] Mamba attention module refactor - Final part (#44857) 2026-08-07 11:27:10 -07:00
Harry Mellor 45273b8dcb [1/N] Harden Transformers modelling backend multi-modal path (#51408) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-07 17:30:43 +00:00
Jinzhen LinandOpenAI Codex fcde8e1460 [Refactor] refactor humming linear and moe backends to use explicit layer configs (#49610) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-07 10:03:19 -07:00
vanshbhatia-amd e229fdbc87 [ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI325X (#50007) 
Signed-off-by: vanshbhatia-amd <vansh.bhatia@amd.com>
 2026-08-07 12:58:36 -04:00
haic0 f2bfad9167 [Model] Enable Qwen3.8 for AMD Rocm (#50068) 
Signed-off-by: haic0 <haic0@users.noreply.github.com>
Signed-off-by: haic0 <haichzha@amd.com>
 2026-08-07 21:19:07 +08:00
Xiaochang WuandKunshang Ji d5aae2b464 Fix ROCm architecture import on non-ROCm platforms (#51357) 
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-07 05:32:56 -07:00
Julien Debache ae934ba8a5 feat: extended EPLB support for Mistral Large 3 and additional MoE backends (#48355) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-08-07 05:32:52 -07:00
Hank_andCyrus Leung b8db7f4abd [Bugfix][Quantization] Fix dynamic INT8 W8A8 MoE config being built as W8A16 (#50833) 
Signed-off-by: Hank <hcc.mayday@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-07 01:21:13 -07:00
Wentao Ye c84789c40b [Refactor] Remove kernel dead code (#51051) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-07 01:20:19 -07:00
jimmy-adamsAndreas KaratzashujiCursor
 		da788334bc Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel (#47972) 
Signed-off-by: Jimmy Adams <Adam_ji@outlook.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: huji <huidong.ji@amd.com>
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: huji <huidong.ji@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-07 01:19:31 -07:00
stefankoncarevicAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0de0362ea1 [ROCm][CI] Loosen block-FP8 fused MoE test tolerance for large-K shapes (#48847) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-07 01:19:26 -07:00
Bi TiekaiClaude Opus 5Yongye Zhu
 		e08111211b [Kernel] Support Nvfp4 Cutedsl Moe Swiglu-oai and Relu2(non-gated) Activation (#47106) 
Signed-off-by: Tiekai Bi <tiekaib@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-06 17:42:45 -07:00
Matej SirovatkaOpenAI Codexaoshen02Claude Opus 5Roger Wang
 		8170c23c4f [Quantization] Share online weight scales across TP (#49764) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-06 16:34:33 -07:00
Taneem Ibrahim 9464529612 [ModelRunner v2] Enable decoder token-wise pooling (#50931) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-06 22:21:52 +00:00
Wentao Ye adc3e03517 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[aA][bB]" (#48977) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-06 19:30:15 +00:00
fxmarty-amdClaudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		81be2e09ae [Weight processing] Copy over new_data attributes in replace_parameter (#49601) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 13:03:37 -04:00
Matthew BonanniClaude Opus 5OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b38e111d3e [Attention][MLA] Per-request scheduling for MLA chunked context (#50613) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 16:45:44 +00:00
Harry Mellor e7b8d59460 Fully generalise input embedding handling in Transformers modelling backend (#51247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-06 16:21:24 +00:00
Ning Xie 62a86318de [VocabParallelEmbedding] fix extra_repr fields concat (#51224) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-08-06 05:22:18 -07:00
Liangqiusong 1e05b21d61 Remove the XPU branch of topk_softplus_sqrt (#51242) 
Signed-off-by: xiaolong <xiaolong.guo@intel.com>
 2026-08-06 12:18:45 +00:00
Lyu Han 22013f74ff Interns2mobius support (#51149) 
Signed-off-by: Lyu Han <lvhan_028@163.com>
 2026-08-06 10:39:35 +00:00
anhtra3889andKhang Ngo 872fd5973e [Bugfix][LoRA] Guard TrtLlm BF16 MoE LoRA gate on activation type (#51002) 
Signed-off-by: Anh Tran <anhtra@nvidia.com>
Co-authored-by: Khang Ngo <khngo@nvidia.com>
 2026-08-06 18:12:54 +08:00
wang.yuqiandCyrus Leung 2fa490470d [Model] Fused mm preprocess normalisation on the Device (#50411) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-08-06 03:11:14 -07:00
Matej Sirovatka 9c22668436 [Quantization] Preserve precision in online NVFP4 expert packing (#50029) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
 2026-08-06 02:23:54 -07:00
zofiaandopencode febea17f6a [Model] Fix weight prefix mapping for native Qwen3.5 text-only checkp… (#50355) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: opencode <opencode@anomaly.co>
 2026-08-06 07:40:44 +00:00
Cheng JiangchengvjiangVAthree
 		470297c143 [HPC Attention Backend] hpc attention backend support bf16 kv cache with fp8 weight (#50980) 
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: VAthree <56064364+VAthree@users.noreply.github.com>
 2026-08-06 15:40:14 +08:00
zofiaandKunshang Ji 8060260e1d [Linear] [Kernel] add block-wise scaled_mm (#49932) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 14:40:26 +08:00
Gabriel WuandYongye Zhu 2e35c529b8 [Bugfix][Quantization] Fix MXFP4 conversion for FlashInfer CUTLASS (#51038) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-05 23:19:22 -07:00
zofiaopencodeKunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b976e980ef [XPU] Route AWQ linear through choose_mp_linear_kernel (#50840) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: opencode <noreply@opencode.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 13:50:11 +08:00
Xiaochang WuCopilotKunshang Ji
 		d779835a11 [XPU] Support MXFP8 linear weights for INC DeepSeek V4 model (#48476) 
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 13:17:48 +08:00
Qiuyang Yue 777b01d1a8 [MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference (#45254) 
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>
 2026-08-06 05:04:19 +00:00