iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,683 Commits 3 Branches 0 Tags
42fadebecb79290ad722f33f3094de23b121f33d
Commit Graph
4305 Commits
Author SHA1 Message Date
david guanandClaude Sonnet 4.5 bc0d291bfe feat: Complete LoRA support for MiniMaxM2 Fixes #32736 (#32763) 
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-01-24 20:48:46 +08:00
Isotr0py 9ad7f89f55 [Models]: Make Multimodal config implicit in ViT implementation (#31972) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-24 20:34:26 +08:00
Isotr0py 8edaf38570 [Models] Add SharedFusedMoE support to Qwen3MoE (#32082) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-23 23:36:31 -08:00
ElizaWszolamgoinVarun Sundar RabindranathMatthew BonanniLuka GovedičLuka GovedičLucas WilkinsonLuka Govedič
 		a28b94e6ef [Performance] Split FlashAttn attention and cache update (#25954) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Luka Govedič <luka.govedic@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <luka.govedic@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2026-01-23 17:28:06 -08:00
joninco ebd0a17e0e [Bugfix] Fix missing is_layer_skipped check for FusedMoE in AWQConfig (#32935) 
Signed-off-by: jon <joninco@bullpoint.org>
 2026-01-23 17:19:56 -05:00
Wentao Ye 37c9859fab [Refactor] Clean up unused variables & func (#32692) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-23 17:04:25 -05:00
Michael Goin 4561f13985 [Refactor] Rename gptq_marlin to marlin to match MoE (#32952) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-23 16:48:12 -05:00
Markus / MarkClaude Opus 4.5mgoin
 		586a57ad7e fix: Add glm4_moe_lite to MLA detection (#32614) 
Signed-off-by: marksverdhei <marksverdhei@hotmail.com>
Signed-off-by: Markus / Mark <46672778+marksverdhei@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-01-23 12:38:57 -08:00
Harry HuangandChen Zhang 5206e5e28c [V1][Hybrid] Mamba Prefix Caching with align mode (#30877) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2026-01-23 09:56:48 -08:00
Matteo Fari fec9da0af4 [Model] Enable LoRA support for internvl2 (#32397) 
Signed-off-by: Matteo Fari <matteofari06@gmail.com>
 2026-01-24 01:39:01 +08:00
baonudesifeizhai 1fb648bf10 [Bugfix] Fix FP8 MoE EP Weight Loading for ModelOpt Llama4 (#32886) 
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
 2026-01-23 10:31:48 -05:00
Xin Yang 90c2007932 [Bugfix] Disable tma_aligned_scales in test_fusions_e2e (#32916) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-01-23 14:34:30 +00:00
Raushan Turganbay d95d650762 [Bugfix] Fix getting vision features in Transformer Multimodal backend (#32933) 
Signed-off-by: raushan <raushan@huggingface.co>
 2026-01-23 13:34:48 +00:00
tianshu-Michael-yu 13d8746c54 [Feature]: Remove DtoH Copy for lfm2_vl On Default Stream (#32815) 
Signed-off-by: Tianshu Yu <tianshuyu.formal@gmail.com>
 2026-01-23 13:20:30 +00:00
Patrick von Platenandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 3f3f89529d [Voxtral] Add new streaming arch (#32861) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-23 12:41:52 +01:00
Li, Jiangandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 5da4c7d789 [CI/Build][CPU] Fix failed pooling tests and macos smoke test (#32907) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Li, Jiang <bigpyj64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-23 10:48:20 +00:00
Andreas Karatzas a8eb1182f1 [CI][Models] Add VLM Support for Sequence Classification Conversion (#32885) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-01-23 16:22:51 +08:00
Karan Bansal fa6e599a61 [Bugfix] Fix _CPU_MOE_ACT AssertionError when vLLM config not set (#32777) 
Signed-off-by: Karan Bansal <karanb192@gmail.com>
 2026-01-23 08:22:37 +00:00
Luka GovedičandMichael Goin 5e4e0e51f4 [torch.compile] Compile CustomOp.forward_native for SiluAndMul and QuantFP8 to avoid raw torch ops inside opaque custom ops (#32806) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-22 19:52:26 -08:00
bnellnm dc917cceb8 [MoE Refactor] Move select_experts from FusedMoEQuantMethod -> FusedMoE (#31996) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-22 18:21:35 -05:00
Fadi Arafeh fc56f4a071 [BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registration (#32855) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-01-22 22:27:40 +00:00
Xin Yang d08b356ee0 [Perf] Create TMA-aligned input scale tensor for DeepGemm on Hopper (#32619) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-01-22 15:47:04 -05:00
Eldar Kurtić 44f08af3a7 Add llmcompressor fp8 kv-cache quant (per-tensor and per-attn_head) (#30141) 
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
Signed-off-by: eldarkurtic <8884008+eldarkurtic@users.noreply.github.com>
 2026-01-22 13:29:57 -07:00
Matthew Bonanni 955b43a5a5 [Bugfix][Attention] Explicitly report support for kv_cache_dtype bfloat16 (#32795) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-22 19:05:18 +00:00
Maximilien de Bayser ff365eea94 Support bge-m3 sparse embeddings and colbert embeddings (#14526) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
 2026-01-22 23:52:57 +08:00
Richard Zou 654a71fc3c [torch.compile] Improve Cold Start for MoEs (#32805) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-01-22 10:44:40 -05:00
Alex Sun 49a1262267 [AMD][ROCm] MoRI EP: a high-performance all2all backend (#28664) 
Signed-off-by: Alex Sun <alex.s@amd.com>
 2026-01-22 16:33:18 +08:00
Cyrus Leung 2b8a38b6d6 [Model] Extend collect_children and no_init_weights contexts (#32757) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-22 08:20:27 +00:00
Andreas KaratzasandMatthew Wong eb1629da24 [ROCm][CI] Fix AITER test flakiness by using explicit attention backend (#32346) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-01-22 13:55:25 +08:00
Patrick von Platen 1579c9b5fd [Llama.py -> mistral.py] Extract mistral-only relevant code into separate file (#32780) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2026-01-22 05:14:57 +00:00
Xin Yang 63227accf5 [Kernel] Add topk_sigmoid kernel (#31246) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-01-21 22:49:51 +00:00
Divakar Verma 180fba653e [ROCm] fix import for on_gfx9 (#32783) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-01-21 18:41:11 +00:00
elvischenv 808d6fd7b9 Bump Flashinfer to v0.6.1 (#30993) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2026-01-21 08:49:50 -08:00
whx 1861ae8aae [PluggableLayer][1/N] Define PluggableLayer (Fix ci) (#32744) 
Signed-off-by: whx-sjtu <2952154980@qq.com>
 2026-01-21 11:38:04 -05:00
Robert ShawandRobert Shaw 4e31b7f228 [Quantization][Deprecation] Remove RTN (#32697) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-21 16:34:42 +00:00
Pleaplusone 6c20e89c02 [ROCm][Deepseekv3.2] Refactor Sparse Indexer as CustomOp (#29287) 
Signed-off-by: ganyi <ygan@amd.com>
 2026-01-21 23:16:30 +08:00
Robert ShawandRobert Shaw 85f55c943c [Quantization][Deprecation] Deprecate HQQ (#32681) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-21 09:32:40 -05:00
Robert ShawandRobert Shaw cea3c754c4 [Quantization][Deprecation] Remove DeepSpeedFp8 (#32679) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-21 09:32:12 -05:00
Robert Shaw 42135d6898 [MoE Refactor] Oracle Select FP8+NVFP4 Kernels In Priority (#32414) 2026-01-21 08:22:33 -05:00
Divakar Verma e14467be43 [bugfix] Aria model (#32727) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-01-21 05:11:31 -08:00
Kim Hee Su 7727ce35c2 [Model] Add Eagle2.5-8B Vision-Language Model support (#32456) 
Signed-off-by: kimheesu <wlskaka4@gmail.com>
 2026-01-21 09:39:53 +00:00
RickyChen / 陳昭儒 f23fb5a7c1 [Bugfix] Support HF sharded weights for Mistral3/Pixtral models (#32673) 
Signed-off-by: ricky-chaoju <ricky.chen@infinirc.com>
Signed-off-by: vllm-dev <ricky.chen@infinirc.com>
 2026-01-20 23:27:30 -08:00
Netanel Haber 27ca95b3c9 [Bugfix] Fix Nemotron-Nano-v2-vlm static resolution (#32682) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-01-21 06:28:21 +00:00
shanjiaz 7ab80a8e37 Added qwen3 vision language moe support for speculative decoding (#32048) 
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com>
Signed-off-by: shanjiaz <43143795+shanjiaz@users.noreply.github.com>
 2026-01-21 03:24:05 +00:00
gopalsarda 0900cedb3f Enable Eagle3 speculative decoding for Pixtral (LlavaForConditionalGeneration) (#32542) 
Signed-off-by: gopalsarda <gopal.sarda@servicenow.com>
 2026-01-21 11:18:05 +08:00
Alex Brooks 27b81e010d [Bugfix] Fix Granite Vision / Don't use Siglip Pooling Head Nested Models by Default (#32299) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2026-01-21 11:11:52 +08:00
Robert Shaw c78ee240b3 Revert "[PluggableLayer][1/N] Define PluggableLayer" (#32725) 2026-01-21 00:21:06 +00:00
Vasiliy Kuznetsov d2389c1262 fp8 online quant: split out Fp8OnlineLinearMethod (#32189) 2026-01-20 18:13:22 -05:00
TJian 9b67338b78 [Bugfix] Suppress log on non-ROCm platform (#32703) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-20 13:38:20 -08:00
Cyrus Leung 193069d129 [5/N] Initialize MM components in context managers (Q-Z) (#32695) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-20 19:10:23 +00:00