iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,683 Commits 3 Branches 0 Tags
42fadebecb79290ad722f33f3094de23b121f33d
Commit Graph
4305 Commits
Author SHA1 Message Date
chengchengpeiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 965525667b Onboard voyage-4-nano (#33720) 
Signed-off-by: Chengcheng Pei <chengchengpei@outlook.com>
Signed-off-by: chengchengpei <5881383+chengchengpei@users.noreply.github.com>
Co-authored-by: chengchengpei <5881383+chengchengpei@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-06 06:23:34 +00:00
Kunshang Ji 7439e4f41b [XPU][4/N] add mxfp4 moe model support (#33679) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-02-06 13:03:59 +08:00
Rabi Mishra 20d7454c9b fix(ROCm): Make flash_attn import optional in MLA attention (#33511) 
Signed-off-by: rabi <ramishra@redhat.com>
 2026-02-06 02:22:53 +00:00
Xin Yang 79028d4388 [Perf] Disable clean_logits in deepgemm fp8_mqa_logits kernel (#33568) 2026-02-05 20:34:00 -05:00
emricksini-h 325ab6b0a8 [Feature] OTEL tracing during loading (#31162) 2026-02-05 16:59:28 -08:00
Hashem Hashemi d5c4800112 Adds padding and perf improvements to wvSplitK_fp8 (#33527) 
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
 2026-02-05 22:16:02 +00:00
Lumosisandmgoin 42d5d705f9 [Minor] Sort safetensors files to ensure deterministic loading order (#33491) 
Signed-off-by: Lihao Ran <imlihao.ran@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-02-05 17:05:09 -05:00
Cyrus Leung 116880a5a0 [Bugfix] Make MM batching more robust (#33817) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-05 20:40:58 +00:00
Matthew Bonanni 4145e50d85 [Bugfix] Fix DSV3.2 NVFP4 (#33932) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-05 19:22:19 +00:00
Isotr0py 87d0d17ab5 [Models] Consolidate Deepseek-OCR2 processor (#33909) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-05 18:29:20 +00:00
bnellnmandRobert Shaw a57c8228ff [Moe Refactor] Make Inplace Flag for FusedMoEModularKernel part of the constructor (#33375) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-05 18:07:18 +00:00
Benjamin Chislett af3162d3aa [Spec Decode] Unified Parallel Drafting (#32887) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-02-05 12:37:18 -05:00
Aaron HaoandSumanthRH c1858b7ec8 [Feat][RL][1/2] Native Weight Syncing API: NCCL (#31943) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: SumanthRH <sumanthrh99@gmail.com>
 2026-02-05 12:13:23 -05:00
wang.yuqi 1c3a221d3b [Bugfix] Fix corner case of sparse embedding (#33886) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-02-05 02:51:22 -08:00
Isotr0py a2522839d8 [Bugfix] Fix Kimi-K2.5 NVFP4 checkpoints weight loading (#33876) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-05 10:29:54 +00:00
jiahanc 59a5cb387a [perf] Integrate flashinfer concat_mla_k (#31171) 2026-02-05 05:23:11 -05:00
Andreas KaratzasandMatthew Wong 3e472e81f9 [ROCm][Bugfix][CI] Fix hybrid models and their tests (Mamba/Jamba/Bamba) (#32710) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-02-05 10:01:23 +00:00
Pavani Majety d2f4a71cd5 [Bugfix] Kimi-K2 grouped_topk usage for Flashinfer monolithic kernels. (#33858) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2026-02-05 09:32:10 +00:00
Fadi Arafeh fd03538bf9 [CPU][BugFix] Allow w8a8 oneDNN quantized matmul to support 3D inputs (#33727) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-02-05 06:26:09 +00:00
Chauncey a7be77beef [Bugfix] fix DeepSeek R1 with CUTLASS MLA Broken on B200 (#33637) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-02-05 01:28:36 +00:00
Ilya BoytsovCyrus Leungwang.yuqi
 		439afa4eea feat: Add ColBERT late interaction model support (#33686) 
Signed-off-by: Ilya Boytsov <ilyaboytsov1805@gmail.com>
Signed-off-by: Ilya Boytsov <boytsovpanamera@mail.ru>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-02-05 08:05:13 +08:00
Sage MooreandRobert Shaw ce498a6d61 Change the type signature of MixtureOfExperts.expert_weights to MutableSequence[Sequence[Tensor]] (#33573) 
Signed-off-by: Sage Moore <sagmoore@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-04 17:02:46 -05:00
Muhammad HashmiandNickLucche 535de06cb1 [Model] Add transcription support for Qwen3-Omni (#29828) 
Signed-off-by: Muhammad Hashmi <mhashmi@berkeley.edu>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
 2026-02-04 21:17:47 +00:00
Simon Danielsson 4292c90a2a [Bugfix] Support RotaryEmbedding CustomOp for gpt-oss (#33800) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2026-02-04 20:17:41 +00:00
Taeksang Kim 6e98f6d8b6 Implement zero-copy GQA for multimodal and CPU (#33732) 
Signed-off-by: Taeksang Kim <ts.kim@hyperaccel.ai>
 2026-02-04 20:11:39 +00:00
Isotr0py 192ad4648b [Bugfix] Fix interns1-pro initialization and PP (#33793) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-04 17:54:45 +00:00
Cyrus Leung e57ef99b40 [Model] Apply #32631 for recent models (#33785) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-04 12:23:01 +00:00
Yueqian LinandRoger Wang f8516a1ab9 [Bugfix][Model] Fix audio-in-video support for Qwen2.5-Omni and Qwen3-Omni (#33605) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-02-04 12:15:29 +00:00
Vadim Gimpelson 824058076c [PERF] Change GDN Attention State Layout from [N, HV, K, V] to [N, HV, V, K] (#33291) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-02-04 11:20:52 +00:00
Kunshang Ji f79f777803 [XPU][2/N] add support unquantized moe support for xpu (#33659) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-02-04 02:12:25 -08:00
Frank Wang 45f8fd6f97 [Feature] Enable TRITON_ATTN for Batch Invariance (#33688) 
Signed-off-by: frankwang28 <frank.wbb@hotmail.com>
 2026-02-04 13:27:34 +08:00
Michael Goin eb5ed20743 [Bugfix] Define router_logits_dtype for remaining MoE models (#33737) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-04 13:24:14 +08:00
Shanshan Shen 9fb27dd3b3 [MM] Align the prefix of MMEncoderAttention with Attention (#33750) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-02-04 04:07:30 +00:00
Matthew Bonanni bd8da29a66 [Bugfix] Fix sparse MLA metadata building (#33579) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-03 15:29:48 -08:00
Michael Goin 2a99c5a6c8 [Bugfix] Disable TRTLLM FP8 MoE if router_logits_dtype==float32 and routing_method!=DeepSeekV3 (#33613) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-03 13:26:51 -08:00
Vadim Gimpelson a372f3f40a [MISC] Fix Tensor Parallelism for Quantized Mamba Models with n_groups=1 (#33257) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-02-03 15:10:31 -05:00
Patrick von PlatenandCyrus Leung f0d5251715 [Voxtral models] Skip warm-up to skip confusing error message in warm-up (#33576) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-03 07:22:34 -08:00
Shanshan Shen 5c4f2dd6ef [MM] Pass prefix parameter to MMEncoderAttention (#33674) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-02-03 06:47:41 -08:00
Harry Mellor 2a8d84e66d Fix Gemma3n audio encoder for Transformers v5 (#33673) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-03 05:49:49 -08:00
zxyandIsotr0py a3acfa1071 [Models] Intern-S1-Pro (#33636) 
Signed-off-by: zxy <zhou0493@e.ntu.edu.sg>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-03 05:49:45 -08:00
Song ZhixinandCyrus Leung ceab70c89d [Bugfix] fix qwen3-asr response error (#33644) 
Signed-off-by: jesse <szxfml@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-03 03:33:56 -08:00
Michael Goin e346e2d056 [Bugfix] Disable RoutingMethodType.[Renormalize,RenormalizeNaive] TRTLLM per-tensor FP8 MoE (#33620) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-03 10:37:15 +00:00
Kunshang Ji e10604480b [XPU][1/N] Deprecate ipex and switch to vllm-xpu-kernels for xpu platform (#33379) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-02-02 22:46:10 -08:00
Shengliang XuandCyrus Leung f1cb9b5544 Fix quantized Falcon-H1 model loading issues (#32728) 
Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-02 22:31:27 -08:00
Patrick von Platenandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 5019c59dd2 [Voxtral Realtime] Introduce global log mel max (#33574) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-02 17:01:47 -05:00
Vasiliy Kuznetsov 0130223bd9 fix memory for online fp8 quantization with streaming weight load (#31914) 
Signed-off-by: vasiliy <vasiliy@fb.com>
 2026-02-02 14:17:42 -05:00
Yang Liu 199e3cb476 [Model] Use mm_position to compute mrope positions for GLM-4.xV (#33039) 
Signed-off-by: Yang <lymailforjob@gmail.com>
 2026-02-02 16:55:48 +00:00
Isotr0py 4061dcf4c5 [Bugfix] Enable Kimi k25 processor test (#33562) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-02 14:25:25 +00:00
danielafrimi 0aca8b8c62 [MoE] Enable Shared/Routed Overlap For Latent MoE (Nemotron-H) (#32790) 
Signed-off-by: dafrimi <dafrimi@nvidia.com>
 2026-02-02 09:18:50 -05:00
Cyrus Leung b10d05b8a8 [Model] Use explicit types in get_generation_prompt (#33551) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-02 12:38:49 +00:00