iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
4236 Commits
Author SHA1 Message Date
haosdent dcf6ee8592 [Bugfix] Fix encoder cache underestimation for GLM-4V/GLM-OCR single image (#34483) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-02-12 21:04:06 -08:00
Cyrus Leung 372b2e762a [Bugfix] Standardize getting number of image patches/tokens (#34358) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-12 20:47:01 -08:00
LoganJaneandIsotr0py 62788f99a4 [Bugfix] Delete unused redundant code in Kimi-K2.5 (#34427) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-12 18:18:42 -08:00
bnellnm 04ea31baab [Bugfix] Remove assert that's no longer valid (#34443) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-02-12 18:18:15 -08:00
amitz-nv f120bd42d3 [Kernel] Support Flashinfer trtllm fused MoE non gated FP8 & NVFP4 (#33506) 
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
 2026-02-12 13:06:58 -08:00
Michael Goin 6d4e27ce29 [Bugfix] Enforce DeepGEMM when using sparse_attn_indexer on CUDA (#34374) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-12 12:08:06 -08:00
Patrick von Platenandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 6c0baee610 [Voxtral Realtime] Refactor & Improve buffering logic (#34428) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-12 09:46:43 -08:00
Patrick von Platen 1100a97621 [Voxstral Realtime] Enable tests (#33803) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2026-02-12 09:43:24 -08:00
xuebwang-amdandTJian 766e167821 [ROCm][quantization] improve OCP weight quant parser robust (#34431) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-02-12 09:40:19 -08:00
Isotr0py becbe24808 [Bugfix] Remove broken raw url GGUF model loading support (#34433) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-12 09:40:01 -08:00
Harry Mellor 679ca5d8d3 Fix MoE for the Transformers modelling backend (#34436) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-12 09:29:42 -08:00
Matthew BonanniLucas WilkinsonLucas Wilkinson
 		f2c47886fd [Attention] Add FlashInfer Sparse MLA backend (#33451) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-02-12 17:21:54 +00:00
danisereb dea63512bb Add config file for fused MoE for Nemotron (TP4, B200) (#34411) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-02-12 06:09:55 -08:00
Douglas LehrDoug LehrGregory Shtrasbergtjtanaavllm
 		8a798be929 [ROCm] Enable MXFP4 MoE weight pre-shuffling on gfx950 and update aiter (#34192) 
Signed-off-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
Co-authored-by: tjtanaavllm <tunjian.tan@amd.com>
 2026-02-12 05:06:33 -08:00
AllenDouandzixiao 386bfe5d08 [bugfix] refactor FunASR's _get_data_parser (#34397) 
Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
 2026-02-12 07:26:49 +00:00
Kyle SayersandMichael Goin e9cd691132 [Bugfix] Fix Sparse24 Compressed Tensors models (#33446) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-11 23:15:16 -08:00
Yichuan WangandCursor 80f2ba6ea6 Fix DeepSeek-OCR tensor validation for all size variants (#34085) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-11 22:50:23 -08:00
Lucas WilkinsonandBill Nell 136b0bfa59 [BugFix] Fix DP chunking (#34379) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Bill Nell <bnell@redhat.com>
 2026-02-12 06:44:03 +00:00
Michael Goin ff1f83b056 [Refactor] Replace activation: str with MoEActivation enum (#33843) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-02-11 17:29:32 -08:00
bnellnm 31d992d215 [Bugfix] Fix some issues with MoERunner PR #32344 (#34371) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-02-11 14:33:14 -08:00
Raushan Turganbay 527ca32197 [Bugfix] Fix more multimodal tests for transformers V5 (#34334) 
Signed-off-by: raushan <raushan@huggingface.co>
 2026-02-11 22:02:05 +01:00
elvischenv 83e26c834e [GPT-OSS] Remove unnecessary contiguous (#34337) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2026-02-11 15:29:29 -05:00
Eldar KurtićandYour Name 11c7ace340 [Bugfix] Enable attn quantization of Llama-4 by correctly permuting scales for rope (int8, fp8) (#34243) 
Signed-off-by: Your Name <you@example.com>
Co-authored-by: Your Name <you@example.com>
 2026-02-11 13:24:22 -05:00
Xinyu Dong be7f3d5d20 [Bugfix] fix default is_neox_style is True for deepseek (#34353) 
Signed-off-by: dongxinyu03 <dongxinyu03@baidu.com>
 2026-02-11 18:20:45 +00:00
Isotr0py 0ab06100f4 [Multimodal] Expose mm_processor_kwargs for DummyInputsBuilder (#34330) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-11 09:37:40 -08:00
LindaandPavani Majety 275e0d2a99 [NVIDIA][test] Tests for flashinfer TRTLLM BF16 MoE (#33715) 
Signed-off-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-02-11 12:38:11 +00:00
Harry Mellor 0f5e55e7a8 Make JAIS compatible with Transformers v5 (#34264) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-11 12:30:37 +00:00
Harry MellorandRoger Wang 1e9204bff3 Make Qwen3VL compatible with Transformers v5 (#34262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-02-11 04:13:23 -08:00
Li, Jiang 05339a7b20 [Bugfix][CPU] Fix llama4 inference on CPU (#34321) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-02-11 19:07:23 +08:00
Harry Mellor 40b8f55358 [Docs] Reduce time spent generating API docs (#34255) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-11 02:56:02 -08:00
Kunshang Ji cb9574eb85 [XPU][9/N] clean up existing ipex code/doc (#34111) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-02-11 00:27:15 -08:00
AllenDouandzixiao 21dfb842d7 [model] support FunASR model (#33247) 
Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
 2026-02-11 07:37:09 +00:00
Roger Wang 0b20469c62 [Bugfix] Fix weight naming in Qwen3.5 (#34313) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-10 21:37:14 -08:00
Tyler Michael SmithandClaude Opus 4.6 d7982daff5 [Bugfix] Fix fused MoE IMA (sans chunking) by using int64 for strides (#34279) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-11 05:15:52 +00:00
Robert ShawandRobert Shaw 9b17c57460 [ModelBash][DSR1 NVFp4] Removed Bf16 Bias Cast (#34298) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-02-11 05:00:00 +00:00
zofia b482f71e9f [XPU][7/N] enable xpu fp8 moe (#34202) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
 2026-02-11 03:33:59 +00:00
Tyler Michael SmithandRobert Shaw 066c6da6a0 [WideEP] Fix nvfp4 DeepEP High Throughput All2All backend (#33738) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-10 19:15:43 -08:00
bnellnm d1481ba783 [MoE Refactor] Introduce MoERunner abstraction and move execution logic from FusedMoE to DefaultMoERunner (#32344) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-02-10 19:51:07 -05:00
Zhengkai Zhangandzzhengkai 6f2f59f2b3 [Misc][Spec Decode] support different load config for draft model (#34022) 
Signed-off-by: zzhengkai <zzhengkai@devgpu049.ldc1.facebook.com>
Co-authored-by: zzhengkai <zzhengkai@devgpu049.ldc1.facebook.com>
 2026-02-10 14:52:43 -08:00
Pavani Majety 578977bb5e [SM100] Resubmit FMHA FP8 prefill for MLA (#31195) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2026-02-10 16:18:43 -05:00
Roger Wang 9615575afc [Bugfix] Fix mamba cache dtype for Qwen3.5 (#34200) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-10 13:12:31 -08:00
Andy Lo ae871ca923 Minor cleanup for Voxtral (#34247) 
Signed-off-by: Andy Lo <andy@mistral.ai>
 2026-02-10 18:18:30 +00:00
Roberto L. CastroandClaude Sonnet 4.5 afdce12c89 [Perf][Kernel] Add faster topKperRow decode kernel for DeepSeek-V3.2 sparse attention (#33680) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-02-10 10:29:52 -05:00
xuebwang-amdRobert ShawRobert Shaw
 		b129136c7a [ROCm][Quantization] GPT_OSS in amd-quark format model loading and emulations (#29008) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-10 10:08:05 -05:00
tc-mbandVincent-Xiao e042d7e685 Add flagos in MiniCPM-o (#34126) 
Signed-off-by: tc-mb <caitianchi@modelbest.cn>
Signed-off-by: Vincent-Xiao <vincent.xiao.me@gmail.com>
Co-authored-by: Vincent-Xiao <vincent.xiao.me@gmail.com>
 2026-02-10 02:51:48 -08:00
Roger Wang ae4e280602 [Bugfix] Fix FI kernelchunk_gated_delta_rule output shape for Qwen3.5 (#34219) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-10 10:41:24 +00:00
Balaxxe 8d48d0a9d9 [Bugfix] Sort hf_weights_files in fastsafetensors_weights_iterator to match #33491 (#34190) 
Signed-off-by: Balaxxe <136368465+jaim12005@users.noreply.github.com>
 2026-02-09 23:06:30 -08:00
Roger Wang 047a457fa4 [Bugfix] Adopt ChunkGatedDeltaRule for Qwen3.5 (#34198) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-10 03:47:54 +00:00
Gregory Shtrasberg c60f8e3b49 [Bugfix][ROCm][GPT-OSS] Use old triton_kernels implementation on ROCm if the new API is not available (#34153) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2026-02-09 17:38:54 -06:00
Artus Krohn-Grimberghe 8fd31f6245 [Bugfix] Voxtral prompt/audio placeholder alignment (#34140) 
Signed-off-by: Artus KG <artuskg@gmail.com>
 2026-02-09 19:30:38 +00:00