iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
15,118 Commits 3 Branches 0 Tags
eeee5b262d9a730794ecfc82cb3d55ff030ca3e1
Commit Graph
4439 Commits
Author SHA1 Message Date
Robert ShawandRobert Shaw eeee5b262d [Quantization][Deprecation] Remove PTPC FP8 (#32700) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-03-21 22:10:16 +00:00
Robert Shaw 5ad0446572 Revert "Consolidate AWQ quantization into single awq_marlin.py file" (#37768) 2026-03-21 17:20:41 -04:00
Robert ShawandClaude 8cc700dd6a Consolidate AWQ quantization into single awq_marlin.py file 
Merge awq.py and awq_marlin.py into a single file, eliminating the
circular import between them. awq.py becomes a backward-compat shim.
Follows the same structure as gptq_marlin.py.

Co-authored-by: Claude

Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-03-21 17:09:17 -04:00
Chaitanya Sri Krishna LollaandTej Kiran 3982bc2cd0 [ROCm] Enable DeepEP ROCm as all2allbackend for AMD GPUs. (#34692) 
Signed-off-by: Tej Kiran <vpolamre@amd.com>
Co-authored-by: Tej Kiran <vpolamre@amd.com>
 2026-03-21 00:32:31 -07:00
Yongye ZhuandClaude Opus 4.6 87bd91892f [MoE Refactor] Mxfp4 oracle rebased (#37128) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 03:37:04 +00:00
Isotr0py c7f98b4d0a [Frontend] Remove librosa from audio dependency (#37058) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-21 11:36:15 +08:00
Wentao Ye b3d0b37908 [Refactor] Remove unused dead code (#36171) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-20 16:12:51 -06:00
Santino Ramos 85f671b8e1 [Model Runner V2] Support Streaming Inputs (#37028) 
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
 2026-03-20 20:42:25 +00:00
Vadim Gimpelson 4f16ebbbd3 [Bugfix] Disable monolithic TRTLLM MoE for Renormalize routing (#37591) (#37605) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-03-20 12:19:26 -07:00
Cyrus Leung 37aadf6237 [Model] Update Kimi-K25 and Isaac processors to fit HF-style (#37693) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-20 18:30:22 +00:00
Le Yang d7d2b5e405 [Bugfix] Disable --calculate-kv-scales for hybrid GDN/Mamba+Attention… (#37565) 
Signed-off-by: Young-Leo <562593859@qq.com>
 2026-03-20 18:28:34 +00:00
Xin Yang d0532bf38d [Perf] Eliminate redundant SparseMatrix creation in gpt_oss_triton_kernels (#37683) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-03-20 11:28:41 -06:00
Rémi Delacourt aa84e43ccb [Pixtral] Enable Pixtral language model support Eagle3 (#37182) 
Signed-off-by: remi <remi@mistral.ai>
 2026-03-20 15:50:15 +00:00
Matthias Gehre 5e806bcf54 [Bugfix] Fix ConchLinearKernel channelwise quantization (group_size=-1) (#37329) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-03-20 10:32:21 -05:00
Matthias Gehre 56a62c310c [Bugfix] Reject channelwise quantization (group_size <= 0) in ExllamaLinearKernel (#37331) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-03-20 10:31:57 -05:00
L.B.R.andL.B.R. 1779c09898 [ROCm] Enable wvSplitK skinny GEMM kernel for RDNA4/gfx1x decode (#34709) 
Signed-off-by: L.B.R. <lbr@mmonad.com>
Co-authored-by: L.B.R. <lbr@mmonad.com>
 2026-03-20 10:11:23 -05:00
xuebwang-amd 44eea10f68 [ROCm][Quantization] make quark ocp mx dtype parser robust for weight-only quantization (#36232) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
 2026-03-20 10:10:03 -05:00
Ilya Boytsov 8b6c6b9505 [Model] Add LFM2-ColBERT-350M support (#37528) 
Signed-off-by: Ilya Boytsov <ilyaboytsov1805@gmail.com>
 2026-03-20 14:57:57 +00:00
wang.yuqi ed359c497a [Model] Deprecate the score task (this will not affect users). (#37537) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-20 08:07:56 +00:00
Wangbei25andWangbei25 0674d1fee7 [PluggableLayer][MM] Add PluggableLayer for CustomQwen2Decoder (#37293) 
Signed-off-by: Wangbei25 <wangbei41@huawie.com>
Signed-off-by: Wangbei25 <wangbei41@huawei.com>
Co-authored-by: Wangbei25 <wangbei41@huawie.com>
 2026-03-20 06:24:07 +00:00
Cyrus Leung 30108fc8b0 [Model] Refactor Step3-VL processor to HF style (#37579) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-20 06:05:08 +00:00
Jee Jee Li 8fbe3f303f [Bugfix][LoRA] Fix Qwen35 LoRA (#36976) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-03-20 11:09:32 +08:00
Tianmu Li 47b7af0d87 [Feat] Enable CompressedTensorW4A8Int for XPU (#37207) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
 2026-03-20 02:34:28 +00:00
Jim SmithmgoinXin Yang
 		4120a05ff1 Fix AttributeError in Qwen3.5 GDN layers with quantized models (#37448) 
Signed-off-by: Jim Smith <jim@joshua8.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Xin Yang <105740670+xyang16@users.noreply.github.com>
 2026-03-19 19:21:14 -04:00
bnellnm 91be5f9be3 [MoE Refactor] Rename "naive" all2all backend (#36294) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-03-19 15:50:34 -04:00
bnellnm 9279c59a0e [MoE Refactor] DefaultMoERunner simplifcation (#33049) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-03-19 15:07:44 -04:00
EdalatiAliandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> daa05bf340 [Bugfix] Fix AttributeError when serving MXFP8 models with DeepGEMM installed (#37358) 
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-19 17:58:33 +00:00
Lucas Kabela 7769b58307 [torch.compile][BE][Multimodal] Remove requirement to set_model_tag to avoid cache conflict (#37345) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-03-19 17:26:12 +00:00
Cyrus Leung 657855ab41 [Misc] Cleanup more configs and processors (#37560) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-19 15:45:23 +00:00
Wei Zhao e27b8ba3d1 [Bug] Fix fp8 trtllm MoE modular kernel supported routing methods (#37346) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-03-19 11:43:06 -04:00
Cyrus Leung 9515c20868 [Misc] Clean up processing logic (#37541) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-19 13:30:20 +00:00
Cyrus Leung 7a6ebcbfcf [Model] Remove unnecessary get_language_model (#37545) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-19 20:00:36 +08:00
Cyrus Leung 765e461065 [Bugfix] Fix Nemotron Parse loading (#37407) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-19 09:55:29 +00:00
Duyi-Wang 6a9cceb219 [Bugfix][ROCm] Fix MoRI + AITER FP8 dispatch compatibility for defer_input_quant (#37418) 
Signed-off-by: Duyi-Wang <duyi.wang@amd.com>
 2026-03-19 09:49:27 +00:00
Collin McCarthy 0b6d52629f Support temporal compression for Nemotron-3-VL videos (#36808) 
Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
 2026-03-19 08:02:19 +00:00
Aaron Hao 5f82706a21 [BUG] Exclude SKIP_TENSORS from get_layer_size() + new weight sync example for dpep (#37334) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2026-03-19 00:45:10 +00:00
Michael Goin 9482b0b085 [Bugfix] Remove assertion for NVFP4 scale dynamic range (#37465) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-03-18 15:37:49 -07:00
Woosuk Kwon 5bc1da147f [LoRA][BugFix] Fix skipped LoRA adapters for Mistral3 (#36928) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-18 22:34:19 +00:00
Wentao Ye 0d81a1fe61 [V0 Deprecation] Deprecate virtual engine (#37195) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-18 14:30:14 -07:00
Netanel Haber 6ae4c8d6fc chunk parakeet into 30s clips to prevent OOMs on long audios (#36671) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-03-18 14:22:24 -07:00
JartX a913b612d8 [Bugfix] Fix ROCm crash in qwen3_next multi-stream events (#36795) (#37427) 
Signed-off-by: JartX <sagformas@epdcenter.es>
 2026-03-18 16:06:31 -04:00
Chengyu Fang 738d0a281f [Bugfix] Fix incorrect use of merge_size in Qwen3-VL video timestamp calculation (#37439) 
Signed-off-by: chengyufang <cnyvfang@outlook.com>
 2026-03-18 11:36:34 -07:00
Cyrus Leung 7476d148db [Model] Remove unnecessary processor definition for Nemotron Parse (#37456) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-18 18:25:13 +00:00
Cyrus Leung f3732bd931 [Misc] Clean up model registry (#37457) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-18 18:24:44 +00:00
Wentao Ye 0ef7f79054 [Perf] Add tuned triton moe config for Qwen3.5 H200, 9.9% E2E throughput improvement (#37340) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-18 14:18:34 -04:00
Xin Yang b1169d7be8 [Kernel] Add gpt-oss Router GEMM kernel (#37205) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-03-18 08:15:56 -07:00
XLiu-2000andXuLiu 17808394bc standardize load_weights using AutoWeightsLoader for kimi_linear and minimax_text_01 (#37371) 
Signed-off-by: XuLiu <xuliu40@gmail.com>
Co-authored-by: XuLiu <xuliu40@gmail.com>
 2026-03-18 15:05:37 +00:00
elvischenv 296839a1b0 [Perf] Eliminate padding and slicing op for GPT-OSS with Flashinfer MXFP4 MXFP8 MoE (#30647) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2026-03-18 15:01:26 +00:00
Wentao Ye c373b5c00d [Log] Reduce duplicate log (#37313) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-18 10:57:44 -04:00
Cyrus Leung 99267c23ca [2/3] Refactor InternVL-based processors (#37324) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-18 22:22:19 +08:00