iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
15,163 Commits 3 Branches 0 Tags
1cbbcfe8a334bab004c43a60be201c8ab528e0d2
Commit Graph
4455 Commits
Author SHA1 Message Date
Yufeng He ec2280611a [Bugfix] Fix RoBERTa position_ids accumulation on CUDA graph padding (#37884) 2026-03-23 15:15:12 +00:00
Kunshang Ji debd6e768c [XPU][MoE Refactor] Refactor xpu mxfp4 support into oracle (#37784) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-23 11:10:41 +00:00
Kunshang Ji 27d5ee3e6f [FP8]add FP8 WoQ kernel abstraction. (#32929) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
 2026-03-23 09:47:47 +00:00
Chuan (Richard) Li e99fb98867 [ROCm] Fix fused_moe_fake signature mismatch and other AITER bugs (#36100) 
Signed-off-by: Li <chuali@amd.com>
 2026-03-23 15:48:31 +08:00
Artem Perevedentsev a16133a0f1 [Perf] [Bugfix] Fix Triton autotuning in inference for Qwen3.5 (#37338) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-03-23 00:37:58 -07:00
Hojin YangandRoger Wang 54ab804e87 [Bugfix] Store Qwen3Next A_log in fp32 (#37810) 
Signed-off-by: effortprogrammer <yhjhoward7@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-03-23 15:36:57 +08:00
r266-techandr266-tech 02e6efe56d [Bugfix] JAIS: Only apply ALiBi when position_embedding_type='alibi' (#37820) 
Co-authored-by: r266-tech <r266-tech@users.noreply.github.com>
 2026-03-23 07:36:34 +00:00
Matthias GehreandMichael Goin 410d300893 [ROCm][Refactor] Enable AWQMarlinConfig on ROCm to use choose_mp_linear_kernel (#36505) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-23 15:36:08 +08:00
Baorun (Lauren) Mu f85e479e66 [Feature] ViT Full CUDA Graph (#35963) 
Signed-off-by: Baorun Mu <bmu@nvidia.com>
 2026-03-23 13:01:10 +08:00
Lasha Koroshinadze e7767eccae Fix AudioFlamingo3/MusicFlamingo HF parity and RoTE handling (#37643) 
Signed-off-by: Lasha <26011196+lashahub@users.noreply.github.com>
 2026-03-23 10:29:07 +08:00
Yongye Zhu c058ff44d4 [Bigfix]fix lora test by pass padded size back to the layer (#37811) 2026-03-22 13:20:13 -06:00
Netanel Haber e74c17e153 Enable NemotronHPuzzle + NemotronHMTP (#37803) 2026-03-22 15:13:58 +00:00
Wentao Ye 77d24c4bfe [Bug] Fix fp8 deepgemm batch invariant (#37718) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-22 08:57:20 -04:00
Robert Shaw 4383f1532e [MoE] Move PF Methods to Folder (#35927) 2026-03-22 02:42:59 -06:00
Yang Liu b050700462 [Perf] Optimize glm4.xv VIT (#37779) 
Signed-off-by: Yang <lymailforjob@gmail.com>
 2026-03-22 06:12:34 +00:00
Robert Shaw 6b2fa3a762 [MoE] Move FlashInfer CuteDSL experts into fused_moe/experts/ (#37759) 
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-03-21 19:15:16 -04:00
Robert ShawandRobert Shaw eeee5b262d [Quantization][Deprecation] Remove PTPC FP8 (#32700) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-03-21 22:10:16 +00:00
Robert Shaw 5ad0446572 Revert "Consolidate AWQ quantization into single awq_marlin.py file" (#37768) 2026-03-21 17:20:41 -04:00
Robert ShawandClaude 8cc700dd6a Consolidate AWQ quantization into single awq_marlin.py file 
Merge awq.py and awq_marlin.py into a single file, eliminating the
circular import between them. awq.py becomes a backward-compat shim.
Follows the same structure as gptq_marlin.py.

Co-authored-by: Claude

Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-03-21 17:09:17 -04:00
Chaitanya Sri Krishna LollaandTej Kiran 3982bc2cd0 [ROCm] Enable DeepEP ROCm as all2allbackend for AMD GPUs. (#34692) 
Signed-off-by: Tej Kiran <vpolamre@amd.com>
Co-authored-by: Tej Kiran <vpolamre@amd.com>
 2026-03-21 00:32:31 -07:00
Yongye ZhuandClaude Opus 4.6 87bd91892f [MoE Refactor] Mxfp4 oracle rebased (#37128) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 03:37:04 +00:00
Isotr0py c7f98b4d0a [Frontend] Remove librosa from audio dependency (#37058) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-21 11:36:15 +08:00
Wentao Ye b3d0b37908 [Refactor] Remove unused dead code (#36171) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-20 16:12:51 -06:00
Santino Ramos 85f671b8e1 [Model Runner V2] Support Streaming Inputs (#37028) 
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
 2026-03-20 20:42:25 +00:00
Vadim Gimpelson 4f16ebbbd3 [Bugfix] Disable monolithic TRTLLM MoE for Renormalize routing (#37591) (#37605) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-03-20 12:19:26 -07:00
Cyrus Leung 37aadf6237 [Model] Update Kimi-K25 and Isaac processors to fit HF-style (#37693) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-20 18:30:22 +00:00
Le Yang d7d2b5e405 [Bugfix] Disable --calculate-kv-scales for hybrid GDN/Mamba+Attention… (#37565) 
Signed-off-by: Young-Leo <562593859@qq.com>
 2026-03-20 18:28:34 +00:00
Xin Yang d0532bf38d [Perf] Eliminate redundant SparseMatrix creation in gpt_oss_triton_kernels (#37683) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-03-20 11:28:41 -06:00
Rémi Delacourt aa84e43ccb [Pixtral] Enable Pixtral language model support Eagle3 (#37182) 
Signed-off-by: remi <remi@mistral.ai>
 2026-03-20 15:50:15 +00:00
Matthias Gehre 5e806bcf54 [Bugfix] Fix ConchLinearKernel channelwise quantization (group_size=-1) (#37329) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-03-20 10:32:21 -05:00
Matthias Gehre 56a62c310c [Bugfix] Reject channelwise quantization (group_size <= 0) in ExllamaLinearKernel (#37331) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-03-20 10:31:57 -05:00
L.B.R.andL.B.R. 1779c09898 [ROCm] Enable wvSplitK skinny GEMM kernel for RDNA4/gfx1x decode (#34709) 
Signed-off-by: L.B.R. <lbr@mmonad.com>
Co-authored-by: L.B.R. <lbr@mmonad.com>
 2026-03-20 10:11:23 -05:00
xuebwang-amd 44eea10f68 [ROCm][Quantization] make quark ocp mx dtype parser robust for weight-only quantization (#36232) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
 2026-03-20 10:10:03 -05:00
Ilya Boytsov 8b6c6b9505 [Model] Add LFM2-ColBERT-350M support (#37528) 
Signed-off-by: Ilya Boytsov <ilyaboytsov1805@gmail.com>
 2026-03-20 14:57:57 +00:00
wang.yuqi ed359c497a [Model] Deprecate the score task (this will not affect users). (#37537) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-20 08:07:56 +00:00
Wangbei25andWangbei25 0674d1fee7 [PluggableLayer][MM] Add PluggableLayer for CustomQwen2Decoder (#37293) 
Signed-off-by: Wangbei25 <wangbei41@huawie.com>
Signed-off-by: Wangbei25 <wangbei41@huawei.com>
Co-authored-by: Wangbei25 <wangbei41@huawie.com>
 2026-03-20 06:24:07 +00:00
Cyrus Leung 30108fc8b0 [Model] Refactor Step3-VL processor to HF style (#37579) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-20 06:05:08 +00:00
Jee Jee Li 8fbe3f303f [Bugfix][LoRA] Fix Qwen35 LoRA (#36976) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-03-20 11:09:32 +08:00
Tianmu Li 47b7af0d87 [Feat] Enable CompressedTensorW4A8Int for XPU (#37207) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
 2026-03-20 02:34:28 +00:00
Jim SmithmgoinXin Yang
 		4120a05ff1 Fix AttributeError in Qwen3.5 GDN layers with quantized models (#37448) 
Signed-off-by: Jim Smith <jim@joshua8.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Xin Yang <105740670+xyang16@users.noreply.github.com>
 2026-03-19 19:21:14 -04:00
bnellnm 91be5f9be3 [MoE Refactor] Rename "naive" all2all backend (#36294) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-03-19 15:50:34 -04:00
bnellnm 9279c59a0e [MoE Refactor] DefaultMoERunner simplifcation (#33049) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-03-19 15:07:44 -04:00
EdalatiAliandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> daa05bf340 [Bugfix] Fix AttributeError when serving MXFP8 models with DeepGEMM installed (#37358) 
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-19 17:58:33 +00:00
Lucas Kabela 7769b58307 [torch.compile][BE][Multimodal] Remove requirement to set_model_tag to avoid cache conflict (#37345) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-03-19 17:26:12 +00:00
Cyrus Leung 657855ab41 [Misc] Cleanup more configs and processors (#37560) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-19 15:45:23 +00:00
Wei Zhao e27b8ba3d1 [Bug] Fix fp8 trtllm MoE modular kernel supported routing methods (#37346) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-03-19 11:43:06 -04:00
Cyrus Leung 9515c20868 [Misc] Clean up processing logic (#37541) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-19 13:30:20 +00:00
Cyrus Leung 7a6ebcbfcf [Model] Remove unnecessary get_language_model (#37545) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-19 20:00:36 +08:00
Cyrus Leung 765e461065 [Bugfix] Fix Nemotron Parse loading (#37407) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-19 09:55:29 +00:00
Duyi-Wang 6a9cceb219 [Bugfix][ROCm] Fix MoRI + AITER FP8 dispatch compatibility for defer_input_quant (#37418) 
Signed-off-by: Duyi-Wang <duyi.wang@amd.com>
 2026-03-19 09:49:27 +00:00