iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
15,216 Commits 3 Branches 0 Tags
189ddefbfdf74e676c8f74ebe89826e94bf44ccf
Commit Graph
4466 Commits
Author SHA1 Message Date
Chauncey 09c3dc9186 [Revert] Remove CUDA torch fallbacks for fp8_mqa_logits/fp8_paged_mqa_logits_torch function (#37968) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-03-25 06:19:37 +00:00
Artem Perevedentsev a93a53f8a1 [Performance] Auto-enable prefetch on NFS with RAM guard (#37673) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-03-24 17:31:14 -07:00
Andreas Karatzas 679c6a3ecc [Bugfix][ROCm][MoE] Fix mxfp4 oracle regressions from #37128 (#37787) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-25 08:17:33 +08:00
Nick Cao 935c46dd9b [Model] Add Granite 4.0 1B speech to supported models (#38019) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-03-24 18:23:41 +00:00
Harry Mellor b3601da6e7 [Mypy] Fix mypy for vllm/model_executor (except vllm/model_executor/layers) (#37904) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-24 17:14:01 +00:00
Li, Jiang 352b90c4a4 [Bugfix] Add replacement of _compute_slot_mapping_kernel on CPU (#37987) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-03-24 07:00:20 -07:00
Wentao Ye c59a132f96 [V0 Deprecation] Refactor kv cache from list to element (#37487) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-23 20:10:11 -07:00
RanranandWentao Ye dc6908ac6a [Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning (#35007) 
Signed-off-by: Ranran <1012869439@qq.com>
Signed-off-by: Ranran <hzz5361@psu.edu>
Signed-off-by: ran <hzz5361@psu.edu>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-03-23 18:31:14 -04:00
yzong-rh e85f8f0932 [Bug][MoE] Strengthen _supports_current_device() checks in the TRTLLM FP8, NVFP4, and FlashInfer CuteDSL MoE experts (#36728) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-03-23 17:02:57 -04:00
Robert ShawandRobert Shaw 5bf3c42d4c [Bug][MoE] Fix TRTLLM NVFP4 Routing Kernel Precision (#36725) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-03-23 20:19:06 +00:00
Kyle Sayers 38364a7e32 [Sparse24] [Deprecation] Remove Sparse24 CT integration and kernels (#36799) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-03-23 16:03:29 -04:00
Yufeng He ec2280611a [Bugfix] Fix RoBERTa position_ids accumulation on CUDA graph padding (#37884) 2026-03-23 15:15:12 +00:00
Kunshang Ji debd6e768c [XPU][MoE Refactor] Refactor xpu mxfp4 support into oracle (#37784) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-23 11:10:41 +00:00
Kunshang Ji 27d5ee3e6f [FP8]add FP8 WoQ kernel abstraction. (#32929) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
 2026-03-23 09:47:47 +00:00
Chuan (Richard) Li e99fb98867 [ROCm] Fix fused_moe_fake signature mismatch and other AITER bugs (#36100) 
Signed-off-by: Li <chuali@amd.com>
 2026-03-23 15:48:31 +08:00
Artem Perevedentsev a16133a0f1 [Perf] [Bugfix] Fix Triton autotuning in inference for Qwen3.5 (#37338) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-03-23 00:37:58 -07:00
Hojin YangandRoger Wang 54ab804e87 [Bugfix] Store Qwen3Next A_log in fp32 (#37810) 
Signed-off-by: effortprogrammer <yhjhoward7@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-03-23 15:36:57 +08:00
r266-techandr266-tech 02e6efe56d [Bugfix] JAIS: Only apply ALiBi when position_embedding_type='alibi' (#37820) 
Co-authored-by: r266-tech <r266-tech@users.noreply.github.com>
 2026-03-23 07:36:34 +00:00
Matthias GehreandMichael Goin 410d300893 [ROCm][Refactor] Enable AWQMarlinConfig on ROCm to use choose_mp_linear_kernel (#36505) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-23 15:36:08 +08:00
Baorun (Lauren) Mu f85e479e66 [Feature] ViT Full CUDA Graph (#35963) 
Signed-off-by: Baorun Mu <bmu@nvidia.com>
 2026-03-23 13:01:10 +08:00
Lasha Koroshinadze e7767eccae Fix AudioFlamingo3/MusicFlamingo HF parity and RoTE handling (#37643) 
Signed-off-by: Lasha <26011196+lashahub@users.noreply.github.com>
 2026-03-23 10:29:07 +08:00
Yongye Zhu c058ff44d4 [Bigfix]fix lora test by pass padded size back to the layer (#37811) 2026-03-22 13:20:13 -06:00
Netanel Haber e74c17e153 Enable NemotronHPuzzle + NemotronHMTP (#37803) 2026-03-22 15:13:58 +00:00
Wentao Ye 77d24c4bfe [Bug] Fix fp8 deepgemm batch invariant (#37718) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-22 08:57:20 -04:00
Robert Shaw 4383f1532e [MoE] Move PF Methods to Folder (#35927) 2026-03-22 02:42:59 -06:00
Yang Liu b050700462 [Perf] Optimize glm4.xv VIT (#37779) 
Signed-off-by: Yang <lymailforjob@gmail.com>
 2026-03-22 06:12:34 +00:00
Robert Shaw 6b2fa3a762 [MoE] Move FlashInfer CuteDSL experts into fused_moe/experts/ (#37759) 
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-03-21 19:15:16 -04:00
Robert ShawandRobert Shaw eeee5b262d [Quantization][Deprecation] Remove PTPC FP8 (#32700) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-03-21 22:10:16 +00:00
Robert Shaw 5ad0446572 Revert "Consolidate AWQ quantization into single awq_marlin.py file" (#37768) 2026-03-21 17:20:41 -04:00
Robert ShawandClaude 8cc700dd6a Consolidate AWQ quantization into single awq_marlin.py file 
Merge awq.py and awq_marlin.py into a single file, eliminating the
circular import between them. awq.py becomes a backward-compat shim.
Follows the same structure as gptq_marlin.py.

Co-authored-by: Claude

Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-03-21 17:09:17 -04:00
Chaitanya Sri Krishna LollaandTej Kiran 3982bc2cd0 [ROCm] Enable DeepEP ROCm as all2allbackend for AMD GPUs. (#34692) 
Signed-off-by: Tej Kiran <vpolamre@amd.com>
Co-authored-by: Tej Kiran <vpolamre@amd.com>
 2026-03-21 00:32:31 -07:00
Yongye ZhuandClaude Opus 4.6 87bd91892f [MoE Refactor] Mxfp4 oracle rebased (#37128) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 03:37:04 +00:00
Isotr0py c7f98b4d0a [Frontend] Remove librosa from audio dependency (#37058) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-21 11:36:15 +08:00
Wentao Ye b3d0b37908 [Refactor] Remove unused dead code (#36171) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-20 16:12:51 -06:00
Santino Ramos 85f671b8e1 [Model Runner V2] Support Streaming Inputs (#37028) 
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
 2026-03-20 20:42:25 +00:00
Vadim Gimpelson 4f16ebbbd3 [Bugfix] Disable monolithic TRTLLM MoE for Renormalize routing (#37591) (#37605) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-03-20 12:19:26 -07:00
Cyrus Leung 37aadf6237 [Model] Update Kimi-K25 and Isaac processors to fit HF-style (#37693) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-20 18:30:22 +00:00
Le Yang d7d2b5e405 [Bugfix] Disable --calculate-kv-scales for hybrid GDN/Mamba+Attention… (#37565) 
Signed-off-by: Young-Leo <562593859@qq.com>
 2026-03-20 18:28:34 +00:00
Xin Yang d0532bf38d [Perf] Eliminate redundant SparseMatrix creation in gpt_oss_triton_kernels (#37683) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-03-20 11:28:41 -06:00
Rémi Delacourt aa84e43ccb [Pixtral] Enable Pixtral language model support Eagle3 (#37182) 
Signed-off-by: remi <remi@mistral.ai>
 2026-03-20 15:50:15 +00:00
Matthias Gehre 5e806bcf54 [Bugfix] Fix ConchLinearKernel channelwise quantization (group_size=-1) (#37329) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-03-20 10:32:21 -05:00
Matthias Gehre 56a62c310c [Bugfix] Reject channelwise quantization (group_size <= 0) in ExllamaLinearKernel (#37331) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-03-20 10:31:57 -05:00
L.B.R.andL.B.R. 1779c09898 [ROCm] Enable wvSplitK skinny GEMM kernel for RDNA4/gfx1x decode (#34709) 
Signed-off-by: L.B.R. <lbr@mmonad.com>
Co-authored-by: L.B.R. <lbr@mmonad.com>
 2026-03-20 10:11:23 -05:00
xuebwang-amd 44eea10f68 [ROCm][Quantization] make quark ocp mx dtype parser robust for weight-only quantization (#36232) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
 2026-03-20 10:10:03 -05:00
Ilya Boytsov 8b6c6b9505 [Model] Add LFM2-ColBERT-350M support (#37528) 
Signed-off-by: Ilya Boytsov <ilyaboytsov1805@gmail.com>
 2026-03-20 14:57:57 +00:00
wang.yuqi ed359c497a [Model] Deprecate the score task (this will not affect users). (#37537) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-20 08:07:56 +00:00
Wangbei25andWangbei25 0674d1fee7 [PluggableLayer][MM] Add PluggableLayer for CustomQwen2Decoder (#37293) 
Signed-off-by: Wangbei25 <wangbei41@huawie.com>
Signed-off-by: Wangbei25 <wangbei41@huawei.com>
Co-authored-by: Wangbei25 <wangbei41@huawie.com>
 2026-03-20 06:24:07 +00:00
Cyrus Leung 30108fc8b0 [Model] Refactor Step3-VL processor to HF style (#37579) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-20 06:05:08 +00:00
Jee Jee Li 8fbe3f303f [Bugfix][LoRA] Fix Qwen35 LoRA (#36976) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-03-20 11:09:32 +08:00
Tianmu Li 47b7af0d87 [Feat] Enable CompressedTensorW4A8Int for XPU (#37207) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
 2026-03-20 02:34:28 +00:00