iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
5,126 Commits 3 Branches 0 Tags
d47807ba0806c5bbd8fd08c19013c327b34dcac5
Commit Graph
1451 Commits
Author SHA1 Message Date
Lucas Wilkinson d47807ba08 [Attention] Remove slow setattr in MLA (#14769) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-03-13 21:31:14 +00:00
Cyrus Leung f53a0586b9 [Bugfix] Fix prompt format of GLM4V (#14539) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-13 11:37:17 +00:00
Isotr0py b1cc4dfef5 [VLM] Support loading InternVideo2.5 models as original InternVLChatModel (#14738) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-13 03:10:02 -07:00
Cyrus LeungWoosuk KwonRoger Wang
 		382403921f [VLM] Support pan-and-scan for Gemma3 multi-modal processor (#14672) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-03-13 02:23:12 -07:00
Szymon Ożóg 55211b01e8 [Bugfix] Fix chunked prefill for GGUF (#14666) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
 2025-03-13 07:19:03 +00:00
Kyle Sayers 5d043c1685 [Quant] Bamba SupportsQuant (#14698) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-03-13 04:57:05 +00:00
Kyle Sayers 36d1ccb286 [Quant] BartModel SupportsQuant (#14699) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-03-13 04:55:59 +00:00
TY-AMD 128bf75283 [BugFix][TritonMLA] Process weights after model loading for GGUF (#14555) 
Signed-off-by: TianyuanWu <Tianyuan.Wu@amd.com>
 2025-03-12 20:14:36 -07:00
Gregory Shtrasberg a94a699c3f [ROCm][FP8] Fix for adjustments needed only for fnuz (#14689) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-03-12 20:14:04 -07:00
ameyanjarlekar 4a754fcf15 [Bugfix] Missing thumbnail from NVLM-D processor (#14633) 
Signed-off-by: ameyanjarlekar <aanjarlekar@nvidia.com>
 2025-03-12 08:50:49 -07:00
Woosuk KwonRoger WangDarkLight1337
 		c0c25e25fa [Model] Add support for Gemma 3 (#14660) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-12 08:36:33 -07:00
Li, JiangandIsotr0py ff47aab056 [CPU] Upgrade CPU backend to torch-2.6 (#13381) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-03-12 10:41:13 +00:00
Pavani Majety debd6bbf09 [Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-03-12 05:13:11 +00:00
Szymon Ożóg e22ee1e7a2 [Kernel] GGUF MoE kernel (#14613) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
 2025-03-12 03:33:27 +00:00
Isotr0py e392d85831 [Core] Refactor QKVCrossParallelLinear implementation to support BNB 4-bit quantization (#14545) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-11 20:12:52 -07:00
Farzad Abdolhosseini 80e78d02ac [Model] Extend Ultravox to accept audio longer than 30s (#13631) 
Signed-off-by: Farzad Abdolhosseini <farzad@fixie.ai>
 2025-03-12 10:27:10 +08:00
Jeff Daily a1c8f3796c dynamic distpatch of fp8 kernels (#14245) 
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
 2025-03-11 10:54:56 -04:00
Isotr0py 1477ffc381 [VLM] Cleanup siglip legacy code and fix broken paligemma multimodal processor (#14602) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-11 11:27:36 +00:00
yexin(叶鑫) 70b808fe1a [Perf]:Optimize qwen2-vl to reduce cudaMemcpyAsync (#14377) 
Signed-off-by: cynthieye <987073381@qq.com>
 2025-03-11 07:39:56 +00:00
Concurrensee c982ac5722 [Bugfix] Fix FP16 overflow for DeepSeek V2 (#13232) 
Signed-off-by: Yida Wu <yida.wu@amd.com>
 2025-03-10 20:46:59 -07:00
Jinzhen Linandmgoin 90e88ab756 [Kernel] moe wna16 cuda kernel (#13321) 
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-03-10 20:12:40 -04:00
ChaunceyandDarkLight1337 92b0ce2ac7 [Bugfix][v1] fixed llava-hf/llava-1.5-7b-hf is broken on V1 (#14554) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-10 18:24:51 +00:00
Harry Mellor 3b352a2f92 Correct capitalisation: VLLM -> vLLM (#14562) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-10 16:36:21 +00:00
Roger Wang dea985aef0 [V1][Bugfix] Fix handing of second_per_grid_ts for Qwen2-VL & Qwen2.5-VL (#14548) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-10 16:03:11 +00:00
Szymon Ożóg 89cdaa83e7 [Kernel] Add more dtype support for GGUF kernels (#14043) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
 2025-03-10 07:30:04 -07:00
Isotr0py fb16eea48b [Bugfix] Revert QKVCrossParallelLinear usage in Mllama to keep BNB quantization work (#14498) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-09 04:47:45 +00:00
Aaron Pham 0b7f06b447 [Misc] add use_tqdm_on_load to reduce logs (#14407) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-03-08 05:57:46 -08:00
Isotr0py 03fe18ae0f [VLM] Add TP support for Phi-4-MM (#14453) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-08 05:57:14 -08:00
Tyler Michael Smith ca7a2d5f28 Revert "[Perf] Reduce MLA CPU overheads in V1 (#14384)" (#14471) 2025-03-07 22:18:53 -08:00
Jee Jee Li 952a074980 [Misc] Add Phi4-MM example (#14343) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-07 17:28:52 +00:00
Jinzhen Lin d0feea31c7 [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) 
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
 2025-03-07 11:53:38 -05:00
Harry Mellor f7a6bd0fa1 Fix missing kv_caches and attn_metadata in OpenVINOCausalLM (#14271) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-07 12:30:42 +00:00
Cyrus Leung 05fb6718f0 [Bugfix] Clean up multi-modal processors (#14417) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-07 10:33:38 +00:00
Luka Govedič e1744502c2 [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-03-07 05:20:16 +00:00
Lucas Wilkinson dae6896977 [Perf] Reduce MLA CPU overheads in V1 (#14384) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-03-06 19:59:14 -08:00
Tyler Michael Smith 958adce478 [Bugfix] Fix use_direct_call condition in FusedMoE layer for (#14382) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-06 14:17:21 -08:00
Tyler Michael Smith cc2f9b32c8 [Distributed] Add enable_expert_parallel arg (#14305) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-06 18:54:45 +00:00
Nicolò LucchesiandNickLucche 69ff99fdcd [Core] Optimizing cross-attention QKVParallelLinear computation (#12325) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
Co-authored-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
 2025-03-06 09:37:26 +00:00
lkchen 5d802522a7 [V1][VLM][Pixtral-HF] Support Pixtral-HF on V1 (#14275) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2025-03-06 08:58:41 +00:00
kYLeandCyrus Leung 1769928079 [Model] Update Paligemma multimodal processing with PromptUpdate (#14015) 
Signed-off-by: Kyle Huang <kylhuang@nvidia.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-03-06 08:31:38 +00:00
Ce Gao f5f7f00cd9 [Bugfix][Structured Output] Support outlines engine with reasoning outputs for DeepSeek R1 (#14114) 2025-03-06 03:49:20 +00:00
Isotr0py e17e4488bd [LoRA] Remove linear hack outside transformers backend (#14177) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-05 15:06:28 +00:00
Roger Wang ec79b67c77 [Misc][V1] Avoid using envs.VLLM_USE_V1 in mm processing (#14256) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-05 07:37:16 +00:00
Tyler Michael Smith 72c62eae5f [V1] EP/TP MoE + DP Attention (#13931) 2025-03-04 21:27:26 -08:00
Congcong Chen 0a995d5434 [Model] New model support for Phi-4-multimodal-instruct (#14119) 2025-03-04 20:57:01 -08:00
rainkertanddangshunya 0df25101d6 [Bugfix] Fix gptq_marlin for deepseek-v3 (#13750) 
Signed-off-by: dangshunya <dangshunya@baichuan-inc.com>
Co-authored-by: dangshunya <dangshunya@baichuan-inc.com>
 2025-03-05 12:25:53 +08:00
Tyler Michael Smith 4f5b059f14 Clean up unused padding_idx variables across many model definitions (#13240) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-04 21:27:00 +00:00
lkchen b3cf368d79 [V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (#14161) 2025-03-04 15:43:59 +00:00
Divakar Verma bb5b640359 [core] moe fp8 block quant tuning support (#14068) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2025-03-04 01:30:23 +00:00
Travis JohnsonandCyrus Leung c060b71408 [Model] Add support for GraniteMoeShared models (#13313) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-04 08:04:52 +08:00