iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
4236 Commits
Author SHA1 Message Date
Farzad Abdolhosseini 80e78d02ac [Model] Extend Ultravox to accept audio longer than 30s (#13631) 
Signed-off-by: Farzad Abdolhosseini <farzad@fixie.ai>
 2025-03-12 10:27:10 +08:00
Jeff Daily a1c8f3796c dynamic distpatch of fp8 kernels (#14245) 
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
 2025-03-11 10:54:56 -04:00
Isotr0py 1477ffc381 [VLM] Cleanup siglip legacy code and fix broken paligemma multimodal processor (#14602) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-11 11:27:36 +00:00
yexin(叶鑫) 70b808fe1a [Perf]:Optimize qwen2-vl to reduce cudaMemcpyAsync (#14377) 
Signed-off-by: cynthieye <987073381@qq.com>
 2025-03-11 07:39:56 +00:00
Concurrensee c982ac5722 [Bugfix] Fix FP16 overflow for DeepSeek V2 (#13232) 
Signed-off-by: Yida Wu <yida.wu@amd.com>
 2025-03-10 20:46:59 -07:00
Jinzhen Linandmgoin 90e88ab756 [Kernel] moe wna16 cuda kernel (#13321) 
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-03-10 20:12:40 -04:00
ChaunceyandDarkLight1337 92b0ce2ac7 [Bugfix][v1] fixed llava-hf/llava-1.5-7b-hf is broken on V1 (#14554) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-10 18:24:51 +00:00
Harry Mellor 3b352a2f92 Correct capitalisation: VLLM -> vLLM (#14562) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-10 16:36:21 +00:00
Roger Wang dea985aef0 [V1][Bugfix] Fix handing of second_per_grid_ts for Qwen2-VL & Qwen2.5-VL (#14548) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-10 16:03:11 +00:00
Szymon Ożóg 89cdaa83e7 [Kernel] Add more dtype support for GGUF kernels (#14043) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
 2025-03-10 07:30:04 -07:00
Isotr0py fb16eea48b [Bugfix] Revert QKVCrossParallelLinear usage in Mllama to keep BNB quantization work (#14498) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-09 04:47:45 +00:00
Aaron Pham 0b7f06b447 [Misc] add use_tqdm_on_load to reduce logs (#14407) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-03-08 05:57:46 -08:00
Isotr0py 03fe18ae0f [VLM] Add TP support for Phi-4-MM (#14453) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-08 05:57:14 -08:00
Tyler Michael Smith ca7a2d5f28 Revert "[Perf] Reduce MLA CPU overheads in V1 (#14384)" (#14471) 2025-03-07 22:18:53 -08:00
Jee Jee Li 952a074980 [Misc] Add Phi4-MM example (#14343) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-07 17:28:52 +00:00
Jinzhen Lin d0feea31c7 [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) 
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
 2025-03-07 11:53:38 -05:00
Harry Mellor f7a6bd0fa1 Fix missing kv_caches and attn_metadata in OpenVINOCausalLM (#14271) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-07 12:30:42 +00:00
Cyrus Leung 05fb6718f0 [Bugfix] Clean up multi-modal processors (#14417) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-07 10:33:38 +00:00
Luka Govedič e1744502c2 [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-03-07 05:20:16 +00:00
Lucas Wilkinson dae6896977 [Perf] Reduce MLA CPU overheads in V1 (#14384) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-03-06 19:59:14 -08:00
Tyler Michael Smith 958adce478 [Bugfix] Fix use_direct_call condition in FusedMoE layer for (#14382) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-06 14:17:21 -08:00
Tyler Michael Smith cc2f9b32c8 [Distributed] Add enable_expert_parallel arg (#14305) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-06 18:54:45 +00:00
Nicolò LucchesiandNickLucche 69ff99fdcd [Core] Optimizing cross-attention QKVParallelLinear computation (#12325) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
Co-authored-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
 2025-03-06 09:37:26 +00:00
lkchen 5d802522a7 [V1][VLM][Pixtral-HF] Support Pixtral-HF on V1 (#14275) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2025-03-06 08:58:41 +00:00
kYLeandCyrus Leung 1769928079 [Model] Update Paligemma multimodal processing with PromptUpdate (#14015) 
Signed-off-by: Kyle Huang <kylhuang@nvidia.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-03-06 08:31:38 +00:00
Ce Gao f5f7f00cd9 [Bugfix][Structured Output] Support outlines engine with reasoning outputs for DeepSeek R1 (#14114) 2025-03-06 03:49:20 +00:00
Isotr0py e17e4488bd [LoRA] Remove linear hack outside transformers backend (#14177) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-05 15:06:28 +00:00
Roger Wang ec79b67c77 [Misc][V1] Avoid using envs.VLLM_USE_V1 in mm processing (#14256) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-05 07:37:16 +00:00
Tyler Michael Smith 72c62eae5f [V1] EP/TP MoE + DP Attention (#13931) 2025-03-04 21:27:26 -08:00
Congcong Chen 0a995d5434 [Model] New model support for Phi-4-multimodal-instruct (#14119) 2025-03-04 20:57:01 -08:00
rainkertanddangshunya 0df25101d6 [Bugfix] Fix gptq_marlin for deepseek-v3 (#13750) 
Signed-off-by: dangshunya <dangshunya@baichuan-inc.com>
Co-authored-by: dangshunya <dangshunya@baichuan-inc.com>
 2025-03-05 12:25:53 +08:00
Tyler Michael Smith 4f5b059f14 Clean up unused padding_idx variables across many model definitions (#13240) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-04 21:27:00 +00:00
lkchen b3cf368d79 [V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (#14161) 2025-03-04 15:43:59 +00:00
Divakar Verma bb5b640359 [core] moe fp8 block quant tuning support (#14068) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2025-03-04 01:30:23 +00:00
Travis JohnsonandCyrus Leung c060b71408 [Model] Add support for GraniteMoeShared models (#13313) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-04 08:04:52 +08:00
Michael Goin 19d98e0c7d [Kernel] Optimize moe intermediate_cache usage (#13625) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-03-03 16:29:53 -05:00
Michael Goin 2b04c209ee [Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (#14100) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-03-03 14:20:24 -07:00
Harry Mellor 91373a0d15 Fix head_dim not existing in all model configs (Transformers backend) (#14141) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-03 17:48:11 +00:00
Mengqing Cao b87c21fc89 [Misc][Platform] Move use allgather to platform (#14010) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-03-03 15:40:04 +08:00
wang.yuqi e584b85afd [Misc] duplicate code in deepseek_v2 (#14106) 2025-03-03 14:10:11 +08:00
Ce Gao bf33700ecd [v0][structured output] Support reasoning output (#12955) 
Signed-off-by: Ce Gao <cegao@tensorchord.ai>
 2025-03-02 14:49:42 -05:00
Jun Duan 82fbeae92b [Misc] Accurately capture the time of loading weights (#14063) 
Signed-off-by: Jun Duan <jun.duan.phd@outlook.com>
 2025-03-01 17:20:30 -08:00
Jee Jee Li cc5e8f6db8 [Model] Add LoRA support for TransformersModel (#13770) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-02 09:17:34 +08:00
Isotr0py fdcc405346 [Doc] Consolidate whisper and florence2 examples (#14050) 2025-02-28 22:49:15 -08:00
YajieWang 6a92ff93e1 [Misc][Kernel]: Add GPTQAllSpark Quantization (#12931) 2025-02-28 22:30:59 -08:00
Jee Jee Li 6a84164add [Bugfix] Add file lock for ModelScope download (#14060) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-01 06:10:28 +00:00
Luka Govedič bd56c983d6 [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-02-28 16:20:11 -07:00
Yang Liu 9b61dd41e7 [Bugfix] Initialize attention bias on the same device as Query/Key/Value for QwenVL Series (#14031) 2025-02-28 07:36:08 -08:00
Cyrus Leung f7bee5c815 [VLM][Bugfix] Enable specifying prompt target via index (#14038) 2025-02-28 07:35:55 -08:00
Jee Jee Li e0734387fb [Bugfix] Fix MoeWNA16Method activation (#14024) 2025-02-28 15:22:42 +00:00