iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
3,524 Commits 3 Branches 0 Tags
2385b60d8300ce730ae67d9ea945f06de9ec4e21
Commit Graph
998 Commits
Author SHA1 Message Date
Xiaoyu Zhang 4d676f0852 [Bugfix] Embedding model pooling_type equals ALL and multi input's bug (#10494) 2024-11-21 14:40:02 +00:00
Isotr0py d5ec121f95 [Model] Expose dynamic_image_size as mm_processor_kwargs for InternVL2 models (#10518) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-21 14:20:08 +00:00
Alex BrooksandCyrus Leung 1cfde82ffd [Model] Add Support for Multimodal Granite Models (#10291) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-11-21 10:46:20 +00:00
youkaichao 388ee3de66 [torch.compile] limit inductor threads and lazy import quant (#10482) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 18:36:33 -08:00
Sky Lee 343041c4c4 [model] Reduce medusa weight (#10454) 
Signed-off-by: skylee-01 <497627264@qq.com>
 2024-11-20 06:05:55 +00:00
Isotr0py ad44437ba3 [Bugfix] Fix Mamba model initialization and MLP Speculator weights loading (#10456) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-20 05:04:05 +00:00
ElizaWszola b00b33d77e [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) 
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
 2024-11-19 13:31:12 -08:00
Russell Bryant fd9f124971 [Doc] fix link for page that was renamed (#10455) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-19 09:48:30 -08:00
Patrick von Platen 11fd7ea639 [Pixtral-Large] Pixtral actually has no bias in vision-lang adapter (#10449) 2024-11-19 17:33:06 +00:00
Jee Jee Li 382b6a4852 [Misc] Avoid misleading warning messages (#10438) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-19 08:54:58 +00:00
Mengqing Cao 8c1fb50705 [Platform][Refactor] Extract func get_default_attn_backend to Platform (#10358) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2024-11-19 11:22:26 +08:00
Jee Jee Li 7eb719df13 [Bugfix]Fix Phi-3 BNB online quantization (#10417) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-19 03:21:42 +00:00
youkaichao a03ea40792 [3/N][torch.compile] consolidate custom op logging (#10399) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-18 15:14:59 -08:00
Lucas Wilkinson 96d999fbe8 [Kernel] Initial Machete W4A8 support + Refactors (#9855) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2024-11-18 12:59:29 -07:00
Angus Wang c2170a5b39 [Kernel] Explicitly specify other value in tl.load calls (#9014) 
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
 2024-11-18 11:39:40 -08:00
Yan Ma 6b2d25efc7 [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2024-11-18 11:18:05 -07:00
Michael Goin 281cc4b3cd [Model][Bugfix] Support TP for PixtralHF ViT (#10405) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-18 10:04:14 -08:00
youkaichao 7851b45196 [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-18 23:20:06 +08:00
Isotr0py e7ebb662d7 [Model] Remove transformers attention porting in VITs (#10414) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 21:45:21 +08:00
B-201 5be4e52b65 [Model][LoRA]LoRA support added for glm-4v (#10418) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
 2024-11-18 12:57:10 +00:00
Maybewuss 01aae1cc68 [Model] Remove redundant softmax when using PoolingType.STEP (#10415) 2024-11-18 10:05:36 +00:00
lkchen c7dec926f6 [VLM] Report multi_modal_placeholders in output (#10407) 
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
 2024-11-18 16:06:16 +08:00
Isotr0py c4e464333e [Misc] Add uninitialized params tracking for AutoWeightsLoader (#10327) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 09:07:46 +08:00
电脑星人 80d85c5d7b [Bugfix] Fix mrope_position_delta in non-last prefill chunk (#10403) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-17 08:50:24 +00:00
Roger Wang 643ecf7b11 [V1] Refactor model executable interface for all text-only language models (#10374) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-11-17 05:18:46 +00:00
youkaichao 4fd9375028 [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-16 18:02:14 -08:00
电脑星人 361c29e174 [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-17 02:10:00 +08:00
Sky Lee b98d89efd4 [Misc] Medusa supports custom bias (#10361) 2024-11-16 16:33:01 +00:00
rasmith 1d75472626 [BugFix] [Kernel] Fix GPU SEGV occuring in fused_moe kernel (#10385) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2024-11-16 09:55:05 +00:00
youkaichao 3a763ba0c3 [core][misc] keep compatibility for old-style classes (#10356) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-15 13:55:51 +00:00
wchen61 3d158cdc8d Add default value to avoid Falcon crash (#5363) (#10347) 
Signed-off-by: wchen61 <wchen61@foxmail.com>
 2024-11-15 08:52:20 +00:00
Cyrus Leung 2ac6d0e75b [Misc] Consolidate pooler config overrides (#10351) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-15 06:59:00 +00:00
Cyrus Leung b40cf6402e [Model] Support Qwen2 embeddings and use tags to select model tests (#10184) 2024-11-14 20:23:09 -08:00
Luka GovedičandDaniël de Kok bf2ddc6610 [bugfix] Fix static asymmetric quantization case (#10334) 
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
 2024-11-15 09:35:11 +08:00
Cyrus Leung 972112d82f [Bugfix] Fix unable to load some models (#10312) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-14 16:55:54 -08:00
Murali Andoorveedu b2e0ad3b59 [Perf] Reduce peak memory usage of llama (#10339) 
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
 2024-11-15 00:38:20 +00:00
Maximilien de BayserandFlavia Beo 4a18fd14ba Support Roberta embedding models (#9387) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-14 21:23:29 +00:00
B-201andJee Jee Li 294bf467ba [Model] Add BNB quantization support for Idefics3 (#10310) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-14 06:31:44 +00:00
youkaichao 504ac53d18 [misc] error early for old-style class (#10304) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-13 18:55:39 -08:00
Isotr0py 15bb8330aa [Bugfix] Fix tensor parallel for qwen2 classification model (#10297) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-14 10:54:59 +08:00
HoangCongDuc ac49b59d8b [Bugfix] bitsandbytes models fail to run pipeline parallel (#10200) 
Signed-off-by: Hoang Cong Duc <hoangcongducltt@gmail.com>
 2024-11-13 09:56:39 -07:00
Cyrus Leung 0b8bb86bf1 [1/N] Initial prototype for multi-modal processor (#10044) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-13 12:39:03 +00:00
B-201 d909acf9fe [Model][LoRA]LoRA support added for idefics3 (#10281) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
 2024-11-13 17:25:59 +08:00
Austin VeselkaandFurtherAI 1b886aa104 [Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944) 
Signed-off-by: FurtherAI <austin.veselka@lighton.ai>
Co-authored-by: FurtherAI <austin.veselka@lighton.ai>
 2024-11-13 08:28:13 +00:00
电脑星人 3945c82346 [Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (#10221) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-13 07:07:22 +00:00
Woosuk KwonandRoger Wang bbd3e86926 [V1] Support VLMs with fine-grained scheduling (#9871) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-11-13 04:53:13 +00:00
sroy745 b41fb9d3b1 [Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982) 
Signed-off-by: Sourashis Roy <sroy@roblox.com>
 2024-11-12 10:53:57 -08:00
Jie Fu (傅杰) 176fcb1c71 [Bugfix] Fix QwenModel argument (#10262) 
Signed-off-by: Jie Fu <jiefu@tencent.com>
 2024-11-12 16:36:51 +00:00
Jee Jee Li a838ba7254 [Misc]Fix Idefics3Model argument (#10255) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-12 13:07:11 +00:00
Isotr0py 58170d6503 [Hardware][CPU] Add embedding models support for CPU backend (#10193) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-11 08:54:28 +00:00