iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
3,568 Commits 3 Branches 0 Tags
7ea3cd7c3e9fa1db06cdf8ad1973237b061b7d64
Commit Graph
1009 Commits
Author SHA1 Message Date
Maximilien de BayserandFlavia Beo 214efc2c3c Support Cross encoder models (#10400) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-24 18:56:20 -08:00
youkaichao c055747867 [model][utils] add extract_layer_index utility function (#10599) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-23 22:22:54 -08:00
Jee Jee LiandCyrus Leung 1700c543a5 [Bugfix] Fix LoRA weight sharding (#10450) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-11-23 17:23:17 -08:00
Jee Jee Li 17d8fc1806 [bugfix] Fix example/tensorize_vllm_model tests (#10595) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-23 17:22:33 -08:00
Isotr0py 04668ebe7a [Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-23 18:12:20 +00:00
Chen Wu cfea9c04ef [Model] Fix Baichuan BNB online quantization (#10572) 
Signed-off-by: Chen Wu <cntryroa@gmail.com>
 2024-11-22 21:13:59 -08:00
youkaichao 4aba6e3d1a [core] gemma2 full context length support (#10584) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-22 20:13:54 -08:00
zixuanzhang226 948c859571 support bitsandbytes quantization with qwen model (#10549) 
Signed-off-by: Ubuntu <zixuanzhang@bytedance.com>
 2024-11-22 16:16:14 -08:00
youkaichao eebad39f26 [torch.compile] support all attention backends (#10558) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-22 14:04:42 -08:00
Isotr0py b6374e09b0 [Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-22 15:01:56 +08:00
Cyrus Leung e7a8341c7c [Bugfix] Allow token ID-only inputs in Qwen2-Audio (#10536) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-21 18:09:43 +00:00
Xiaoyu Zhang 4d676f0852 [Bugfix] Embedding model pooling_type equals ALL and multi input's bug (#10494) 2024-11-21 14:40:02 +00:00
Isotr0py d5ec121f95 [Model] Expose dynamic_image_size as mm_processor_kwargs for InternVL2 models (#10518) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-21 14:20:08 +00:00
Alex BrooksandCyrus Leung 1cfde82ffd [Model] Add Support for Multimodal Granite Models (#10291) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-11-21 10:46:20 +00:00
youkaichao 388ee3de66 [torch.compile] limit inductor threads and lazy import quant (#10482) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 18:36:33 -08:00
Sky Lee 343041c4c4 [model] Reduce medusa weight (#10454) 
Signed-off-by: skylee-01 <497627264@qq.com>
 2024-11-20 06:05:55 +00:00
Isotr0py ad44437ba3 [Bugfix] Fix Mamba model initialization and MLP Speculator weights loading (#10456) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-20 05:04:05 +00:00
ElizaWszola b00b33d77e [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) 
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
 2024-11-19 13:31:12 -08:00
Russell Bryant fd9f124971 [Doc] fix link for page that was renamed (#10455) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-19 09:48:30 -08:00
Patrick von Platen 11fd7ea639 [Pixtral-Large] Pixtral actually has no bias in vision-lang adapter (#10449) 2024-11-19 17:33:06 +00:00
Jee Jee Li 382b6a4852 [Misc] Avoid misleading warning messages (#10438) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-19 08:54:58 +00:00
Mengqing Cao 8c1fb50705 [Platform][Refactor] Extract func get_default_attn_backend to Platform (#10358) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2024-11-19 11:22:26 +08:00
Jee Jee Li 7eb719df13 [Bugfix]Fix Phi-3 BNB online quantization (#10417) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-19 03:21:42 +00:00
youkaichao a03ea40792 [3/N][torch.compile] consolidate custom op logging (#10399) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-18 15:14:59 -08:00
Lucas Wilkinson 96d999fbe8 [Kernel] Initial Machete W4A8 support + Refactors (#9855) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2024-11-18 12:59:29 -07:00
Angus Wang c2170a5b39 [Kernel] Explicitly specify other value in tl.load calls (#9014) 
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
 2024-11-18 11:39:40 -08:00
Yan Ma 6b2d25efc7 [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2024-11-18 11:18:05 -07:00
Michael Goin 281cc4b3cd [Model][Bugfix] Support TP for PixtralHF ViT (#10405) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-18 10:04:14 -08:00
youkaichao 7851b45196 [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-18 23:20:06 +08:00
Isotr0py e7ebb662d7 [Model] Remove transformers attention porting in VITs (#10414) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 21:45:21 +08:00
B-201 5be4e52b65 [Model][LoRA]LoRA support added for glm-4v (#10418) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
 2024-11-18 12:57:10 +00:00
Maybewuss 01aae1cc68 [Model] Remove redundant softmax when using PoolingType.STEP (#10415) 2024-11-18 10:05:36 +00:00
lkchen c7dec926f6 [VLM] Report multi_modal_placeholders in output (#10407) 
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
 2024-11-18 16:06:16 +08:00
Isotr0py c4e464333e [Misc] Add uninitialized params tracking for AutoWeightsLoader (#10327) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 09:07:46 +08:00
电脑星人 80d85c5d7b [Bugfix] Fix mrope_position_delta in non-last prefill chunk (#10403) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-17 08:50:24 +00:00
Roger Wang 643ecf7b11 [V1] Refactor model executable interface for all text-only language models (#10374) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-11-17 05:18:46 +00:00
youkaichao 4fd9375028 [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-16 18:02:14 -08:00
电脑星人 361c29e174 [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-17 02:10:00 +08:00
Sky Lee b98d89efd4 [Misc] Medusa supports custom bias (#10361) 2024-11-16 16:33:01 +00:00
rasmith 1d75472626 [BugFix] [Kernel] Fix GPU SEGV occuring in fused_moe kernel (#10385) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2024-11-16 09:55:05 +00:00
youkaichao 3a763ba0c3 [core][misc] keep compatibility for old-style classes (#10356) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-15 13:55:51 +00:00
wchen61 3d158cdc8d Add default value to avoid Falcon crash (#5363) (#10347) 
Signed-off-by: wchen61 <wchen61@foxmail.com>
 2024-11-15 08:52:20 +00:00
Cyrus Leung 2ac6d0e75b [Misc] Consolidate pooler config overrides (#10351) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-15 06:59:00 +00:00
Cyrus Leung b40cf6402e [Model] Support Qwen2 embeddings and use tags to select model tests (#10184) 2024-11-14 20:23:09 -08:00
Luka GovedičandDaniël de Kok bf2ddc6610 [bugfix] Fix static asymmetric quantization case (#10334) 
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
 2024-11-15 09:35:11 +08:00
Cyrus Leung 972112d82f [Bugfix] Fix unable to load some models (#10312) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-14 16:55:54 -08:00
Murali Andoorveedu b2e0ad3b59 [Perf] Reduce peak memory usage of llama (#10339) 
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
 2024-11-15 00:38:20 +00:00
Maximilien de BayserandFlavia Beo 4a18fd14ba Support Roberta embedding models (#9387) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-14 21:23:29 +00:00
B-201andJee Jee Li 294bf467ba [Model] Add BNB quantization support for Idefics3 (#10310) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-14 06:31:44 +00:00
youkaichao 504ac53d18 [misc] error early for old-style class (#10304) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-13 18:55:39 -08:00