iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,683 Commits 3 Branches 0 Tags
42fadebecb79290ad722f33f3094de23b121f33d
Commit Graph
4305 Commits
Author SHA1 Message Date
Isotr0pyandCyrus Leung a811dd6608 [Model] merged input processor for Phi-3-Vision models (#10977) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-09 12:55:10 -08:00
Roger Wang a11f326528 [V1] Initial support of multimodal models for V1 re-arch (#10699) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-12-08 12:50:51 +00:00
Cyrus Leung c889d5888b [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-07 17:20:49 +00:00
Cyrus Leung 39e227c7ae [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-07 17:10:05 +00:00
Cyrus Leung bf0e382e16 [Model] Composite weight loading for multimodal Qwen2 (#10944) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-07 07:22:52 -07:00
Cyrus LeungandRoger Wang 955fa9533a [3/N] Support and implement merged input processor for LLaVA model (#10676) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-12-07 00:50:58 -08:00
Michael Goin 8b59631855 [Core] Support Lark grammars for XGrammar (#10870) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-12-06 08:34:29 -07:00
Jee Jee Li 1f958a7d52 [Bugfix] Fix BNB loader target_modules (#10720) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-05 13:20:26 +08:00
Michael Goin 8d370e91cb [Bugfix] Fallback to outlines for complex json schemas (#10899) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-12-05 11:14:06 +08:00
Isotr0py 10398b4706 [Model] Consolidate ViTs attention implementation without mask (#10893) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-12-04 18:11:08 +00:00
Michael Goin 7090c27bb2 [Bugfix] Only require XGrammar on x86 (#10865) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-12-03 10:32:21 -08:00
Aaron Phamandmgoin 9323a3153b [Core][Performance] Add XGrammar support for guided decoding and set it as default (#10785) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
 2024-12-03 15:17:00 +08:00
Cyrus Leung 3257d449fa [Misc] Remove deprecated names (#10817) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-03 06:52:57 +00:00
youkaichao dc5ce861bf [torch.compile] remove compilation_context and simplify code (#10838) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-03 06:19:02 +00:00
zixuanzhang226 d746268e92 [Model] support bitsandbytes quantization with minicpm model (#10842) 
Signed-off-by: Ubuntu <zixuanzhang@bytedance.com>
 2024-12-03 03:06:41 +00:00
Isotr0pyandCyrus Leung 4c05edb33a [Model] Add TP and BNB quantization support to LlavaMultiModalProjector (#10834) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-02 23:06:09 +00:00
youkaichao 63a164172d [misc] remove xverse modeling file (#10814) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-02 03:27:13 +00:00
Isotr0py b18c9bbaba [Model] Add BNB support to Llava and Pixtral-HF (#10795) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-12-02 01:31:09 +00:00
Cyrus Leung d2f058e76c [Misc] Rename embedding classes to pooling (#10801) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-01 14:36:51 +08:00
Cyrus Leung 133707123e [Model] Replace embedding models with pooling adapter (#10769) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-01 08:02:54 +08:00
Patrick von Platenandyoukaichao e7cfc4ef4c [Interleaved ATTN] Support for Mistral-8B (#10591) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2024-11-30 07:45:50 +00:00
Isotr0py 16ee07f22a [Model] Refactor Molmo weights loading to use AutoWeightsLoader (#10771) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-30 04:19:14 +00:00
Jee Jee Li 3132aac043 [Bugfix] Fix Idefics3 bug (#10778) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-29 13:56:46 +00:00
wang.yuqi c82b432d4a [Misc] typo find in sampling_metadata.py (#10740) 2024-11-29 05:17:57 +00:00
Cyrus Leung fa6ecb9aa7 [Model] Clean up MiniCPMV (#10751) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-29 04:47:06 +00:00
Isotr0py c83919c7a6 [Model] Add Internlm2 LoRA support (#5064) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-28 17:29:04 +00:00
sixgodIsotr0pyCyrus Leung
 		5fc5ce0fe4 [Model] Added GLM-4 series hf format model support vllm==0.6.4 (#10561) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2024-11-28 14:53:31 +00:00
zixuanzhang226 70dc14fbd0 [Model] support bitsandbytes quantization with minicpm3 model (#10682) 
Signed-off-by: Ubuntu <zixuanzhang@bytedance.com>
 2024-11-27 23:58:02 -08:00
Cyrus Leung 9b4b150395 [Bugfix] Ignore lm_head when loading embedding models (#10719) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-27 19:05:29 +00:00
Isotr0py b98c62ba49 [Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (#10675) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-27 10:43:17 -08:00
Roger Wang 9e0a147d50 [V1] Update interface for mistral-format Pixtral (#10703) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-11-27 12:26:27 +00:00
shunxing12345xiangw2Isotr0py
 		1209261e93 [Model] Support telechat2 (#10311) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: xiangw2 <xiangw2@chinatelecom.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-11-27 11:32:35 +00:00
Jee Jee Li 15cc2a9f1a [Misc]Further reduce BNB static variable (#10597) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-26 22:54:12 -08:00
yansh97 cfb3bf25fb [bugfix] fix the default value of llm_int8_threshold in BitsAndBytesConfig (#10657) 2024-11-27 13:55:23 +08:00
Roger Wang 0a4d968500 [V1] Update interface for idefics3 (#10680) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-11-27 10:04:01 +08:00
Chendi.Xue 0a71900bc9 Remove hard-dependencies of Speculative decode to CUDA workers (#10587) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2024-11-26 17:57:11 -08:00
Roger Wang 2f0a0a17a4 [V1] Refactor model executable interface for multimodal models (#10570) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-11-26 20:46:11 +00:00
Michael Goin 7576cd38df [Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (#10642) 2024-11-26 12:29:00 -08:00
Cyrus Leung 940635343a [Misc] Remove outdated init protocols (#10655) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-26 14:55:00 +08:00
youkaichao 45ac4ff270 [bugfix] fix aria model and add torch.compile (#10645) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-25 18:32:09 -08:00
Shane A 9db713a1dc [Model] Add OLMo November 2024 model (#10503) 2024-11-25 17:26:40 -05:00
Cyrus Leung cf73f0c95e [Model] Enable optional prefix when loading embedding models (#10639) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-25 18:14:33 +00:00
zhou fanandIsotr0py b1d920531f [Model]: Add support for Aria model (#10514) 
Signed-off-by: xffxff <1247714429@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-11-25 18:10:55 +00:00
Cyrus Leung ed46f14321 [Model] Support is_causal HF config field for Qwen2 model (#10621) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-25 09:51:20 +00:00
youkaichao 05d1f8c9c6 [misc] move functions to config.py (#10624) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-25 09:27:30 +00:00
youkaichao 571841b7fc [torch.compile] support encoder based models (#10613) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-25 05:24:33 +00:00
Maximilien de BayserandFlavia Beo 214efc2c3c Support Cross encoder models (#10400) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-24 18:56:20 -08:00
youkaichao c055747867 [model][utils] add extract_layer_index utility function (#10599) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-23 22:22:54 -08:00
Jee Jee LiandCyrus Leung 1700c543a5 [Bugfix] Fix LoRA weight sharding (#10450) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-11-23 17:23:17 -08:00
Jee Jee Li 17d8fc1806 [bugfix] Fix example/tensorize_vllm_model tests (#10595) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-23 17:22:33 -08:00