iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
3,814 Commits 3 Branches 0 Tags
d263bd9df7b2f5586910e5d006a11ff11ba7c310
Commit Graph
114 Commits
Author SHA1 Message Date
Cyrus Leung 8f10d5e393 [Misc] Split up pooling tasks (#10820) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-11 01:28:00 -08:00
Cyrus Leung bf0e382e16 [Model] Composite weight loading for multimodal Qwen2 (#10944) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-07 07:22:52 -07:00
Jee Jee Li 1f958a7d52 [Bugfix] Fix BNB loader target_modules (#10720) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-05 13:20:26 +08:00
Isotr0pyandCyrus Leung 4c05edb33a [Model] Add TP and BNB quantization support to LlavaMultiModalProjector (#10834) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-02 23:06:09 +00:00
Cyrus Leung 133707123e [Model] Replace embedding models with pooling adapter (#10769) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-01 08:02:54 +08:00
Jee Jee Li 15cc2a9f1a [Misc]Further reduce BNB static variable (#10597) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-26 22:54:12 -08:00
youkaichao 05d1f8c9c6 [misc] move functions to config.py (#10624) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-25 09:27:30 +00:00
Jee Jee Li 17d8fc1806 [bugfix] Fix example/tensorize_vllm_model tests (#10595) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-23 17:22:33 -08:00
Isotr0py b6374e09b0 [Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-22 15:01:56 +08:00
Russell Bryant fd9f124971 [Doc] fix link for page that was renamed (#10455) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-19 09:48:30 -08:00
Yan Ma 6b2d25efc7 [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2024-11-18 11:18:05 -07:00
Isotr0py c4e464333e [Misc] Add uninitialized params tracking for AutoWeightsLoader (#10327) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 09:07:46 +08:00
youkaichao 4fd9375028 [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-16 18:02:14 -08:00
youkaichao 3a763ba0c3 [core][misc] keep compatibility for old-style classes (#10356) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-15 13:55:51 +00:00
youkaichao 504ac53d18 [misc] error early for old-style class (#10304) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-13 18:55:39 -08:00
HoangCongDuc ac49b59d8b [Bugfix] bitsandbytes models fail to run pipeline parallel (#10200) 
Signed-off-by: Hoang Cong Duc <hoangcongducltt@gmail.com>
 2024-11-13 09:56:39 -07:00
youkaichao 1a95f10ee7 [5/N] pass the whole config to model (#9983) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-09 14:17:28 +08:00
Aaron Pham 21063c11c7 [CI/Build] drop support for Python 3.8 EOL (#8464) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2024-11-06 07:11:55 +00:00
Jee Jee Li b9c64c0ca7 [Misc] Modify BNB parameter name (#9997) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-05 14:40:08 -05:00
Jee Jee Li fb2716d641 [Misc]Reduce BNB static variable (#9987) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-04 17:04:40 +00:00
youkaichao 8d72bb20fa [4/N] make quant config first-class citizen (#9978) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-04 08:51:31 -08:00
Jee Jee Li c49f0407ba [Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-04 03:36:41 +00:00
youkaichao 3bb4befea7 [bugfix] fix tsts (#9959) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-02 15:54:05 -07:00
youkaichao cea808f325 [3/N] model runner pass the whole config to model (#9958) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-02 12:08:49 -07:00
Went-Liang 81f09cfd80 [Model] Support math-shepherd-mistral-7b-prm model (#9697) 
Signed-off-by: Went-Liang <wenteng_liang@163.com>
 2024-10-30 09:33:42 -07:00
Michael Goin bc73e9821c [Bugfix] Fix prefix strings for quantized VLMs (#9772) 2024-10-29 16:02:59 -07:00
yannicks1 0ce7798f44 [Misc]: Typo fix: Renaming classes (casualLM -> causalLM) (#9801) 
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
 2024-10-29 10:39:20 -07:00
Isotr0py 09500f7dde [Model] Add BNB quantization support for Mllama (#9720) 2024-10-29 08:20:02 -04:00
Mengqing Cao 5cbdccd151 [Hardware][openvino] is_openvino --> current_platform.is_openvino (#9716) 2024-10-26 10:59:06 +00:00
Shashwat Srijan bb76538bbd [Hardwware][Neuron] Simplify model load for transformers-neuronx library (#9380) 2024-10-17 15:39:39 -07:00
Cyrus Leung 390be74649 [Misc] Print stack trace using logger.exception (#9461) 2024-10-17 13:55:48 +00:00
Tyler Michael Smith 7342a7d7f8 [Model] Support Mamba (#6484) 2024-10-11 15:40:06 +00:00
chenqianfzh 2f4117c38e support bitsandbytes quantization with more models (#9148) 2024-10-08 19:52:19 -06:00
Chongming NiandAshraf Mahgoub cc90419e89 [Hardware][Neuron] Add on-device sampling support for Neuron (#8746) 
Co-authored-by: Ashraf Mahgoub <ashymahg@amazon.com>
 2024-10-04 16:42:20 -07:00
ElizaWszolaDipikaDipika Sikka
 		05d686432f [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) 
Co-authored-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika Sikka <ds3822@columbia.edu>
 2024-10-04 12:34:44 -06:00
Cyrus Leung 0e36fd4909 [Misc] Move registry to its own file (#9064) 2024-10-04 10:01:37 +00:00
Sergey Shlyapnikov f58d4fccc9 [OpenVINO] Enable GPU support for OpenVINO vLLM backend (#8192) 2024-10-02 17:50:01 -04:00
Divakar Verma 82f3937e59 [Misc] add process_weights_after_loading for DummyLoader (#8969) 2024-10-01 03:46:41 +00:00
Jee Jee Li 13f9f7a3d0 [[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (#8768) 2024-09-24 17:08:55 -07:00
Peter Salas 3f06bae907 [Core][Model] Support loading weights by ID within models (#7931) 2024-09-24 07:14:15 +00:00
Aaron PhamandCyrus Leung 9d104b5beb [CI/Build] Update Ruff version (#8469) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-09-18 11:00:56 +00:00
Cyrus Leung 6ffa3f314c [CI/Build] Avoid CUDA initialization (#8534) 2024-09-18 10:38:11 +00:00
chenqianfzh 9855b99502 [Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434) 2024-09-17 08:09:12 -07:00
ElizaWszolaandDipika a091e2da3e [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) 
Co-authored-by: Dipika <dipikasikka1@gmail.com>
 2024-09-16 09:47:19 -06:00
Li, Jiang 0b952af458 [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) 2024-09-11 09:46:46 -07:00
Pavani Majety efcf946a15 [Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112) 2024-09-11 00:38:40 -04:00
Dipika Sikka 6cd5e5b07e [Misc] Fused MoE Marlin support for GPTQ (#8217) 2024-09-09 23:02:52 -04:00
Cyrus Leung 9f68e00d27 [Bugfix] Fix broken OpenAI tensorizer test (#8258) 2024-09-07 08:02:39 +00:00
Patrick von PlatenandMichael Goin 29f49cd6e3 [Model] Allow loading from original Mistral format (#8168) 
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2024-09-06 17:02:05 -06:00
Harsha vardhan manoj BikkiandHarsha Bikki 008cf886c9 [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) 
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
 2024-09-04 16:33:43 -07:00