iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
1981 Commits
Author SHA1 Message Date
Luka GovedičandDaniël de Kok bf2ddc6610 [bugfix] Fix static asymmetric quantization case (#10334) 
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
 2024-11-15 09:35:11 +08:00
rasmith 127c07480e [Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8 and int8 SmoothQuant, symmetric case (#9857) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2024-11-08 19:59:22 -05:00
Flávia BéoandMax de Bayser aa9078fa03 Adds method to read the pooling types from model's files (#9506) 
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2024-11-07 08:42:40 +00:00
Li, Jiang a4b3e0c1e9 [Hardware][CPU] Update torch 2.5 (#9911) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2024-11-07 04:43:08 +00:00
Michael Goin 4ab3256644 [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-07 00:54:13 +00:00
Joe Runde d58268c56a [V1] Make v1 more testable (#9888) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-11-06 11:57:35 -08:00
Michael Goin 399c798608 Remove ScaledActivation for AWQ (#10057) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-06 14:27:06 +00:00
+26
Konrad ZaworaKunshang JiSanju C SudhakaranMichal AdamczykMarceli FylcekHimangshu LahkarVivek GoelyuwenzhoDominika Olszewskabarak goldbergMichal SzutenbergJan KanieckiAgata DobrzyniewiczKrzysztof WisniewskiDudi LesterIlia TarabanChendi.XueMichał KuligowskiJakub MaksymczukTomasz ZielinskiSun ChoiIryna BoikoBob Zhuhlin99Zehao HuangAndrzej KotłowskiYan TomsinskyNir DavidYu-ZhouRuheena Suhani ShaikKarol DamaszkeMarcin SwiniarskiWoosuk KwonJacek CzajaJacek CzajaYuan
 		a02a50e6e5 [Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyniewicz-habana@users.noreply.github.com>
Co-authored-by: Krzysztof Wisniewski <kwisniewski@habana.ai>
Co-authored-by: Dudi Lester <160421192+dudilester@users.noreply.github.com>
Co-authored-by: Ilia Taraban <tarabanil@gmail.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Michał Kuligowski <mkuligowski@habana.ai>
Co-authored-by: Jakub Maksymczuk <jmaksymczuk@habana.ai>
Co-authored-by: Tomasz Zielinski <85164140+tzielinski-habana@users.noreply.github.com>
Co-authored-by: Sun Choi <schoi@habana.ai>
Co-authored-by: Iryna Boiko <iboiko@habana.ai>
Co-authored-by: Bob Zhu <41610754+czhu15@users.noreply.github.com>
Co-authored-by: hlin99 <73271530+hlin99@users.noreply.github.com>
Co-authored-by: Zehao Huang <zehao.huang@intel.com>
Co-authored-by: Andrzej Kotłowski <Andrzej.Kotlowski@intel.com>
Co-authored-by: Yan Tomsinsky <73292515+Yantom1@users.noreply.github.com>
Co-authored-by: Nir David <ndavid@habana.ai>
Co-authored-by: Yu-Zhou <yu.zhou@intel.com>
Co-authored-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
Co-authored-by: Karol Damaszke <kdamaszke@habana.ai>
Co-authored-by: Marcin Swiniarski <mswiniarski@habana.ai>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Jacek Czaja <jacek.czaja@intel.com>
Co-authored-by: Jacek Czaja <jczaja@habana.ai>
Co-authored-by: Yuan <yuan.zhou@outlook.com>
 2024-11-06 01:09:10 -08:00
Aaron Pham 21063c11c7 [CI/Build] drop support for Python 3.8 EOL (#8464) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2024-11-06 07:11:55 +00:00
Jee Jee Li b9c64c0ca7 [Misc] Modify BNB parameter name (#9997) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-05 14:40:08 -05:00
Michael Goin a53046b16f [Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-05 10:42:20 -08:00
Michael Goin 8f0a9ca890 [Bugfix] Respect modules_to_not_convert within awq_marlin (#9895) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-04 16:57:44 -07:00
Mor Zusman 603a661ae8 [Model] factoring out MambaMixer out of Jamba (#8993) 
Signed-off-by: mzusman <mor.zusmann@gmail.com>
 2024-11-04 18:00:00 +00:00
Jee Jee Li c49f0407ba [Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-04 03:36:41 +00:00
Pavani Majety 598b6d7b07 [Bugfix/Core] Flashinfer k_scale and v_scale (#9861) 2024-11-01 12:15:05 -07:00
Michael Goin 37a4947dcd [Bugfix] Fix layer skip logic with bitsandbytes (#9887) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-01 13:12:44 +08:00
youkaichao 96e0c9cbbd [torch.compile] directly register custom op (#9896) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-31 21:56:09 -07:00
Went-Liang 81f09cfd80 [Model] Support math-shepherd-mistral-7b-prm model (#9697) 
Signed-off-by: Went-Liang <wenteng_liang@163.com>
 2024-10-30 09:33:42 -07:00
Michael Goin 226688bd61 [Bugfix][VLM] Make apply_fp8_linear work with >2D input (#9812) 2024-10-29 19:49:44 -07:00
Isotr0py 09500f7dde [Model] Add BNB quantization support for Mllama (#9720) 2024-10-29 08:20:02 -04:00
youkaichao 32176fee73 [torch.compile] support moe models (#9632) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-27 21:58:04 -07:00
wangshuai09 4e2d95e372 [Hardware][ROCM] using current_platform.is_rocm (#9642) 
Signed-off-by: wangshuai09 <391746016@qq.com>
 2024-10-28 04:07:00 +00:00
kakao-kevin-usandKevin-Yang 6650e6a930 [Model] Add classification Task with Qwen2ForSequenceClassification (#9704) 
Signed-off-by: Kevin-Yang <ykcha9@gmail.com>
Co-authored-by: Kevin-Yang <ykcha9@gmail.com>
 2024-10-26 17:53:35 +00:00
Michael Goin ca0d92227e [Bugfix] Fix compressed_tensors_moe bad config.strategy (#9677) 2024-10-25 12:40:33 -07:00
Charlie Fu 59449095ab [Performance][Kernel] Fused_moe Performance Improvement (#9384) 
Signed-off-by: charlifu <charlifu@amd.com>
 2024-10-24 15:37:52 -07:00
Jee Jee Li 295a061fb3 [Kernel] add kernel for FATReLU (#9610) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-10-24 16:18:27 +08:00
Cyrus LeungandIsotr0py c18e1a3418 [VLM] Enable overriding whether post layernorm is used in vision encoder + fix quant args (#9217) 
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-10-23 11:27:37 +00:00
Woosuk Kwon 6c5af09b39 [V1] Implement vLLM V1 [1/N] (#9289) 2024-10-22 01:24:07 -07:00
Dhia Eddine Rhaiem f6b97293aa [Model] FalconMamba Support (#9325) 2024-10-21 12:50:16 -04:00
Michael Goin 3921a2f29e [Model] Support Pixtral models in the HF Transformers format (#9036) 2024-10-18 13:29:56 -06:00
Dipika Sikka 48138a8415 [BugFix] Stop silent failures on compressed-tensors parsing (#9381) 2024-10-17 18:54:00 -07:00
Robert ShawAndrew Feldmanafeldman-nmWoosuk KwonlaishzhMax de BayserMax de BayserCyrus Leung
 		343f8e0905 Support BERTModel (first encoder-only embedding model) (#9056) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: laishzh <laishengzhang@gmail.com>
Co-authored-by: Max de Bayser <maxdebayser@gmail.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2024-10-17 23:21:01 +00:00
bnellnm eca2c5f7c0 [Bugfix] Fix support for dimension like integers and ScalarType (#9299) 2024-10-17 19:08:34 +00:00
Luka Govedič 0f41fbe5a3 [torch.compile] Fine-grained CustomOp enabling mechanism (#9300) 2024-10-17 18:36:37 +00:00
Lucas Wilkinson e312e52b44 [Kernel] Add Exllama as a backend for compressed-tensors (#9395) 2024-10-17 09:48:26 -04:00
Russell Bryant 776dbd74f1 [CI/Build] mypy: Resolve some errors from checking vllm/engine (#9267) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-10-16 22:55:59 +00:00
Junhao Li 5b8a1fde84 [Model][Bugfix] Add FATReLU activation and support for openbmb/MiniCPM-S-1B-sft (#9396) 2024-10-16 16:40:24 +00:00
Mor Zusman fb60ae9b91 [Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189) 2024-10-16 12:12:43 -04:00
Cyrus Leung 7e7eae338d [Misc] Standardize RoPE handling for Qwen2-VL (#9250) 2024-10-16 13:56:17 +08:00
Michael GoinandDarkLight1337 22f8a69549 [Misc] Directly use compressed-tensors for checkpoint definitions (#8909) 
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-10-15 15:40:25 -07:00
Burkhard Ringlein f710090d8e [Kernel] adding fused moe kernel config for L40S TP4 (#9245) 2024-10-11 08:54:22 -07:00
youkaichaoandBrendan Wong cbc2ef5529 [misc] hide best_of from engine (#9261) 
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
 2024-10-10 21:30:44 -07:00
Isotr0py 07c11cf4d4 [Bugfix] Fix lm_head weights tying with lora for llama (#9227) 2024-10-10 21:11:56 +08:00
Li, Jiang ca77dd7a44 [Hardware][CPU] Support AWQ for CPU backend (#7515) 2024-10-09 10:28:08 -06:00
chenqianfzh 2f4117c38e support bitsandbytes quantization with more models (#9148) 2024-10-08 19:52:19 -06:00
Cyrus LeungRoger WangIsotr0py
 		151ef4efd2 [Model] Support NVLM-D and fix QK Norm in InternViT (#9045) 
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2024-10-07 11:55:12 +00:00
Isotr0py f19da64871 [Core] Refactor GGUF parameters packing and forwarding (#8859) 2024-10-07 10:01:46 +00:00
youkaichao 18b296fdb2 [core] remove beam search from the core (#9105) 2024-10-07 05:47:04 +00:00
ElizaWszolaDipikaDipika Sikka
 		05d686432f [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) 
Co-authored-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika Sikka <ds3822@columbia.edu>
 2024-10-04 12:34:44 -06:00
Divakar Verma 01843c89b8 [Misc] log when using default MoE config (#8971) 2024-10-03 04:31:07 +00:00