iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
542 Commits 3 Branches 0 Tags
24cde76a152fbffde30fa2be0d08dcbad490530e
Commit Graph
70 Commits
Author SHA1 Message Date
Woosuk Kwon 24cde76a15 [Minor] Add comment on skipping rope caches (#2004) 2023-12-10 10:04:12 -08:00
Woosuk Kwon fe470ae5ad [Minor] Fix code style for baichuan (#2003) 2023-12-09 19:24:29 -08:00
Jun Gao 3a8c2381f7 Fix for KeyError on Loading LLaMA (#1978) 2023-12-09 15:59:57 -08:00
firebook 2b981012a6 Fix Baichuan2-7B-Chat (#1987) 2023-12-08 09:38:36 -08:00
Jie Liandlijie8 ebede26ebf Make InternLM follow rope_scaling in config.json (#1956) 
Co-authored-by: lijie8 <lijie8@sensetime.com>
 2023-12-07 08:32:08 -08:00
Woosuk Kwon e5452ddfd6 Normalize head weights for Baichuan 2 (#1876) 2023-11-30 20:03:58 -08:00
Woosuk Kwon 27feead2f8 Refactor Worker & InputMetadata (#1843) 2023-11-29 22:16:37 -08:00
Woosuk Kwon a9e4574261 Refactor Attention (#1840) 2023-11-29 15:37:31 -08:00
Woosuk Kwon a7b3e33078 [Fix] Fix RoPE in ChatGLM-32K (#1841) 2023-11-29 13:01:19 -08:00
Woosuk Kwon b943890484 Fix OPT param names (#1819) 2023-11-28 11:22:44 -08:00
Woosuk Kwon 7c600440f7 Fix model docstrings (#1764) 2023-11-23 23:04:44 -08:00
Woosuk Kwon cf35d8f3d7 [BugFix] Fix TP support for AWQ (#1731) 2023-11-20 21:42:45 -08:00
Simon Mo 5ffc0d13a2 Migrate linter from pylint to ruff (#1665) 2023-11-20 11:58:01 -08:00
ljss e1054247ba [Optimization] Implement fused add rmsnorm (#1667) 2023-11-18 18:18:02 -08:00
Woosuk Kwon 8d17774f92 Add AWQ support for all models (#1714) 2023-11-18 17:56:47 -08:00
maximzubkov 521b35f799 Support Microsoft Phi 1.5 (#1664) 2023-11-16 14:28:39 -08:00
Megha Agarwal b514d3c496 Revert MptConfig to MPTConfig (#1668) 2023-11-16 01:19:39 -08:00
Zhuohan Li 7076fa1c9f TP/quantization/weight loading refactor part 2 - Refactor quantized linear logic and extend quantization support to all models (#1622) 
Refactor the tensor parallelism, quantization, and weight-loading codes.

Summary of the new features enabled by this PR:
- **All models** are able to be quantized with AWQ and SqueezeLLM, and [soon GPTQ](https://github.com/vllm-project/vllm/pull/1580).
- Model loading code became much simpler.
- Support model parallelism for all MQA/GQA models when the number of key/value heads is smaller than the tensor parallel size.
 2023-11-15 22:50:41 -08:00
lirui eb825c1e74 Fix #1474 - AssertionError:assert param_slice.shape == loaded_weight.shape (#1631) 2023-11-12 15:53:12 -08:00
GoHomeToMacDonal 1a2bbc9301 ChatGLM Support (#1261) 2023-11-06 16:09:33 -08:00
Roy e7f579eb97 Support Yi model (#1567) 2023-11-06 15:26:03 -08:00
Woosuk Kwon 1fe0990023 Remove MPTConfig (#1529) 2023-11-01 15:29:05 -07:00
Woosuk Kwon aa9af07cac Fix bias in InternLM (#1501) 2023-10-29 16:24:18 -07:00
Qing 28b47d1e49 Add rope_scaling to Aquila model (#1457) 2023-10-29 04:25:21 -07:00
chooper1squeeze-ai-labWoosuk Kwon
 		1f24755bf8 Support SqueezeLLM (#1326) 
Co-authored-by: squeeze-ai-lab <squeezeailab.bair@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2023-10-21 23:14:59 -07:00
Lu Wang de89472897 Fix the issue for AquilaChat2-* models (#1339) 2023-10-13 11:51:29 -07:00
Woosuk Kwon e7c8555d06 Bump up transformers version & Remove MistralConfig (#1254) 2023-10-13 10:05:26 -07:00
Zhuohan Li b95ee898fe [Minor] Fix comment in mistral.py (#1303) 2023-10-09 19:44:37 -07:00
Zhuohan Li ba0bfd40e2 TP/quantization/weight loading refactor part 1 - Simplify parallel linear logic (#1181) 2023-10-02 15:36:09 -07:00
Zhuohan LiandJiCheng a60b353005 support sharding llama2-70b on more than 8 GPUs (#1209) 
Co-authored-by: JiCheng <247153481@qq.com>
 2023-10-02 15:26:33 -07:00
Woosuk Kwon a8e98aee0c Fix Mistral model (#1220) 2023-09-28 10:44:05 -07:00
Chris Bamfordandtimlacroix bb1ba58f06 [Mistral] Mistral-7B-v0.1 support (#1196) 
Co-authored-by: timlacroix <t@mistral.ai>
 2023-09-28 10:41:03 -07:00
Qing 7bedab5748 Add rope_scaling to Qwen (#1210) 2023-09-28 00:49:23 -07:00
Qing 28e616c4e3 fix qwen-14b model (#1173) 2023-09-27 16:33:16 -07:00
Lily LiuWing LianWoosuk Kwon
 		21877b0d75 Support Longchat and RoPE scaling (#555) 
Co-authored-by: Wing Lian <wing.lian@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2023-09-27 03:36:02 -07:00
Antoni BaumWoosuk Kwonwnma3mz
 		3302f0aef3 rope_theta and max_position_embeddings from config (#1096) 
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: wnma3mz <wnma3mz@gmail.com>
 2023-09-20 13:35:11 -07:00
Woosuk Kwon cc796b1358 Convert before transpose (#1073) 2023-09-18 11:51:48 -07:00
Woosuk KwonRobert IrvinerootCasperjulian-q
 		e3e79e9e8a Implement AWQ quantization support for LLaMA (#1032) 
Co-authored-by: Robert Irvine <robert@seamlessml.com>
Co-authored-by: root <rirv938@gmail.com>
Co-authored-by: Casper <casperbh.96@gmail.com>
Co-authored-by: julian-q <julianhquevedo@gmail.com>
 2023-09-16 00:03:37 -07:00
Jasmond LJasmond LohZhuohan Li
 		ab019eea75 Add Model Revision Support (#1014) 
Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
 2023-09-13 15:20:02 -07:00
Zhuohan Li c957c741d9 Enable safetensors loading for all models (#974) 2023-09-07 15:49:52 -07:00
Woosuk Kwon 320a622ec4 [BugFix] Implement RoPE for GPT-J (#941) 2023-09-06 11:54:33 +09:00
Zhuohan Li 002800f081 Align vLLM's beam search implementation with HF generate (#857) 2023-09-04 17:29:42 -07:00
JFDuan 0d93f15694 Accelerate LLaMA model loading (#234) 2023-08-30 01:00:13 -07:00
Antoni Baum 4b6f069b6f Add support for CodeLlama (#854) 2023-08-25 12:44:07 -07:00
Wen Sun eedac9dba0 fix: revert code to avoid no attribute problem (#827) 2023-08-22 11:55:16 -07:00
shunxing1234andftgreat ad5f2fe34c Add support for aquila (#663) 
* add aquila

Signed-off-by: ftgreat <ftgreat@163.com>

* fix some bug

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* delete pdb

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* fix bugs

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* fix bugs

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* delete whitespace

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* format

* fix order

---------

Signed-off-by: ftgreat <ftgreat@163.com>
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
Co-authored-by: ftgreat <ftgreat@163.com>
 2023-08-22 00:13:36 -07:00
zhaoyang-star 4f8584756d Fix mqa is false case in gpt_bigcode (#806) 2023-08-21 22:22:06 -07:00
WRH 462ae5220a [Fix] unwantted bias in InternLM Model (#740) 2023-08-11 11:40:37 -07:00
Jia Guoqing 735ecfff61 add internlm model (#528) 2023-08-08 16:35:06 -07:00
Qingandwq.chu a57d13cc96 add QWen-7b (#685) 
Co-authored-by: wq.chu <wq.chu@tianrang-inc.com>
 2023-08-08 13:50:38 -07:00