|
d6545ad22e
|
add option to shorten prompt print in log (#991)
Signed-off-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
|
2023-09-12 15:10:14 -07:00 |
|
|
Woosuk Kwon
|
90eb3f43ca
|
Bump up the version to v0.1.7 (#1013)
|
2023-09-11 00:54:30 -07:00 |
|
|
Woosuk KwonandOne
|
e67b4f2c2a
|
Use FP32 in RoPE initialization (#1004)
Co-authored-by: One <imone@tuta.io>
|
2023-09-11 00:26:35 -07:00 |
|
|
Kyujin Cho
|
898285c9bf
|
fix: CUDA error when inferencing with Falcon-40B base model (#992)
|
2023-09-10 01:39:02 -07:00 |
|
|
Antoni Baum
|
a62de9ecfd
|
Fix wrong dtype in PagedAttentionWithALiBi bias (#996)
---------
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
|
2023-09-09 14:58:35 -07:00 |
|
|
Jingru
|
4042d192f5
|
fix "tansformers_module" ModuleNotFoundError when load model with trust_remote_code=True (#871)
|
2023-09-08 17:21:30 -07:00 |
|
|
Zhuohan Li
|
1117aa1411
|
Bump up the version to v0.1.6 (#989)
|
2023-09-08 00:07:46 -07:00 |
|
|
Antoni BaumandZhuohan Li
|
080438477f
|
Start background task in AsyncLLMEngine.generate (#988)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
|
2023-09-08 00:03:39 -07:00 |
|
|
Robert IrvineandRobert Irvine
|
4b5bcf8906
|
faster startup of vLLM (#982)
* update
---------
Co-authored-by: Robert Irvine <robert@seamlessml.com>
|
2023-09-08 14:48:54 +09:00 |
|
|
Woosuk Kwon
|
852ef5b4f5
|
Bump up the version to v0.1.5 (#944)
|
2023-09-07 16:15:31 -07:00 |
|
|
Zhuohan Li
|
c957c741d9
|
Enable safetensors loading for all models (#974)
|
2023-09-07 15:49:52 -07:00 |
|
|
Antoni BaumandAvnish Narayan
|
c07ece5ca4
|
Make AsyncLLMEngine more robust & fix batched abort (#969)
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
Co-authored-by: Avnish Narayan <38871737+avnishn@users.noreply.github.com>
|
2023-09-07 13:43:45 -07:00 |
|
|
Antoni Baum
|
005ba458b5
|
Set torch default dtype in a context manager (#971)
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
|
2023-09-07 15:39:37 +09:00 |
|
|
Woosuk Kwon
|
320a622ec4
|
[BugFix] Implement RoPE for GPT-J (#941)
|
2023-09-06 11:54:33 +09:00 |
|
|
Antoni Baum
|
c9927c1a6a
|
Use queue for finished requests (#957)
|
2023-09-05 19:27:23 -07:00 |
|
|
Wen Sun
|
22379d5513
|
fix: typo (#948)
|
2023-09-04 23:22:30 -07:00 |
|
|
Antoni Baum
|
1696725879
|
Initialize AsyncLLMEngine bg loop correctly (#943)
|
2023-09-04 17:41:22 -07:00 |
|
|
Zhuohan Li
|
002800f081
|
Align vLLM's beam search implementation with HF generate (#857)
|
2023-09-04 17:29:42 -07:00 |
|
|
Nelson Liu
|
e15932bb60
|
Only emit warning about internal tokenizer if it isn't being used (#939)
|
2023-09-05 00:50:55 +09:00 |
|
|
Antoni Baum
|
ce741ba3e4
|
Refactor AsyncLLMEngine (#880)
|
2023-09-03 21:43:43 -07:00 |
|
|
Dong-Yong Leeanddongyong-lee
|
e11222333f
|
fix: bug fix when penalties are negative (#913)
Co-authored-by: dongyong-lee <dongyong.lee@navercorp.com>
|
2023-09-01 00:37:17 +09:00 |
|
|
Aman Gupta Karmani
|
28873a2799
|
Improve _prune_hidden_states micro-benchmark (#707)
|
2023-08-31 13:28:43 +09:00 |
|
|
JFDuan
|
0d93f15694
|
Accelerate LLaMA model loading (#234)
|
2023-08-30 01:00:13 -07:00 |
|
|
lplcor
|
becd7a56f1
|
Enable request body OpenAPI spec for OpenAI endpoints (#865)
|
2023-08-29 21:54:08 -07:00 |
|
|
Aman Gupta Karmani
|
75471386de
|
use flash-attn via xformers (#877)
|
2023-08-29 21:52:13 -07:00 |
|
|
Zhuohan Li
|
d2b2eed67c
|
[Fix] Fix a condition for ignored sequences (#867)
|
2023-08-27 23:00:56 -07:00 |
|
|
Antoni Baum
|
4b6f069b6f
|
Add support for CodeLlama (#854)
|
2023-08-25 12:44:07 -07:00 |
|
|
Woosuk Kwon
|
791d79de32
|
Bump up the version to v0.1.4 (#846)
|
2023-08-25 12:28:00 +09:00 |
|
|
Woosuk Kwon
|
94d2f59895
|
Set replacement=True in torch.multinomial (#858)
|
2023-08-25 12:22:01 +09:00 |
|
|
wenjun93
|
75c0ca9d43
|
Clean up code (#844)
|
2023-08-23 16:44:15 -07:00 |
|
|
Woosuk Kwon
|
2a4ec90854
|
Fix for breaking changes in xformers 0.0.21 (#834)
|
2023-08-23 17:44:21 +09:00 |
|
|
Woosuk Kwon
|
d64bf1646c
|
Implement approximate GELU kernels (#828)
|
2023-08-23 07:43:21 +09:00 |
|
|
Wen Sun
|
eedac9dba0
|
fix: revert code to avoid no attribute problem (#827)
|
2023-08-22 11:55:16 -07:00 |
|
|
shunxing1234andftgreat
|
ad5f2fe34c
|
Add support for aquila (#663)
* add aquila
Signed-off-by: ftgreat <ftgreat@163.com>
* fix some bug
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
* delete pdb
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
* fix bugs
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
* fix bugs
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
* delete whitespace
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
* format
* fix order
---------
Signed-off-by: ftgreat <ftgreat@163.com>
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
Co-authored-by: ftgreat <ftgreat@163.com>
|
2023-08-22 00:13:36 -07:00 |
|
|
zhaoyang-star
|
4f8584756d
|
Fix mqa is false case in gpt_bigcode (#806)
|
2023-08-21 22:22:06 -07:00 |
|
|
wangcx18
|
0c04ce3234
|
Fix typo in sampling_params.py (#788)
|
2023-08-18 10:12:46 +09:00 |
|
|
Xinyu Yang
|
73b3de79ea
|
explicitly del state (#784)
|
2023-08-17 12:56:04 -07:00 |
|
|
Abraham-Xu
|
d1744376ae
|
Align with huggingface Top K sampling (#753)
|
2023-08-15 16:44:33 -07:00 |
|
|
Ikko Eltociear Ashimine
|
805de738f6
|
Fix typo in tokenizer.py (#750)
conjuction -> conjunction
|
2023-08-14 22:26:36 -07:00 |
|
|
WanMok
|
e06f504a76
|
Supports tokens and arrays of tokens as inputs to the OpenAI completion API (#715)
|
2023-08-11 12:14:34 -07:00 |
|
|
WRH
|
462ae5220a
|
[Fix] unwantted bias in InternLM Model (#740)
|
2023-08-11 11:40:37 -07:00 |
|
|
Nicolas Basile
|
66c54aa9c3
|
Check the max prompt length for the OpenAI completions API (#472)
|
2023-08-08 17:43:49 -07:00 |
|
|
Jia Guoqing
|
735ecfff61
|
add internlm model (#528)
|
2023-08-08 16:35:06 -07:00 |
|
|
Qingandwq.chu
|
a57d13cc96
|
add QWen-7b (#685)
Co-authored-by: wq.chu <wq.chu@tianrang-inc.com>
|
2023-08-08 13:50:38 -07:00 |
|
|
Wen Sun
|
621980bdc0
|
fix: incorrect bigcode attention heads num (#676)
|
2023-08-04 10:35:22 -07:00 |
|
|
Zhuohan Li
|
aa84c92ef6
|
Bump up version to 0.1.3 (#657)
|
2023-08-02 16:46:53 -07:00 |
|
|
Zhuohan Li
|
f7389f4763
|
[Doc] Add Baichuan 13B to supported models (#656)
|
2023-08-02 16:45:12 -07:00 |
|
|
Woosuk Kwon
|
55fe8a81ec
|
Refactor scheduler (#658)
|
2023-08-02 16:42:01 -07:00 |
|
|
YHPeterandhao.yu
|
e8ddc08ec8
|
[BUG FIX] upgrade fschat version to 0.2.23 (#650)
Co-authored-by: hao.yu <hao.yu@cn-c017.server.mila.quebec>
|
2023-08-02 14:05:59 -07:00 |
|
|
Zhuohan Li
|
1b0bd0fe8a
|
Add Falcon support (new) (#592)
|
2023-08-02 14:04:39 -07:00 |
|