Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

3302f0aef3 rope_theta and max_position_embeddings from config (#1096)
Antoni Baum Woosuk Kwon wnma3mz
2023-09-20 13:35:11 -07:00
6f2dd6c37e Add documentation to Triton server tutorial (#983) Tanmay Verma 2023-09-20 10:32:40 -07:00
bc0644574c Add gpu_memory_utilization and swap_space to LLM (#1090) Woosuk Kwon 2023-09-19 22:16:04 -07:00
400b8289f7 Add pyarrow to dependencies & Print warning on Ray import error (#1094) Woosuk Kwon 2023-09-18 22:36:17 -07:00
c1026311b5 [Community] Add vLLM Discord server (#1086) Zhuohan Li 2023-09-18 12:23:35 -07:00
2b1c116b5a Add minimum capability requirement for AWQ (#1064) Woosuk Kwon 2023-09-18 12:02:01 -07:00
cc796b1358 Convert before transpose (#1073) Woosuk Kwon 2023-09-18 11:51:48 -07:00
f029ef94d7 Fix get_max_num_running_seqs for waiting and swapped seq groups (#1068) Zhuohan Li 2023-09-18 11:49:40 -07:00
95592fa00a align llm_engine and async_engine. (#1081) Roy 2023-09-19 02:49:10 +08:00
fbe66e1d0b added support for quantize on LLM module (#1080) orellavie1212 2023-09-18 21:04:21 +03:00
90979c38f8 [FIX] Don't initialize parameter by default (#1067) Zhuohan Li 2023-09-17 17:15:38 -07:00
e21d7687a9 Fix hanging when prompt exceeds limit (#1029) 陈序 2023-09-17 16:48:56 +08:00
ff36139ffc Remove AsyncLLMEngine busy loop, shield background task (#1059) Antoni Baum 2023-09-17 00:29:08 -07:00
e3e79e9e8a Implement AWQ quantization support for LLaMA (#1032)
Woosuk Kwon Robert Irvine root Casper julian-q
2023-09-16 00:03:37 -07:00
b9fe4616f9 Abort when coroutine is cancelled (#1020) Jerry Yang 2023-09-15 08:40:18 +08:00
64ca424e75 Fix warning message on LLaMA FastTokenizer (#1037) Woosuk Kwon 2023-09-14 17:33:32 -07:00
b5f93d0631 Only fail if logit_bias has actual values (#1045) Lukas Kreussel 2023-09-15 02:33:01 +02:00
a58936966f Add pandas to requirements.txt (#1047) Woosuk Kwon 2023-09-14 17:31:38 -07:00
dd54a4b026 Fix detokenization leaving special tokens (#1044) Antoni Baum 2023-09-14 16:37:03 -07:00
eda1a7cad3 Announce paper release (#1036) Woosuk Kwon 2023-09-13 17:38:13 -07:00
f04908cae7 [FIX] Minor bug fixes (#1035) Zhuohan Li 2023-09-13 16:38:12 -07:00
ab019eea75 Add Model Revision Support (#1014)
Jasmond L Jasmond Loh Zhuohan Li
2023-09-14 06:20:02 +08:00
9841d48a10 Use TGI-like incremental detokenization (#984) Antoni Baum 2023-09-13 13:38:01 -07:00
3272d7a0b7 Fix typo in README.md (#1033) Ikko Eltociear Ashimine 2023-09-14 04:55:23 +09:00
0bb1e885a0 Make max_model_len configurable (#972) Antoni Baum 2023-09-12 16:29:19 -07:00
d6545ad22e add option to shorten prompt print in log (#991)
leiwen83 Lei Wen Zhuohan Li
2023-09-13 06:10:14 +08:00
90eb3f43ca Bump up the version to v0.1.7 (#1013) Woosuk Kwon 2023-09-11 00:54:30 -07:00
e67b4f2c2a Use FP32 in RoPE initialization (#1004) Woosuk KwonandOne 2023-09-11 00:26:35 -07:00
d6770d1f23 Update setup.py (#1006) Woosuk Kwon 2023-09-10 23:42:45 -07:00
b9cecc2635 [Docs] Update installation page (#1005) Woosuk Kwon 2023-09-10 14:23:31 -07:00
898285c9bf fix: CUDA error when inferencing with Falcon-40B base model (#992) Kyujin Cho 2023-09-10 17:39:02 +09:00
a62de9ecfd Fix wrong dtype in PagedAttentionWithALiBi bias (#996) Antoni Baum 2023-09-09 14:58:35 -07:00
4042d192f5 fix "tansformers_module" ModuleNotFoundError when load model with trust_remote_code=True (#871) Jingru 2023-09-09 08:21:30 +08:00
1117aa1411 Bump up the version to v0.1.6 (#989) Zhuohan Li 2023-09-08 00:07:46 -07:00
080438477f Start background task in AsyncLLMEngine.generate (#988) Antoni BaumandZhuohan Li 2023-09-08 00:03:39 -07:00
4b5bcf8906 faster startup of vLLM (#982) Robert IrvineandRobert Irvine 2023-09-08 06:48:54 +01:00
852ef5b4f5 Bump up the version to v0.1.5 (#944) Woosuk Kwon 2023-09-08 08:15:31 +09:00
db09d4ad83 [FIX] Fix Alibi implementation in PagedAttention kernel (#945)
Zhuohan Li Woosuk Kwon Oliver-ss
2023-09-07 15:53:14 -07:00
c957c741d9 Enable safetensors loading for all models (#974) Zhuohan Li 2023-09-07 15:49:52 -07:00
c07ece5ca4 Make AsyncLLMEngine more robust & fix batched abort (#969) Antoni BaumandAvnish Narayan 2023-09-07 13:43:45 -07:00
7a9c20c715 Bum up transformers version (#976) Woosuk Kwon 2023-09-08 05:15:53 +09:00
005ba458b5 Set torch default dtype in a context manager (#971) Antoni Baum 2023-09-06 23:39:37 -07:00
320a622ec4 [BugFix] Implement RoPE for GPT-J (#941) Woosuk Kwon 2023-09-06 11:54:33 +09:00
c9927c1a6a Use queue for finished requests (#957) Antoni Baum 2023-09-05 19:27:23 -07:00
fbd80ad409 Clean up kernel unit tests (#938) Woosuk Kwon 2023-09-06 08:57:38 +09:00
22379d5513 fix: typo (#948) Wen Sun 2023-09-05 14:22:30 +08:00
1696725879 Initialize AsyncLLMEngine bg loop correctly (#943) Antoni Baum 2023-09-04 17:41:22 -07:00
002800f081 Align vLLM's beam search implementation with HF generate (#857) Zhuohan Li 2023-09-04 17:29:42 -07:00
e15932bb60 Only emit warning about internal tokenizer if it isn't being used (#939) Nelson Liu 2023-09-04 08:50:55 -07:00
ce741ba3e4 Refactor AsyncLLMEngine (#880) Antoni Baum 2023-09-03 21:43:43 -07:00
bf87484efa [BugFix] Fix NaN errors in paged attention kernel (#936) Woosuk Kwon 2023-09-04 09:20:06 +09:00
8ce9c50d40 Avoid compiling kernels for double data type (#933) Woosuk Kwon 2023-09-02 14:59:47 +09:00
32b6816e55 Add tests for models (#922) Woosuk Kwon 2023-09-01 11:19:43 +09:00
c128d69856 Fix README.md Link (#927) Zhuohan Li 2023-08-31 17:18:34 -07:00
55b28b1eee [Docs] Minor fixes in supported models (#920) Woosuk KwonandZhuohan Li 2023-09-01 08:28:39 +09:00
e11222333f fix: bug fix when penalties are negative (#913) Dong-Yong Leeanddongyong-lee 2023-09-01 00:37:17 +09:00
28873a2799 Improve _prune_hidden_states micro-benchmark (#707) Aman Gupta Karmani 2023-08-31 00:28:43 -04:00
0080d8329d Add acknowledgement to a16z grant Zhuohan Li 2023-08-30 02:17:27 -07:00
0d93f15694 Accelerate LLaMA model loading (#234) JFDuan 2023-08-30 16:00:13 +08:00
becd7a56f1 Enable request body OpenAPI spec for OpenAI endpoints (#865) lplcor 2023-08-29 21:54:08 -07:00
75471386de use flash-attn via xformers (#877) Aman Gupta Karmani 2023-08-30 00:52:13 -04:00
d2b2eed67c [Fix] Fix a condition for ignored sequences (#867) Zhuohan Li 2023-08-27 23:00:56 -07:00
4b6f069b6f Add support for CodeLlama (#854) Antoni Baum 2023-08-25 12:44:07 -07:00
791d79de32 Bump up the version to v0.1.4 (#846) Woosuk Kwon 2023-08-25 12:28:00 +09:00
94d2f59895 Set replacement=True in torch.multinomial (#858) Woosuk Kwon 2023-08-25 12:22:01 +09:00
75c0ca9d43 Clean up code (#844) wenjun93 2023-08-24 07:44:15 +08:00
2a4ec90854 Fix for breaking changes in xformers 0.0.21 (#834) Woosuk Kwon 2023-08-23 17:44:21 +09:00
85ebcda94d Fix typo of Aquila in README.md (#836) ldwang 2023-08-23 11:48:36 +08:00
d64bf1646c Implement approximate GELU kernels (#828) Woosuk Kwon 2023-08-23 07:43:21 +09:00
a41c20435e Add compute capability 8.9 to default targets (#829) Woosuk Kwon 2023-08-23 07:28:38 +09:00
eedac9dba0 fix: revert code to avoid no attribute problem (#827) Wen Sun 2023-08-23 02:55:16 +08:00
14f9c72bfd Update Supported Model List (#825) Zhuohan Li 2023-08-22 11:51:44 -07:00
ad5f2fe34c Add support for aquila (#663) shunxing1234andftgreat 2023-08-22 15:13:36 +08:00
4f8584756d Fix mqa is false case in gpt_bigcode (#806) zhaoyang-star 2023-08-22 13:22:06 +08:00
65fc1c3127 set default coompute capability according to cuda version (#773) Xudong Zhang 2023-08-22 07:05:44 +08:00
c393af6cd7 [Feature | CI] Added a github action to build wheels (#746) Daniel 2023-08-21 10:59:15 +03:00
0c04ce3234 Fix typo in sampling_params.py (#788) wangcx18 2023-08-18 09:12:46 +08:00
73b3de79ea explicitly del state (#784) Xinyu Yang 2023-08-18 03:56:04 +08:00
d1744376ae Align with huggingface Top K sampling (#753) Abraham-Xu 2023-08-16 07:44:33 +08:00
805de738f6 Fix typo in tokenizer.py (#750) Ikko Eltociear Ashimine 2023-08-15 14:26:36 +09:00
1b151ed181 Fix baichuan doc style (#748) Uranus 2023-08-14 11:57:31 +08:00
e06f504a76 Supports tokens and arrays of tokens as inputs to the OpenAI completion API (#715) WanMok 2023-08-11 12:14:34 -07:00
462ae5220a [Fix] unwantted bias in InternLM Model (#740) WRH 2023-08-12 02:40:37 +08:00
66c54aa9c3 Check the max prompt length for the OpenAI completions API (#472) Nicolas Basile 2023-08-08 17:43:49 -07:00
735ecfff61 add internlm model (#528) Jia Guoqing 2023-08-09 07:35:06 +08:00
a57d13cc96 add QWen-7b (#685) Qingandwq.chu 2023-08-09 04:50:38 +08:00
79af7e96a0 [OPTIMIZATION] Optimizes the single_query_cached_kv_attention kernel (#420) Dean Leitersdorf 2023-08-04 20:57:29 +03:00
621980bdc0 fix: incorrect bigcode attention heads num (#676) Wen Sun 2023-08-05 01:35:22 +08:00
aa84c92ef6 Bump up version to 0.1.3 (#657) Zhuohan Li 2023-08-02 16:46:53 -07:00
f7389f4763 [Doc] Add Baichuan 13B to supported models (#656) Zhuohan Li 2023-08-02 16:45:12 -07:00
55fe8a81ec Refactor scheduler (#658) Woosuk Kwon 2023-08-02 16:42:01 -07:00
e8ddc08ec8 [BUG FIX] upgrade fschat version to 0.2.23 (#650) YHPeterandhao.yu 2023-08-02 17:05:59 -04:00
1b0bd0fe8a Add Falcon support (new) (#592) Zhuohan Li 2023-08-02 14:04:39 -07:00
20044cab7a Fix log message in scheduler (#652) Lily Liu 2023-08-02 13:35:10 -07:00
64f23c2900 fix baichuan for different position embedding for 7b and 13b models (#643) Song 2023-08-02 13:22:51 +08:00
d4c7755ca8 fix biachuan-7b tp (#598) Qingandwq.chu 2023-08-02 06:41:36 +08:00
aa39e42c5a fix doc (#622) Chaofan Lin 2023-08-01 04:11:57 +08:00
953f28cf9a fix ModuleNotFoundError (#599) Fang liandfangli 2023-07-30 11:52:41 +08:00
c0d00f5be6 [Fix] fix import error of RayWorker (#604) (#605) Xudong Zhang 2023-07-28 14:37:40 +08:00
58a072be15 [Fix] Add model sequence length into model config (#575) Zhuohan Li 2023-07-25 23:46:30 -07:00

... 229 230 231 232 233 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86