 Hanzhi Zhou
|
380170038e
|
Implement custom all reduce kernels (#2192)
|
2024-01-27 12:46:35 -08:00 |
|
|
Simon Mo
|
3a7dd7e367
|
Support Batch Completion in Server (#2529)
|
2024-01-24 17:11:07 -08:00 |
|
|
Federico Galatolo
|
f1f6cc10c7
|
Added include_stop_str_in_output and length_penalty parameters to OpenAI API (#2562)
|
2024-01-24 10:21:56 -08:00 |
|
|
|
9b945daaf1
|
[Experimental] Add multi-LoRA support (#1804)
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
|
2024-01-23 15:26:37 -08:00 |
|
|
Erfan Al-Hossami
|
9c1352eb57
|
[Feature] Simple API token authentication and pluggable middlewares (#1106)
|
2024-01-23 15:13:00 -08:00 |
|
|
Jannis Schönleber
|
71d63ed72e
|
migrate pydantic from v1 to v2 (#2531)
|
2024-01-21 16:05:56 -08:00 |
|
|
Simon Mo
|
dd7e8f5f64
|
refactor complemention api for readability (#2499)
|
2024-01-18 16:45:14 -08:00 |
|
|
|
d10f8e1d43
|
[Experimental] Prefix Caching Support (#1669)
Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
|
2024-01-17 16:32:10 -08:00 |
|
|
FlorianJoncour
|
14cc317ba4
|
OpenAI Server refactoring (#2360)
|
2024-01-16 21:33:14 -08:00 |
|
|
Chirag Jain
|
ce036244c9
|
Allow setting fastapi root_path argument (#2341)
|
2024-01-12 10:59:59 -08:00 |
|
|
Iskren Ivov Chernev
|
d0215a58e7
|
Ensure metrics are logged regardless of requests (#2347)
|
2024-01-05 05:24:42 -08:00 |
|
|
Ronen Schaffer
|
74d8d77626
|
Remove unused const TIMEOUT_TO_PREVENT_DEADLOCK (#2321)
|
2024-01-03 15:49:07 -08:00 |
|
|
Harry Mellor
|
08133c4d1a
|
Add SSL arguments to API servers (#2109)
|
2023-12-18 10:56:23 +08:00 |
|
|
Woosuk Kwon
|
30fb0956df
|
[Minor] Add more detailed explanation on quantization argument (#2145)
|
2023-12-17 01:56:16 -08:00 |
|
|
|
37ca558103
|
Optimize model execution with CUDA graph (#1926)
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
|
2023-12-16 21:12:08 -08:00 |
|
|
CHU Tianxiang
|
0fbfc4b81b
|
Add GPTQ support (#916)
|
2023-12-15 03:04:22 -08:00 |
|
|
Simon Mo
|
2e8fc0d4c3
|
Fix completion API echo and logprob combo (#1992)
|
2023-12-10 13:20:30 -08:00 |
|
|
Jin Shang
|
1aa1361510
|
Fix OpenAI server completion_tokens referenced before assignment (#1996)
|
2023-12-09 21:01:21 -08:00 |
|
|
Roy
|
60dc62dc9e
|
add custom server params (#1868)
|
2023-12-03 12:59:18 -08:00 |
|
|
Simon Mo
|
5313c2cb8b
|
Add Production Metrics in Prometheus format (#1890)
|
2023-12-02 16:37:44 -08:00 |
|
|
Adam Brusselback
|
66785cc05c
|
Support chat template and echo for chat API (#1756)
|
2023-11-30 16:43:13 -08:00 |
|
|
Michael McCullochandMichael McCulloch
|
c782195662
|
Disable Logs Requests should Disable Logging of requests. (#1779)
Co-authored-by: Michael McCulloch <mjm.gitlab@fastmail.com>
|
2023-11-29 21:50:02 -08:00 |
|
|
Yunmo Chen
|
665cbcec4b
|
Added echo function to OpenAI API server. (#1504)
|
2023-11-26 21:29:17 -08:00 |
|
|
Simon Mo
|
5ffc0d13a2
|
Migrate linter from pylint to ruff (#1665)
|
2023-11-20 11:58:01 -08:00 |
|
|
liuyhwangyh
|
edb305584b
|
Support download models from www.modelscope.cn (#1588)
|
2023-11-17 20:38:31 -08:00 |
|
|
Iskren Ivov Chernev
|
686f5e3210
|
Return usage for openai streaming requests (#1663)
|
2023-11-16 15:28:36 -08:00 |
|
|
Fluder-Paradyne
|
7e90a2d117
|
Add /health Endpoint for both Servers (#1540)
|
2023-11-01 10:29:44 -07:00 |
|
|
Dan Lord
|
7013a80170
|
Add support for spaces_between_special_tokens
|
2023-10-30 16:52:56 -07:00 |
|
|
Yunfeng BaiandZhuohan Li
|
09ff7f106a
|
API server support ipv4 / ipv6 dualstack (#1288)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
|
2023-10-07 15:15:54 -07:00 |
|
|
Antoni Baum
|
acbed3ef40
|
Use monotonic time where appropriate (#1249)
|
2023-10-02 19:22:05 -07:00 |
|
|
Federico CassanoandZhuohan Li
|
66d18a7fb0
|
add support for tokenizer revision (#1163)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
|
2023-10-02 19:19:46 -07:00 |
|
|
Woosuk Kwon
|
f936657eb6
|
Provide default max model length (#1224)
|
2023-09-28 14:44:02 -07:00 |
|
|
Dan Lord
|
20f7cc4cde
|
Add skip_special_tokens sampling params (#1186)
|
2023-09-27 19:21:42 -07:00 |
|
|
Wen Sun
|
bbbf86565f
|
Align max_tokens behavior with openai (#852)
|
2023-09-23 18:10:13 -07:00 |
|
|
Ricardo Lu
|
f98b745a81
|
feat: support stop_token_ids parameter. (#1097)
|
2023-09-21 15:34:02 -07:00 |
|
|
Roy
|
2d1e86f1b1
|
clean api code, remove redundant background task. (#1102)
|
2023-09-21 13:25:05 -07:00 |
|
|
Woosuk Kwon
|
bc0644574c
|
Add gpu_memory_utilization and swap_space to LLM (#1090)
|
2023-09-19 22:16:04 -07:00 |
|
|
orellavie1212
|
fbe66e1d0b
|
added support for quantize on LLM module (#1080)
|
2023-09-18 11:04:21 -07:00 |
|
|
Lukas Kreussel
|
b5f93d0631
|
Only fail if logit_bias has actual values (#1045)
|
2023-09-14 17:33:01 -07:00 |
|
|
|
ab019eea75
|
Add Model Revision Support (#1014)
Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
|
2023-09-13 15:20:02 -07:00 |
|
|
Antoni BaumandZhuohan Li
|
080438477f
|
Start background task in AsyncLLMEngine.generate (#988)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
|
2023-09-08 00:03:39 -07:00 |
|
|
Antoni BaumandAvnish Narayan
|
c07ece5ca4
|
Make AsyncLLMEngine more robust & fix batched abort (#969)
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
Co-authored-by: Avnish Narayan <38871737+avnishn@users.noreply.github.com>
|
2023-09-07 13:43:45 -07:00 |
|
|
Antoni Baum
|
1696725879
|
Initialize AsyncLLMEngine bg loop correctly (#943)
|
2023-09-04 17:41:22 -07:00 |
|
|
lplcor
|
becd7a56f1
|
Enable request body OpenAPI spec for OpenAI endpoints (#865)
|
2023-08-29 21:54:08 -07:00 |
|
|
WanMok
|
e06f504a76
|
Supports tokens and arrays of tokens as inputs to the OpenAI completion API (#715)
|
2023-08-11 12:14:34 -07:00 |
|
|
Nicolas Basile
|
66c54aa9c3
|
Check the max prompt length for the OpenAI completions API (#472)
|
2023-08-08 17:43:49 -07:00 |
|
|
YHPeterandhao.yu
|
e8ddc08ec8
|
[BUG FIX] upgrade fschat version to 0.2.23 (#650)
Co-authored-by: hao.yu <hao.yu@cn-c017.server.mila.quebec>
|
2023-08-02 14:05:59 -07:00 |
|
|
Zhuohan Li
|
58a072be15
|
[Fix] Add model sequence length into model config (#575)
|
2023-07-25 23:46:30 -07:00 |
|
|
Zhuohan Li
|
82ad323dee
|
[Fix] Add chat completion Example and simplify dependencies (#576)
|
2023-07-25 23:45:48 -07:00 |
|
|
Ricardo Lu
|
8c4b2592fb
|
fix: enable trust-remote-code in api server & benchmark. (#509)
|
2023-07-19 17:06:15 -07:00 |
|