 zifeitong
|
3ce2c050dd
|
[Fix] Correct OpenAI batch response format (#5554)
|
2024-06-15 16:57:54 -07:00 |
|
|
Cyrus Leung
|
0e9164b40a
|
[mypy] Enable type checking for test directory (#5017)
|
2024-06-15 04:45:31 +00:00 |
|
|
Cyrus Leung
|
03dccc886e
|
[Misc] Add vLLM version getter to utils (#5098)
|
2024-06-13 11:21:39 -07:00 |
|
|
Michael Goin
|
7d19de2e9c
|
[Frontend] Add "input speed" to tqdm postfix alongside output speed (#5425)
|
2024-06-12 18:42:12 -04:00 |
|
|
Cyrus Leung
|
640052b069
|
[Bugfix][Frontend] Cleanup "fix chat logprobs" (#5026)
|
2024-06-10 22:36:46 -07:00 |
|
|
maor-psandDarkLight1337
|
351d5e7b82
|
[Bugfix] OpenAI entrypoint limits logprobs while ignoring server defined --max-logprobs (#5312)
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-06-11 10:30:31 +08:00 |
|
|
Itay Etelis
|
774d1035e4
|
[Feature][Frontend]: Continued stream_options implementation also in CompletionRequest (#5319)
|
2024-06-10 14:22:09 +00:00 |
|
|
Roger Wang
|
68bc81703e
|
[Frontend][Misc] Enforce Pixel Values as Input Type for VLMs in API Server (#5374)
|
2024-06-10 09:13:39 +00:00 |
|
|
Roger WangandDarkLight1337
|
7a9cb294ae
|
[Frontend] Add OpenAI Vision API Support (#5237)
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-06-07 11:23:32 -07:00 |
|
|
Itay Etelis
|
baa15a9ec3
|
[Feature][Frontend]: Add support for stream_options in ChatCompletionRequest (#5135)
|
2024-06-07 03:29:24 +00:00 |
|
|
Matthew Goldey
|
828da0d44e
|
[Frontend] enable passing multiple LoRA adapters at once to generate() (#5300)
|
2024-06-06 15:48:13 -05:00 |
|
|
DriverSongandqiujiawei9
|
eb8fcd2666
|
[BugFix] Apply get_cached_tokenizer to the tokenizer setter of LLM (#5207)
Co-authored-by: qiujiawei9 <qiujiawei9@jd.com>
|
2024-06-05 10:59:02 -07:00 |
|
|
tomeras91
|
f0a500545f
|
[Frontend] OpenAI API server: Add add_special_tokens to ChatCompletionRequest (default False) (#5278)
|
2024-06-05 09:32:58 -07:00 |
|
|
Toshiki Kataoka
|
06b2550cbb
|
[Bugfix] Support prompt_logprobs==0 (#5217)
|
2024-06-03 17:59:30 -07:00 |
|
|
Breno Faria
|
f775a07e30
|
[FRONTEND] OpenAI tools support named functions (#5032)
|
2024-06-03 18:25:29 -05:00 |
|
|
Cyrus Leung
|
7a64d24aad
|
[Core] Support image processor (#4197)
|
2024-06-02 22:56:41 -07:00 |
|
|
Avinash Raj
|
f790ad3c50
|
[Frontend][OpenAI] Support for returning max_model_len on /v1/models response (#4643)
|
2024-06-02 08:06:13 +00:00 |
|
|
Robert Shawandmgoin
|
044793d8df
|
[BugFix] Prevent LLM.encode for non-generation Models (#5184)
Co-authored-by: mgoin <michael@neuralmagic.com>
|
2024-06-01 23:35:41 +00:00 |
|
|
Nadav Shmayovits
|
37464a0f74
|
[Bugfix] Fix call to init_logger in openai server (#4765)
|
2024-06-01 17:18:50 +00:00 |
|
|
Cyrus Leung
|
a9bcc7afb2
|
[Doc] Use intersphinx and update entrypoints docs (#5125)
|
2024-05-30 09:59:23 -07:00 |
|
|
Breno FariaandBreno Faria
|
87d41c849d
|
[BUGFIX] [FRONTEND] Correct chat logprobs (#5029)
Co-authored-by: Breno Faria <breno.faria@intrafind.com>
|
2024-05-30 02:52:14 -07:00 |
|
|
Itay Etelis
|
7c3604fb68
|
[Bugfix] logprobs is not compatible with the OpenAI spec #4795 (#5031)
|
2024-05-29 16:13:22 -07:00 |
|
|
Cyrus LeungandRoger Wang
|
5ae5ed1e60
|
[Core] Consolidate prompt arguments to LLM engines (#4328)
Co-authored-by: Roger Wang <ywang@roblox.com>
|
2024-05-28 13:29:31 -07:00 |
|
|
|
8e192ff967
|
[Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799)
Co-authored-by: beagleski <yunanzhang@microsoft.com>
Co-authored-by: bapatra <bapatra@microsoft.com>
Co-authored-by: Barun Patra <codedecde@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
|
2024-05-24 22:00:52 -07:00 |
|
|
bofeng huang
|
0150a10630
|
[Frontend] OpenAI API server: Do not add bos token by default when encoding (#4688)
|
2024-05-16 18:47:22 -07:00 |
|
|
Pierre Dulac
|
9216b9cc38
|
[Bugfix] Bypass authorization API token for preflight requests (#4862)
|
2024-05-16 09:42:21 -07:00 |
|
|
Alex Wu
|
5e0391c040
|
[Frontend] Separate OpenAI Batch Runner usage from API Server (#4851)
|
2024-05-17 00:42:41 +09:00 |
|
|
Alex WuandRobert Shaw
|
52f8107cf2
|
[Frontend] Support OpenAI batch file format (#4794)
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
|
2024-05-15 19:13:36 -04:00 |
|
|
Cyrus Leung
|
fc0d9dfc3a
|
[Frontend] Re-enable custom roles in Chat Completions API (#4758)
|
2024-05-15 14:58:46 -07:00 |
|
|
Chang Su
|
e254497b66
|
[Model][Misc] Add e5-mistral-7b-instruct and Embedding API (#3734)
|
2024-05-11 11:30:37 -07:00 |
|
|
Cyrus Leung
|
f12b20decc
|
[Frontend] Move async logic outside of constructor (#4674)
|
2024-05-08 22:48:33 -07:00 |
|
|
Mahmoud AshrafandMichael Goin
|
16bc0a098f
|
[Frontend] add tok/s speed metric to llm class when using tqdm (#4400)
Co-authored-by: Michael Goin <michael@neuralmagic.com>
|
2024-05-08 22:02:31 -07:00 |
|
|
Cyrus Leung
|
323f27b904
|
[Bugfix] Fix asyncio.Task not being subscriptable (#4623)
|
2024-05-06 09:31:05 -07:00 |
|
|
DearPlanet
|
4302987069
|
[Bugfix] Fix inappropriate content of model_name tag in Prometheus metrics (#3937)
|
2024-05-04 15:39:34 -07:00 |
|
|
Sebastian Schoennenbeck
|
f8e7adda21
|
Fix/async chat serving (#2727)
|
2024-05-03 11:04:14 -07:00 |
|
|
Michael Goin
|
7e65477e5e
|
[Bugfix] Allow "None" or "" to be passed to CLI for string args that default to None (#4586)
|
2024-05-03 10:32:21 -07:00 |
|
|
SangBin Cho
|
3521ba4f25
|
[Core][Model runner refactoring 1/N] Refactor attn metadata term (#4518)
|
2024-05-03 10:20:12 -07:00 |
|
|
Yang, Bo
|
808632d3b4
|
[BugFix] Prevent the task of _force_log from being garbage collected (#4567)
|
2024-05-03 01:35:18 +00:00 |
|
|
youkaichao
|
5b8a7c1cb0
|
[Misc] centralize all usage of environment variables (#4548)
|
2024-05-02 11:13:25 -07:00 |
|
|
sasha0552
|
c47ba4aaa9
|
[Bugfix] Add validation for seed (#4529)
|
2024-05-01 19:31:22 +00:00 |
|
|
Robert Shaw
|
4dc8026d86
|
[Bugfix] Fix 307 Redirect for /metrics (#4523)
|
2024-05-01 09:14:13 -07:00 |
|
|
Robert Caulk
|
c3845d82dc
|
Allow user to define whitespace pattern for outlines (#4305)
|
2024-04-30 20:48:39 -07:00 |
|
|
|
a494140433
|
[Frontend] Support complex message content for chat completions endpoint (#3467)
Co-authored-by: Lily Liu <lilyliupku@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2024-04-30 16:28:46 -07:00 |
|
|
Roy
|
7134303cbb
|
[Bugfix][Core] Fix get decoding config from ray (#4335)
|
2024-04-27 11:30:08 +00:00 |
|
|
Cyrus Leung
|
8947bc3c15
|
[Frontend][Bugfix] Disallow extra fields in OpenAI API (#4355)
|
2024-04-27 05:08:24 +00:00 |
|
|
SangBin ChoandDanny Guinther
|
a88081bf76
|
[CI] Disable non-lazy string operation on logging (#4326)
Co-authored-by: Danny Guinther <dguinther@neuralmagic.com>
|
2024-04-26 00:16:58 -07:00 |
|
|
Norman Mu
|
2f30e7c72f
|
[Frontend] Add --log-level option to api server (#4377)
|
2024-04-26 05:36:01 +00:00 |
|
|
Cyrus Leung
|
1e8f4252aa
|
[Bugfix][Frontend] Raise exception when file-like chat template fails to be opened (#4292)
|
2024-04-23 18:19:03 +00:00 |
|
|
Jack Gordley
|
d3c8180ac4
|
[Bugfix] Fixing max token error message for openai compatible server (#4016)
|
2024-04-23 19:06:29 +08:00 |
|
|
SangBin Cho
|
0ae11f78ab
|
[Mypy] Part 3 fix typing for nested directories for most of directory (#4161)
|
2024-04-22 21:32:44 -07:00 |
|