 Wallas Henrique
|
8baf85e4e9
|
[Doc] Compatibility matrix for mutual exclusive features (#8512)
Signed-off-by: Wallas Santos <wallashss@ibm.com>
|
2024-10-11 11:18:50 -07:00 |
|
|
Tyler Michael Smith
|
7342a7d7f8
|
[Model] Support Mamba (#6484)
|
2024-10-11 15:40:06 +00:00 |
|
|
Sebastian Schoennenbeck
|
df3dcdf49d
|
[Bugfix] Fix priority in multiprocessing engine (#9277)
|
2024-10-11 15:35:35 +00:00 |
|
|
Cyrus Leung
|
e808156f30
|
[Misc] Collect model support info in a single process per model (#9233)
|
2024-10-11 11:08:11 +00:00 |
|
|
youkaichaoandBrendan Wong
|
cbc2ef5529
|
[misc] hide best_of from engine (#9261)
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
|
2024-10-10 21:30:44 -07:00 |
|
|
Russell Bryant
|
cf25b93bdd
|
[Core] Fix invalid args to _process_request (#9201)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
2024-10-10 12:10:09 +08:00 |
|
|
Alex Brooks
|
a3691b6b5e
|
[Core][Frontend] Add Support for Inference Time mm_processor_kwargs (#9131)
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
|
2024-10-08 14:12:56 +00:00 |
|
|
Brendan Wong
|
8c746226c9
|
[Frontend] API support for beam search for MQLLMEngine (#9117)
|
2024-10-08 05:51:43 +00:00 |
|
|
youkaichao
|
18b296fdb2
|
[core] remove beam search from the core (#9105)
|
2024-10-07 05:47:04 +00:00 |
|
|
sroy745
|
c8f26bb636
|
[BugFix][Core] Fix BlockManagerV2 when Encoder Input is None (#9103)
|
2024-10-07 03:52:42 +00:00 |
|
|
Varun Sundar RabindranathandVarun Sundar Rabindranath
|
cb3b2b9ba4
|
[Bugfix] Fix incorrect updates to num_computed_tokens in multi-step scheduling (#9038)
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
|
2024-10-06 12:48:11 -07:00 |
|
|
Brendan Wongandyoukaichao
|
168cab6bbf
|
[Frontend] API support for beam search (#9087)
Co-authored-by: youkaichao <youkaichao@126.com>
|
2024-10-05 23:39:03 -07:00 |
|
|
Chen ZhangandRoger Wang
|
dac914b0d6
|
[Bugfix] use blockmanagerv1 for encoder-decoder (#9084)
Co-authored-by: Roger Wang <ywang@roblox.com>
|
2024-10-05 04:45:38 +00:00 |
|
|
Roger WangandDarkLight1337
|
26aa325f4f
|
[Core][VLM] Test registration for OOT multimodal models (#8717)
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-10-04 10:38:25 -07:00 |
|
|
Michael Goin
|
303d44790a
|
[Misc] Enable multi-step output streaming by default (#9047)
|
2024-10-03 22:55:42 -04:00 |
|
|
xendoandJerzy Zagorski
|
63e39937f9
|
[Frontend] [Neuron] Parse literals out of override-neuron-config (#8959)
Co-authored-by: Jerzy Zagorski <jzagorsk@amazon.com>
|
2024-10-03 18:02:07 +00:00 |
|
|
sroy745
|
f5d72b2fc6
|
[Core] Make BlockSpaceManagerV2 the default BlockManager to use. (#8678)
|
2024-10-03 09:44:21 -07:00 |
|
|
|
563649aafe
|
[Core] Combined support for multi-step scheduling, chunked prefill & prefix caching (#8804)
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
|
2024-10-02 07:52:20 +00:00 |
|
|
Lily Liu
|
1570203864
|
[Spec Decode] (1/2) Remove batch expansion (#8839)
|
2024-10-01 16:04:42 -07:00 |
|
|
Sebastian Schoennenbeck
|
35bd215168
|
[Core] [Frontend] Priority scheduling for embeddings and in the OpenAI-API (#8965)
|
2024-10-01 09:58:06 +00:00 |
|
|
Joe RundeandNick Hill
|
062c89e7c9
|
[Frontend][Core] Move guided decoding params into sampling params (#8252)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
|
2024-10-01 09:34:25 +08:00 |
|
|
Sebastian Schoennenbeck
|
be76e5aabf
|
[Core] Make scheduling policy settable via EngineArgs (#8956)
|
2024-09-30 12:28:44 +00:00 |
|
|
Varun Sundar Rabindranath
|
19d02ff938
|
[Bugfix] Fix PP for Multi-Step (#8887)
|
2024-09-28 08:52:46 -07:00 |
|
|
Sebastian Schoennenbeck
|
bd429f2b75
|
[Core] Priority-based scheduling in async engine (#8850)
|
2024-09-27 15:07:10 -07:00 |
|
|
Varun Sundar RabindranathandVarun Sundar Rabindranath
|
c2ec430ab5
|
[Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378)
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
|
2024-09-27 13:32:07 -07:00 |
|
|
Cyrus Leung
|
3b00b9c26c
|
[Core] renamePromptInputs and inputs (#8876)
|
2024-09-26 20:35:15 -07:00 |
|
|
|
770ec6024f
|
[Model] Add support for the multi-modal Llama 3.2 model (#8811)
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
|
2024-09-25 13:29:32 -07:00 |
|
|
Simon Mo
|
4f1ba0844b
|
Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810)
|
2024-09-25 10:36:26 -07:00 |
|
|
科英
|
64840dfae4
|
[Frontend] MQLLMEngine supports profiling. (#8761)
|
2024-09-25 09:37:41 -07:00 |
|
|
Cyrus Leung
|
28e1299e60
|
rename PromptInputs and inputs with backward compatibility (#8760)
|
2024-09-25 09:36:47 -07:00 |
|
|
Joe Runde
|
6e0c9d6bd0
|
[Bugfix] Use heartbeats instead of health checks (#8583)
|
2024-09-24 20:37:38 -07:00 |
|
|
Archit Patke
|
6da1ab6b41
|
[Core] Adding Priority Scheduling (#5958)
|
2024-09-24 19:50:50 -07:00 |
|
|
Travis Johnson
|
01b6f9e1f0
|
[Core][Bugfix] Support prompt_logprobs returned with speculative decoding (#8047)
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
|
2024-09-24 17:29:56 -07:00 |
|
|
Simon Mo
|
3185fb0cca
|
Revert "[Core] Rename PromptInputs to PromptType, and inputs to prompt" (#8750)
|
2024-09-24 05:45:20 +00:00 |
|
|
Alexander Matveev
|
1a2aef3e59
|
Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335)
|
2024-09-23 15:38:04 -07:00 |
|
|
Alex Brooks
|
9b8c8ba119
|
[Core][Frontend] Support Passing Multimodal Processor Kwargs (#8657)
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
|
2024-09-23 07:44:48 +00:00 |
|
|
Cyrus Leung
|
0057894ef7
|
[Core] Rename PromptInputs and inputs(#8673)
|
2024-09-20 19:00:54 -07:00 |
|
|
Nick Hill
|
76515f303b
|
[Frontend] Use MQLLMEngine for embeddings models too (#8584)
|
2024-09-19 12:51:06 -04:00 |
|
|
Joe Runde
|
0d47bf3bf4
|
[Bugfix] add dead_error property to engine client (#8574)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
|
2024-09-18 22:10:01 +00:00 |
|
|
|
7c7714d856
|
[Core][Bugfix][Perf] Introduce MQLLMEngine to avoid asyncio OH (#8157)
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
|
2024-09-18 13:56:58 +00:00 |
|
|
Aaron PhamandCyrus Leung
|
9d104b5beb
|
[CI/Build] Update Ruff version (#8469)
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2024-09-18 11:00:56 +00:00 |
|
|
sroy745
|
1009e93c5d
|
[Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631)
|
2024-09-17 07:35:01 -07:00 |
|
|
Alex Brooks
|
1c1bb388e0
|
[Frontend] Improve Nullable kv Arg Parsing (#8525)
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
|
2024-09-17 04:17:32 +00:00 |
|
|
Nick Hill
|
acd5511b6d
|
[BugFix] Fix clean shutdown issues (#8492)
|
2024-09-16 09:33:46 -07:00 |
|
|
William Lin
|
ba77527955
|
[bugfix] torch profiler bug for single gpu with GPUExecutor (#8354)
|
2024-09-12 21:30:00 -07:00 |
|
|
Alexander Matveev
|
6821020109
|
[Bugfix] Fix async log stats (#8417)
|
2024-09-12 20:48:59 -07:00 |
|
|
Cyrus Leung
|
5ec9c0fb3c
|
[Core] Factor out input preprocessing to a separate class (#7329)
|
2024-09-13 02:56:13 +00:00 |
|
|
Roger Wang
|
c16369455f
|
[Hotfix][Core][VLM] Disable chunked prefill by default and prefix caching for multimodal models (#8425)
|
2024-09-12 14:06:51 -07:00 |
|
|
Nick Hill
|
551ce01078
|
[Core] Add engine option to return only deltas or final output (#7381)
|
2024-09-12 12:02:00 -07:00 |
|
|
youkaichao
|
f842a7aff1
|
[misc] remove engine_use_ray (#8126)
|
2024-09-11 18:23:36 -07:00 |
|