 Tyler Michael Smith
|
7342a7d7f8
|
[Model] Support Mamba (#6484)
|
2024-10-11 15:40:06 +00:00 |
|
|
Cyrus Leung
|
e808156f30
|
[Misc] Collect model support info in a single process per model (#9233)
|
2024-10-11 11:08:11 +00:00 |
|
|
sroy745
|
c8f26bb636
|
[BugFix][Core] Fix BlockManagerV2 when Encoder Input is None (#9103)
|
2024-10-07 03:52:42 +00:00 |
|
|
Chen ZhangandRoger Wang
|
dac914b0d6
|
[Bugfix] use blockmanagerv1 for encoder-decoder (#9084)
Co-authored-by: Roger Wang <ywang@roblox.com>
|
2024-10-05 04:45:38 +00:00 |
|
|
Roger WangandDarkLight1337
|
26aa325f4f
|
[Core][VLM] Test registration for OOT multimodal models (#8717)
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-10-04 10:38:25 -07:00 |
|
|
Michael Goin
|
303d44790a
|
[Misc] Enable multi-step output streaming by default (#9047)
|
2024-10-03 22:55:42 -04:00 |
|
|
xendoandJerzy Zagorski
|
63e39937f9
|
[Frontend] [Neuron] Parse literals out of override-neuron-config (#8959)
Co-authored-by: Jerzy Zagorski <jzagorsk@amazon.com>
|
2024-10-03 18:02:07 +00:00 |
|
|
sroy745
|
f5d72b2fc6
|
[Core] Make BlockSpaceManagerV2 the default BlockManager to use. (#8678)
|
2024-10-03 09:44:21 -07:00 |
|
|
|
563649aafe
|
[Core] Combined support for multi-step scheduling, chunked prefill & prefix caching (#8804)
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
|
2024-10-02 07:52:20 +00:00 |
|
|
Lily Liu
|
1570203864
|
[Spec Decode] (1/2) Remove batch expansion (#8839)
|
2024-10-01 16:04:42 -07:00 |
|
|
Sebastian Schoennenbeck
|
be76e5aabf
|
[Core] Make scheduling policy settable via EngineArgs (#8956)
|
2024-09-30 12:28:44 +00:00 |
|
|
Varun Sundar RabindranathandVarun Sundar Rabindranath
|
c2ec430ab5
|
[Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378)
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
|
2024-09-27 13:32:07 -07:00 |
|
|
Alexander Matveev
|
1a2aef3e59
|
Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335)
|
2024-09-23 15:38:04 -07:00 |
|
|
Alex Brooks
|
9b8c8ba119
|
[Core][Frontend] Support Passing Multimodal Processor Kwargs (#8657)
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
|
2024-09-23 07:44:48 +00:00 |
|
|
sroy745
|
1009e93c5d
|
[Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631)
|
2024-09-17 07:35:01 -07:00 |
|
|
Alex Brooks
|
1c1bb388e0
|
[Frontend] Improve Nullable kv Arg Parsing (#8525)
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
|
2024-09-17 04:17:32 +00:00 |
|
|
Roger Wang
|
c16369455f
|
[Hotfix][Core][VLM] Disable chunked prefill by default and prefix caching for multimodal models (#8425)
|
2024-09-12 14:06:51 -07:00 |
|
|
youkaichao
|
f842a7aff1
|
[misc] remove engine_use_ray (#8126)
|
2024-09-11 18:23:36 -07:00 |
|
|
Aarni Koskela
|
8baa454937
|
[Misc] Move device options to a single place (#8322)
|
2024-09-11 13:25:58 -07:00 |
|
|
Cody Yu
|
b1f3e18958
|
[MISC] Keep chunked prefill enabled by default with long context when prefix caching is enabled (#8342)
|
2024-09-10 22:28:28 +00:00 |
|
|
Cyrus Leung
|
9f68e00d27
|
[Bugfix] Fix broken OpenAI tensorizer test (#8258)
|
2024-09-07 08:02:39 +00:00 |
|
|
Patrick von PlatenandMichael Goin
|
29f49cd6e3
|
[Model] Allow loading from original Mistral format (#8168)
Co-authored-by: Michael Goin <michael@neuralmagic.com>
|
2024-09-06 17:02:05 -06:00 |
|
|
Harsha vardhan manoj BikkiandHarsha Bikki
|
008cf886c9
|
[Neuron] Adding support for adding/ overriding neuron configuration a… (#8062)
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
|
2024-09-04 16:33:43 -07:00 |
|
|
Isotr0py
|
4ca65a9763
|
[Core][Bugfix] Accept GGUF model without .gguf extension (#8056)
|
2024-09-02 08:43:26 -04:00 |
|
|
Cyrus Leung
|
98cef6a227
|
[Core] Increase default max_num_batched_tokens for multimodal models (#8028)
|
2024-08-30 08:20:34 -07:00 |
|
|
Patrick von Platen
|
6fc4e6e07a
|
[Model] Add Mistral Tokenization to improve robustness and chat encoding (#7739)
|
2024-08-27 12:40:02 +00:00 |
|
|
Megha AgarwalandAlexander Matveev
|
2eedede875
|
[Core] Asynchronous Output Processor (#7049)
Co-authored-by: Alexander Matveev <alexm@neuralmagic.com>
|
2024-08-26 20:53:20 -07:00 |
|
|
omrishiv
|
760e9f71a8
|
[Bugfix] neuron: enable tensor parallelism (#7562)
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
|
2024-08-26 15:13:13 -07:00 |
|
|
Cyrus Leung
|
029c71de11
|
[CI/Build] Avoid downloading all HF files in RemoteOpenAIServer (#7836)
|
2024-08-26 05:31:10 +00:00 |
|
|
Michael Goin
|
15310b5101
|
[Bugfix] Use LoadFormat values for vllm serve --load-format (#7784)
|
2024-08-22 11:37:08 -07:00 |
|
|
Travis Johnson
|
67e02fa8a4
|
[Bugfix] use StoreBoolean instead of type=bool for --disable-logprobs-during-spec-decoding (#7665)
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
|
2024-08-20 00:43:09 +00:00 |
|
|
William Linandafeldman-nm
|
47b65a5508
|
[core] Multi Step Scheduling (#7000)
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
|
2024-08-19 13:52:13 -07:00 |
|
|
Ali Panahi
|
dad961ef5c
|
[Bugfix] fix lora_dtype value type in arg_utils.py - part 2 (#5428)
|
2024-08-19 20:47:00 +00:00 |
|
|
SangBin Cho
|
ff7ec82c4d
|
[Core] Optimize SPMD architecture with delta + serialization optimization (#7109)
|
2024-08-18 17:57:20 -07:00 |
|
|
Roger Wang
|
bbf55c4805
|
[VLM] Refactor MultiModalConfig initialization and profiling (#7530)
|
2024-08-17 13:30:55 -07:00 |
|
|
Mahesh Keralapura
|
93478b63d2
|
[Core] Fix tracking of model forward time in case of PP>1 (#7440)
[Core] Fix tracking of model forward time to the span traces in case of PP>1 (#7440)
|
2024-08-16 13:46:01 -07:00 |
|
|
shangmingc
|
b67ae00cdb
|
[Misc] Add quantization config support for speculative model. (#7343)
|
2024-08-15 19:34:28 -07:00 |
|
|
omrishivandCyrus Leung
|
9c1f78d5d6
|
[Bugfix] update neuron for version > 0.5.0 (#7175)
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2024-08-15 09:44:14 -07:00 |
|
|
William Lin
|
2ecf7b1757
|
[core] [3/N] multi-step args and sequence.py (#7452)
|
2024-08-14 12:32:45 -07:00 |
|
|
Cyrus Leung
|
3f674a49b5
|
[VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126)
|
2024-08-14 17:55:42 +00:00 |
|
|
|
70b746efcf
|
[Misc] Deprecation Warning when setting --engine-use-ray (#7424)
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: youkaichao <youkaichao@126.com>
|
2024-08-14 09:44:27 -07:00 |
|
|
Cyrus Leung
|
4ddc4743d7
|
[Core] Consolidate GB constant and enable float GB arguments (#7416)
|
2024-08-12 14:14:14 -07:00 |
|
|
Cyrus Leung
|
24154f8618
|
[Frontend] Disallow passing model as both argument and option (#7347)
|
2024-08-12 12:58:34 +00:00 |
|
|
Mahesh Keralapura
|
933790c209
|
[Core] Add span metrics for model_forward, scheduler and sampler time (#7089)
|
2024-08-09 13:55:13 -07:00 |
|
|
|
fd95e026e0
|
[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
|
2024-08-06 16:51:47 -04:00 |
|
|
Isotr0pyandMichael Goin
|
360bd67cf0
|
[Core] Support loading GGUF model (#5191)
Co-authored-by: Michael Goin <michael@neuralmagic.com>
|
2024-08-05 17:54:23 -06:00 |
|
|
Cade Daniel
|
82a1b1a82b
|
[Speculative decoding] Add periodic log with time spent in proposal/scoring/verification (#6963)
|
2024-08-05 08:46:44 +00:00 |
|
|
fzyzcjy
|
f058403683
|
[Doc] Super tiny fix doc typo (#6949)
|
2024-07-30 09:14:03 -07:00 |
|
|
tomeras91
|
ed94e4f427
|
[Bugfix][Model] Jamba assertions and no chunked prefill by default for Jamba (#6784)
|
2024-07-26 20:45:31 -07:00 |
|
|
dongmao zhangandMichael Goin
|
87525fab92
|
[bitsandbytes]: support read bnb pre-quantized model (#5753)
Co-authored-by: Michael Goin <michael@neuralmagic.com>
|
2024-07-23 23:45:09 +00:00 |
|