 Toshiki Kataoka
|
06b2550cbb
|
[Bugfix] Support prompt_logprobs==0 (#5217)
|
2024-06-03 17:59:30 -07:00 |
|
|
Breno Faria
|
f775a07e30
|
[FRONTEND] OpenAI tools support named functions (#5032)
|
2024-06-03 18:25:29 -05:00 |
|
|
Kaiyang Chen
|
10c38e3e46
|
[Misc]: Implement CPU/GPU swapping in BlockManagerV2 (#3834)
|
2024-06-03 13:37:11 -07:00 |
|
|
Tyler Michael Smith
|
cbb2f59cc8
|
[Kernel] Pass a device pointer into the quantize kernel for the scales (#5159)
|
2024-06-03 09:52:30 -07:00 |
|
|
Antoni Baum
|
0ab278ca31
|
[Core] Remove unnecessary copies in flash attn backend (#5138)
|
2024-06-03 09:39:31 -07:00 |
|
|
Cyrus Leung
|
7a64d24aad
|
[Core] Support image processor (#4197)
|
2024-06-02 22:56:41 -07:00 |
|
|
Divakar Verma
|
a66cf40b20
|
[Kernel][ROCm][AMD] enable fused topk_softmax kernel for moe layer (#4927)
This PR enables the fused topk_softmax kernel used in moe layer for HIP
|
2024-06-02 14:13:26 -07:00 |
|
|
Avinash Raj
|
f790ad3c50
|
[Frontend][OpenAI] Support for returning max_model_len on /v1/models response (#4643)
|
2024-06-02 08:06:13 +00:00 |
|
|
Robert Shawandmgoin
|
044793d8df
|
[BugFix] Prevent LLM.encode for non-generation Models (#5184)
Co-authored-by: mgoin <michael@neuralmagic.com>
|
2024-06-01 23:35:41 +00:00 |
|
|
Zhuohan Li
|
8279078e21
|
[Bugfix] Remove deprecated @abstractproperty (#5174)
|
2024-06-01 22:40:25 +00:00 |
|
|
chenqianfzh
|
b9c0605a8e
|
[Feature][Kernel] Support bitsandbytes quantization and QLoRA (#4776)
|
2024-06-01 14:51:10 -06:00 |
|
|
Nadav Shmayovits
|
37464a0f74
|
[Bugfix] Fix call to init_logger in openai server (#4765)
|
2024-06-01 17:18:50 +00:00 |
|
|
Ye Cao
|
c354072828
|
[Minor] Fix the path typo in loader.py: save_sharded_states.py -> save_sharded_state.py (#5151)
Signed-off-by: Ye Cao <caoye.cao@alibaba-inc.com>
|
2024-06-01 17:11:22 +00:00 |
|
|
Tyler Michael Smith
|
260d119e86
|
[Kernel] Refactor CUTLASS kernels to always take scales that reside on the GPU (#5137)
|
2024-06-01 06:45:32 +00:00 |
|
|
Cody Yu
|
e9899fb7a4
|
[Model] Enable FP8 QKV in MoE and refine kernel tuning script (#5039)
|
2024-05-31 14:29:19 -07:00 |
|
|
functionxu123andxuhao
|
a377f0bd5e
|
[Misc]: optimize eager mode host time (#4196)
Co-authored-by: xuhao <xuhao@cambricon.com>
|
2024-05-31 13:14:50 +08:00 |
|
|
SnowDistandZhuohan Li
|
a22dea54d3
|
[Model] Support MAP-NEO model (#5081)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
|
2024-05-30 19:24:41 -07:00 |
|
|
Robert Shaw
|
b35be5403f
|
[Bugfix] Avoid Warnings in SparseML Activation Quantization (#5120)
|
2024-05-30 17:04:37 -07:00 |
|
|
Simon Mo
|
87a658c812
|
Bump version to v0.4.3 (#5046)
|
2024-05-30 11:13:46 -07:00 |
|
|
Cyrus Leung
|
a9bcc7afb2
|
[Doc] Use intersphinx and update entrypoints docs (#5125)
|
2024-05-30 09:59:23 -07:00 |
|
|
Hyunsung Lee
|
d79d9eaaff
|
[Misc] remove duplicate definition of seq_lens_tensor in model_runner.py (#5129)
|
2024-05-30 06:56:19 -07:00 |
|
|
Robert Shaw
|
d910816c73
|
[Bugfix] Automatically Detect SparseML models (#5119)
|
2024-05-30 12:58:37 +00:00 |
|
|
Breno FariaandBreno Faria
|
87d41c849d
|
[BUGFIX] [FRONTEND] Correct chat logprobs (#5029)
Co-authored-by: Breno Faria <breno.faria@intrafind.com>
|
2024-05-30 02:52:14 -07:00 |
|
|
Alexander Matveev
|
5bf185a1c4
|
[Bugfix] gptq_marlin: Ensure g_idx_sort_indices is not a Parameter (#5108)
|
2024-05-30 00:30:18 +00:00 |
|
|
Itay Etelis
|
7c3604fb68
|
[Bugfix] logprobs is not compatible with the OpenAI spec #4795 (#5031)
|
2024-05-29 16:13:22 -07:00 |
|
|
Cyrus Leung
|
b1c255630d
|
[Core] Avoid the need to pass None values to Sequence.inputs (#5099)
|
2024-05-29 16:05:01 -07:00 |
|
|
Cyrus Leung
|
eecd864388
|
[Bugfix][CI/Build] Fix test and improve code for merge_async_iterators (#5096)
|
2024-05-29 16:02:25 -07:00 |
|
|
afeldman-nm
|
4238bc82f2
|
[Core] Cross-attention KV caching and memory-management (towards eventual encoder/decoder model support) (#4837)
|
2024-05-29 16:09:13 +00:00 |
|
|
youkaichao
|
594392d27a
|
[Core][Distributed] improve p2p access check (#4992)
|
2024-05-29 11:29:07 +00:00 |
|
|
youkaichao
|
5bd3c65072
|
[Core][Optimization] remove vllm-nccl (#5091)
|
2024-05-29 05:13:52 +00:00 |
|
|
Junichi Sato
|
dfba529b40
|
[Bugfix] Remove the last EOS token unless explicitly specified (#5077)
|
2024-05-28 17:15:35 -07:00 |
|
|
Cyrus LeungandRoger Wang
|
5ae5ed1e60
|
[Core] Consolidate prompt arguments to LLM engines (#4328)
Co-authored-by: Roger Wang <ywang@roblox.com>
|
2024-05-28 13:29:31 -07:00 |
|
|
Divakar Verma
|
dd8de11f0a
|
[Kernel][ROCm][AMD] Add fused_moe Triton configs for MI300X (#4951)
This PR adds Triton kernel configs for the MoE kernel for MI300X
|
2024-05-28 16:03:23 +00:00 |
|
|
Robert Shaw
|
9ba415588a
|
[BugFix] Fix Embedding Models with TP>1 (#5075)
|
2024-05-28 08:32:42 -07:00 |
|
|
Michał MoskalandRuth Evans
|
d4f3985907
|
[Core] Sliding window for block manager v2 (#4545)
Co-authored-by: Ruth Evans <ruthevans@Ruths-MacBook-Pro.local>
|
2024-05-28 11:07:07 +09:00 |
|
|
Isotr0py
|
890aa93d27
|
[Model] Add support for falcon-11B (#5069)
|
2024-05-27 16:41:43 -07:00 |
|
|
sasha0552
|
fbdb7b3ee2
|
[Core] Allow AQLM on Pascal (#5058)
|
2024-05-27 15:26:14 -07:00 |
|
|
|
1102bef219
|
[Bugfix / Core] Prefix Caching Guards (merged with main) (#4846)
Co-authored-by: rsnm2 <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
|
2024-05-27 15:18:17 -07:00 |
|
|
Lily Liu
|
d5a1697772
|
[Dynamic Spec Decoding] Minor fix for disabling speculative decoding (#5000)
|
2024-05-25 10:00:14 -07:00 |
|
|
youkaichao
|
325c119961
|
[Misc] add logging level env var (#5045)
|
2024-05-24 23:49:49 -07:00 |
|
|
|
8e192ff967
|
[Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799)
Co-authored-by: beagleski <yunanzhang@microsoft.com>
Co-authored-by: bapatra <bapatra@microsoft.com>
Co-authored-by: Barun Patra <codedecde@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
|
2024-05-24 22:00:52 -07:00 |
|
|
leiwen83andLei Wen
|
e64fde4b01
|
[Core][Bugfix]: fix prefix caching for blockv2 (#4764)
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
|
2024-05-24 10:07:09 -07:00 |
|
|
Robert ShawandCody Yu
|
919770957f
|
[Bugfix] Fix Mistral v0.3 Weight Loading (#5005)
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
|
2024-05-24 12:28:27 +00:00 |
|
|
Elisei SmirnovandElisei Smirnov
|
e3470f8753
|
[Core]: Option To Use Prompt Token Ids Inside Logits Processor (#4985)
Co-authored-by: Elisei Smirnov <el.smirnov@innopolis.university>
|
2024-05-23 22:04:24 +00:00 |
|
|
|
a1242324c9
|
[Kernel] Initial Activation Quantization Support (#4525)
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
|
2024-05-23 21:29:18 +00:00 |
|
|
Murali Andoorveedu
|
5eda2ea02a
|
[Core][1/N] Support send/recv in PyNCCL Groups (#4988)
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
|
2024-05-23 09:54:48 -07:00 |
|
|
Alexander Matveev
|
6066253296
|
Marlin 24 prefill performance improvement (about 25% better on average) (#4983)
|
2024-05-23 02:39:27 -04:00 |
|
|
Cody Yu
|
ee3eea0a1b
|
[Misc] Take user preference in attention selector (#4960)
|
2024-05-23 07:55:56 +09:00 |
|
|
Philipp Moritz
|
a36de682d4
|
[Minor] Fix small typo in llama.py: QKVParallelLinear -> QuantizationConfig (#4991)
|
2024-05-22 22:26:56 +00:00 |
|
|
Nick Hill
|
eb6d3c264d
|
[Core] Eliminate parallel worker per-step task scheduling overhead (#4894)
|
2024-05-23 06:17:27 +09:00 |
|