 Michael Goin
|
53b018edcb
|
[Bugfix] Get available quantization methods from quantization registry (#4098)
|
2024-04-18 00:21:55 -07:00 |
|
|
SangBin ChoandSangBin Cho
|
533d2a1f39
|
[Typing] Mypy typing part 2 (#4043)
Co-authored-by: SangBin Cho <sangcho@sangcho-LT93GQWG9C.local>
|
2024-04-17 17:28:43 -07:00 |
|
|
Antoni Baum
|
69e1d2fb69
|
[Core] Refactor model loading code (#4097)
|
2024-04-16 11:34:39 -07:00 |
|
|
Noam GatandSimon Mo
|
05434764cd
|
LM Format Enforcer Guided Decoding Support (#3868)
Co-authored-by: Simon Mo <simon.mo@hey.com>
|
2024-04-16 05:54:57 +00:00 |
|
|
Sanger SteelandRoger Wang
|
d619ae2d19
|
[Doc] Add better clarity for tensorizer usage (#4090)
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
|
2024-04-15 13:28:25 -07:00 |
|
|
Sanger Steel
|
711a000255
|
[Frontend] [Core] feat: Add model loading using tensorizer (#3476)
|
2024-04-13 17:13:01 -07:00 |
|
|
Michael FeilandRoger Wang
|
c2b4a1bce9
|
[Doc] Add typing hints / mypy types cleanup (#3816)
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
|
2024-04-11 17:17:21 -07:00 |
|
|
Antoni Baum
|
a10d3056da
|
[Core] Set linear_weights directly on the layer (#3977)
|
2024-04-11 16:35:51 -04:00 |
|
|
Roger Wang
|
c1dc547129
|
[Kernel] Fused MoE Config for Mixtral 8x22 (#4002)
|
2024-04-11 07:50:00 -07:00 |
|
|
Kunshang Ji
|
e9da5a40c6
|
[Misc] Add indirection layer for custom ops (#3913)
|
2024-04-10 20:26:07 -07:00 |
|
|
youkaichao
|
caada5e50a
|
[Core][Model] torch.compile for layernorm in commandr (#3985)
[Core][Model] Use torch.compile to accelerate layernorm in commandr (#3985)
|
2024-04-11 01:48:26 +00:00 |
|
|
youkaichao
|
63e7176f26
|
[Core][Refactor] move parallel_utils into vllm/distributed (#3950)
[WIP][Core][Refactor] move vllm/model_executor/parallel_utils into vllm/distributed and vllm/device_communicators (#3950)
|
2024-04-10 15:33:30 -07:00 |
|
|
Daniel E Marasco
|
e4c4072c94
|
[Bugfix] Remove key sorting for guided_json parameter in OpenAi compatible Server (#3945)
|
2024-04-10 10:15:51 -07:00 |
|
|
Travis Johnson
|
0258b7a94b
|
[Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty (#3876)
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
|
2024-04-10 01:39:56 -07:00 |
|
|
胡译文
|
b3104b2a10
|
[Bugfix] Fix logits processor when prompt_logprobs is not None (#3899)
|
2024-04-10 00:09:36 -07:00 |
|
|
Junichi Sato
|
e23a43aef8
|
[Bugfix] Fix KeyError on loading GPT-NeoX (#3925)
|
2024-04-09 12:11:31 -07:00 |
|
|
youkaichao
|
6d592eb430
|
[Core] separate distributed_init from worker (#3904)
|
2024-04-09 08:49:02 +00:00 |
|
|
Roy
|
d036198e23
|
[BugFix][Model] Fix commandr RoPE max_position_embeddings (#3919)
|
2024-04-09 06:17:21 +08:00 |
|
|
Kiran Randroy
|
bc0c0192d1
|
[Bugfix] Enable Proper attention_bias Usage in Llama Model Configuration (#3767)
Co-authored-by: roy <jasonailu87@gmail.com>
|
2024-04-08 19:42:35 +00:00 |
|
|
egortolmachevandEgor Tolmachev
|
f46864d68d
|
[Bugfix] Added Command-R GPTQ support (#3849)
Co-authored-by: Egor Tolmachev <t333ga@gmail.com>
|
2024-04-08 14:59:38 +00:00 |
|
|
ywfang
|
b4543c8f6b
|
[Model] add minicpm (#3893)
|
2024-04-08 18:28:36 +08:00 |
|
|
youkaichao
|
95baec828f
|
[Core] enable out-of-tree model register (#3871)
|
2024-04-06 17:11:41 -07:00 |
|
|
Isotr0py
|
54951ac4bf
|
[Bugfix] Fix incorrect output on OLMo models in Tensor Parallelism (#3869)
|
2024-04-05 12:02:09 -07:00 |
|
|
youkaichao
|
c391e4b68e
|
[Core] improve robustness of pynccl (#3860)
|
2024-04-04 16:52:12 -07:00 |
|
|
Saurabh Dash
|
9117f892f0
|
[Model] Cohere CommandR+ (#3829)
|
2024-04-04 13:31:49 -07:00 |
|
|
youkaichao
|
ca81ff5196
|
[Core] manage nccl via a pypi package & upgrade to pt 2.2.1 (#3805)
|
2024-04-04 10:26:19 -07:00 |
|
|
Michael Feil
|
537ee25f43
|
[Core] Enable hf_transfer by default if available (#3817)
|
2024-04-04 04:02:43 +00:00 |
|
|
+2
|
2ff767b513
|
Enable scaled FP8 (e4m3fn) KV cache on ROCm (AMD GPU) (#3290)
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: HaiShaw <hixiao@gmail.com>
Co-authored-by: AdrianAbeyta <Adrian.Abeyta@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: root <root@gt-pla-u18-08.pla.dcgpu>
Co-authored-by: mawong-amd <156021403+mawong-amd@users.noreply.github.com>
Co-authored-by: ttbachyinsda <ttbachyinsda@outlook.com>
Co-authored-by: guofangze <guofangze@kuaishou.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: jacobthebanana <50071502+jacobthebanana@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-04-03 14:15:55 -07:00 |
|
|
Nick Hill
|
991143cfcd
|
[BugFix] Use consistent logger everywhere (#3738)
|
2024-03-29 23:26:44 +00:00 |
|
|
Hongxia Yang
|
9765b5c406
|
[ROCm][Bugfix] Fixed several bugs related to rccl path and attention selector logic (#3699)
|
2024-03-29 14:52:36 -07:00 |
|
|
Roy
|
f510395bbf
|
[BugFix][Frontend] Fix completion logprobs=0 error (#3731)
|
2024-03-29 09:38:21 -07:00 |
|
|
yhu422
|
d8658c8cc1
|
Usage Stats Collection (#2852)
|
2024-03-28 22:16:12 -07:00 |
|
|
youkaichao
|
756b30a5f3
|
[Core][Test] move local_rank to the last arg with default value(#3711)
[Core][Test] move local_rank to the last arg with default value to keep api compatible (#3711)
|
2024-03-28 21:19:45 -07:00 |
|
|
youkaichao
|
0267fef52a
|
[Core] fix del of communicator (#3702)
|
2024-03-29 00:24:58 +00:00 |
|
|
Simon Mo
|
4716a32dd4
|
fix logging msg for block manager (#3701)
|
2024-03-28 23:29:55 +00:00 |
|
|
Woosuk Kwon
|
cb40b3ab6b
|
[Kernel] Add MoE Triton kernel configs for A100 40GB (#3700)
|
2024-03-28 15:26:24 -07:00 |
|
|
Roy
|
515386ef3c
|
[Core] Support multi-node inference(eager and cuda graph) (#3686)
|
2024-03-28 15:01:55 -07:00 |
|
|
Roger Wang
|
ce567a2926
|
[Kernel] DBRX Triton MoE kernel H100 (#3692)
|
2024-03-28 10:05:34 -07:00 |
|
|
wenyujin333
|
d6ea427f04
|
[Model] Add support for Qwen2MoeModel (#3346)
|
2024-03-28 15:19:59 +00:00 |
|
|
Woosuk Kwon
|
8267b06c30
|
[Kernel] Add Triton MoE kernel configs for DBRX on A100 (#3679)
|
2024-03-27 22:22:25 -07:00 |
|
|
|
098e1776ba
|
[Model] Add support for xverse (#3610)
Co-authored-by: willhe <hexin@xverse.cn>
Co-authored-by: root <root@localhost.localdomain>
|
2024-03-27 18:12:54 -07:00 |
|
|
Roy
|
10e6322283
|
[Model] Fix and clean commandr (#3671)
|
2024-03-28 00:20:00 +00:00 |
|
|
|
1182607e18
|
Add support for Cohere's Command-R model (#3433)
Co-authored-by: José Maria Pombal <jose.pombal@unbabel.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
|
2024-03-27 14:19:32 -07:00 |
|
|
Megha Agarwal
|
e24336b5a7
|
[Model] Add support for DBRX (#3660)
|
2024-03-27 13:01:46 -07:00 |
|
|
youkaichao
|
d18f4e73f3
|
[Bugfix] [Hotfix] fix nccl library name (#3661)
|
2024-03-27 17:23:54 +00:00 |
|
|
Woosuk Kwon
|
82c540bebf
|
[Bugfix] More faithful implementation of Gemma (#3653)
|
2024-03-27 09:37:18 -07:00 |
|
|
youkaichao
|
8f44facddd
|
[Core] remove cupy dependency (#3625)
|
2024-03-27 00:33:26 -07:00 |
|
|
Woosuk Kwon
|
e66b629c04
|
[Misc] Minor fix in KVCache type (#3652)
|
2024-03-26 23:14:06 -07:00 |
|
|
Jee LiandAntoni Baum
|
8af890a865
|
Enable more models to inference based on LoRA (#3382)
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
|
2024-03-25 18:09:31 -07:00 |
|
|
Antoni Baum
|
3a243095e5
|
Optimize _get_ranks in Sampler (#3623)
|
2024-03-25 16:03:02 -07:00 |
|