Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• d036198e23 [BugFix][Model] Fix commandr RoPE max_position_embeddings (#3919) Roy 2024-04-09 06:17:21 +08:00
• 59a6abf3c9 [Hotfix][CI/Build][Kernel] CUDA 11.8 does not support layernorm optimizations (#3782) Matt Wong 2024-04-08 14:31:02 -07:00
• bc0c0192d1 [Bugfix] Enable Proper attention_bias Usage in Llama Model Configuration (#3767) Kiran Randroy 2024-04-09 01:12:35 +05:30
• f46864d68d [Bugfix] Added Command-R GPTQ support (#3849) egortolmachevandEgor Tolmachev 2024-04-08 17:59:38 +03:00
• b4543c8f6b [Model] add minicpm (#3893) ywfang 2024-04-08 18:28:36 +08:00
• 0ce0539d47 [Bugfix] Fix Llava inference with Tensor Parallelism. (#3883) Isotr0py 2024-04-07 22:54:13 +08:00
• 2f19283549 [Core] latency optimization (#3890) youkaichao 2024-04-06 19:14:06 -07:00
• 95baec828f [Core] enable out-of-tree model register (#3871) youkaichao 2024-04-06 17:11:41 -07:00
• e4be7d70bb [CI/Benchmark] add more iteration and use median for robust latency benchmark (#3889) youkaichao 2024-04-06 14:32:30 -07:00
• 54951ac4bf [Bugfix] Fix incorrect output on OLMo models in Tensor Parallelism (#3869) Isotr0py 2024-04-06 03:02:09 +08:00
• 18de883489 [Chunked Prefill][4/n] Chunked prefill scheduler. (#3853) SangBin Cho 2024-04-06 02:17:58 +09:00
• 1d7c940d74 Add option to completion API to truncate prompt tokens (#3144) Thomas Parnell 2024-04-05 19:15:42 +02:00
• cfaf49a167 [Misc] Define common requirements (#3841) Woosuk Kwon 2024-04-05 00:39:17 -07:00
• 9edec652e2 [Bugfix] Fixing requirements.txt (#3865) Noam Gat 2024-04-05 09:46:01 +03:00
• e0dd4d3589 [Misc] Fix linter issues in examples/fp8/quantizer/quantize.py (#3864) Cade Daniel 2024-04-04 21:57:33 -07:00
• e5043a3e75 [Misc] Add pytest marker to opt-out of global test cleanup (#3863) Cade Daniel 2024-04-04 21:54:16 -07:00
• d03d64fd2e [CI/Build] refactor dockerfile & fix pip cache youkaichao 2024-04-04 21:53:16 -07:00
• 78107fa091 [Doc]Add asynchronous engine arguments to documentation. (#3810) 3 people 2024-04-04 23:52:01 -05:00
• c391e4b68e [Core] improve robustness of pynccl (#3860) youkaichao 2024-04-04 16:52:12 -07:00
• 9117f892f0 [Model] Cohere CommandR+ (#3829) Saurabh Dash 2024-04-05 02:01:49 +05:30
• db2a6a41e2 [Hardware][CPU] Update cpu torch to match default of 2.2.1 (#3854) Michael Goin 2024-04-04 12:49:49 -07:00
• ca81ff5196 [Core] manage nccl via a pypi package & upgrade to pt 2.2.1 (#3805) youkaichao 2024-04-04 10:26:19 -07:00
• b7782002e1 [Benchmark] Refactor sample_requests in benchmark_throughput (#3613) TianYu GUOandRoger Wang 2024-04-04 17:56:22 +08:00
• 819a309c0f [Bugfix] Fix args in benchmark_serving (#3836) Chang SuandRoger Wang 2024-04-04 00:41:05 -07:00
• aabe8f40f2 [Core] [Frontend] Make detokenization optional (#3749) Matthias GerstgrasserandNick Hill 2024-04-03 21:52:18 -07:00
• 498eb5cfa3 [Bugfix] Add kv_scale input parameter to CPU backend (#3840) Woosuk Kwon 2024-04-03 21:33:08 -07:00
• 537ee25f43 [Core] Enable hf_transfer by default if available (#3817) Michael Feil 2024-04-03 21:02:43 -07:00
• 294f8f6665 [BugFix] Pass tokenizer_config to local_tokenizer_group (#3754) Tao He 2024-04-04 11:31:46 +08:00
• b95047f2da [Misc] Publish 3rd meetup slides (#3835) Woosuk Kwon 2024-04-03 15:46:10 -07:00
• 2ff767b513 Enable scaled FP8 (e4m3fn) KV cache on ROCm (AMD GPU) (#3290) +212 people 2024-04-03 16:15:55 -05:00
• 3dcb3e8b98 [3/N] Refactor scheduler for chunked prefill scheduling (#3550) SangBin Cho 2024-04-04 06:13:49 +09:00
• c64cf38673 [Doc] Update contribution guidelines for better onboarding (#3819) Michael Feil 2024-04-03 00:31:43 -07:00
• 76b889bf1d [Doc] Update README.md (#3806) Robert Shaw 2024-04-02 23:11:10 -07:00
• c9b506dad4 [BugFix] Use different mechanism to get vllm version in is_cpu() (#3804) Nick Hill 2024-04-02 23:06:25 -07:00
• 5757d90e26 [Speculative decoding] Adding configuration object for speculative decoding (#3706) Cade DanielandLily Liu 2024-04-02 17:40:57 -07:00
• a3c226e7eb [CI/Build] 0.4.0.post1, fix sm 7.0/7.5 binary (#3803) youkaichao 2024-04-02 12:57:04 -07:00
• b321d4881b [Bugfix] Add __init__.py files for vllm/core/block/ and vllm/spec_decode/ (#3798) Michael Goin 2024-04-02 12:35:31 -07:00
• ad6eca408b Fix early CUDA init via get_architecture_class_name import (#3770) leiwen83andLei Wen 2024-04-03 02:56:26 +08:00
• 205b94942e [CI/Build] fix TORCH_CUDA_ARCH_LIST in wheel build (#3801) youkaichao 2024-04-02 11:54:33 -07:00
• 3bec41f41a [Doc] Fix vLLMEngine Doc Page (#3791) Roger Wang 2024-04-02 09:49:37 -07:00
• 0739b1947f [Frontend][Bugfix] allow using the default middleware with a root path (#3788) A-MahlaandA-Mahla <> 2024-04-02 10:20:28 +02:00
• 77a6572aa5 [HotFix] [CI/Build] Minor fix for CPU backend CI (#3787) bigPYJ1151 2024-04-02 13:50:53 +08:00
• 0e3f06fe9c [Hardware][Intel] Add CPU inference backend (#3634) 3 people 2024-04-02 13:07:30 +08:00
• eb69d68804 [Misc] [CI/Build] Speed up block manager CPU-only unit tests ~10x by opting-out of GPU cleanup (#3783) Cade Daniel 2024-04-01 17:49:51 -07:00
• 7d4e1b85e7 [Misc] Add support for new autogptq checkpoint_format (#3689) QubitiumandRobert Shaw 2024-04-02 07:32:01 +08:00
• 93deb0b38f [Speculative decoding 4/9] Lookahead scheduling for speculative decoding (#3250) Cade Daniel 2024-04-01 15:55:24 -07:00
• ccb58b23e6 [Misc] Fix Benchmark TTFT Calculation for Chat Completions (#3768) Roger Wang 2024-04-01 15:24:30 -07:00
• 49782fcb76 [Misc] Some minor simplifications to detokenization logic (#3670) Nick Hill 2024-04-01 13:22:06 -07:00
• f03cc667a0 [Misc] Minor fixes in requirements.txt (#3769) Woosuk Kwon 2024-04-01 03:15:48 -07:00
• 563c1d7ec5 [CI/Build] Make Marlin Tests Green (#3753) Robert Shaw 2024-03-30 21:18:34 -05:00
• 9c82a1bec3 [Doc] Update installation doc (#3746) youkaichao 2024-03-30 16:34:38 -07:00
• b6d103542c [Kernel] Layernorm performance optimization (#3662) mawong-amd 2024-03-30 14:26:38 -07:00
• 51c31bc10c CMake build elf without PTX (#3739) Simon Mo 2024-03-29 18:53:08 -07:00
• 3ad438c66f Fix build when nvtools is missing (#3698) bnellnm 2024-03-29 21:52:39 -04:00
• 203d4f82ac [Core][Bugfix] cache len of tokenizer (#3741) youkaichao 2024-03-29 18:46:39 -07:00
• 991143cfcd [BugFix] Use consistent logger everywhere (#3738) Nick Hill 2024-03-29 16:26:44 -07:00
• 8b2d3cbc1b usage lib get version another way (#3735) Simon Mo 2024-03-29 15:57:08 -07:00
• 9765b5c406 [ROCm][Bugfix] Fixed several bugs related to rccl path and attention selector logic (#3699) Hongxia Yang 2024-03-29 17:52:36 -04:00
• 430530fc18 bump version to v0.4.0 (#3712) Simon Mo 2024-03-29 12:28:33 -07:00
• 97356f3c7e [Bugfix] Command-R Max Model Length (#3727) Roger Wang 2024-03-29 12:27:51 -07:00
• f510395bbf [BugFix][Frontend] Fix completion logprobs=0 error (#3731) Roy 2024-03-30 00:38:21 +08:00
• 6110c39dc8 [BugFix] Fix tokenizer out of vocab size (#3685) Roy 2024-03-29 23:18:59 +08:00
• d8658c8cc1 Usage Stats Collection (#2852) yhu422 2024-03-28 22:16:12 -07:00
• 7bc94a0fdd add ccache to docker build image (#3704) Simon Mo 2024-03-28 22:14:24 -07:00
• 756b30a5f3 [Core][Test] move local_rank to the last arg with default value(#3711) youkaichao 2024-03-28 21:19:45 -07:00
• 395aa823ea [Misc] Minor type annotation fix (#3716) Woosuk Kwon 2024-03-28 21:12:24 -07:00
• 26422e477b [Test] Make model tests run again and remove --forked from pytest (#3631) SangBin ChoandSimon Mo 2024-03-29 13:06:40 +09:00
• f342153b48 Revert "bump version to v0.4.0" (#3708) youkaichao 2024-03-28 18:49:42 -07:00
• 27a57cad52 bump version to v0.4.0 (#3705) Simon Mo 2024-03-28 18:26:51 -07:00
• 98a42e7078 [Benchmark] Change mii to use persistent deployment and support tensor parallel (#3628) Yile (Michael) Gu 2024-03-28 17:33:52 -07:00
• 0267fef52a [Core] fix del of communicator (#3702) youkaichao 2024-03-28 17:24:58 -07:00
• 4716a32dd4 fix logging msg for block manager (#3701) Simon Mo 2024-03-28 16:29:55 -07:00
• c0935c96d3 [Bugfix] Set enable_prefix_caching=True in prefix caching example (#3703) Woosuk Kwon 2024-03-28 16:26:30 -07:00
• cb40b3ab6b [Kernel] Add MoE Triton kernel configs for A100 40GB (#3700) Woosuk Kwon 2024-03-28 15:26:24 -07:00
• 515386ef3c [Core] Support multi-node inference(eager and cuda graph) (#3686) Roy 2024-03-29 06:01:55 +08:00
• a4075cba4d [CI] Add test case to run examples scripts (#3638) Simon Mo 2024-03-28 14:36:10 -07:00
• 96aa014d1e fix benchmark format reporting in buildkite (#3693) Simon Mo 2024-03-28 14:35:16 -07:00
• 1715056fef [Bugfix] Update neuron_executor.py to add optional vision_language_config (#3695) Adam Boeglin 2024-03-28 10:43:34 -07:00
• b51c1cc9d2 [2/N] Chunked prefill data update (#3538) SangBin Cho 2024-03-29 02:06:01 +09:00
• ce567a2926 [Kernel] DBRX Triton MoE kernel H100 (#3692) Roger Wang 2024-03-28 10:05:34 -07:00
• d6ea427f04 [Model] Add support for Qwen2MoeModel (#3346) wenyujin333 2024-03-28 23:19:59 +08:00
• 14ccd94c89 [Core][Bugfix]Refactor block manager for better testability (#3492) Cade Daniel 2024-03-27 23:59:28 -07:00
• 8267b06c30 [Kernel] Add Triton MoE kernel configs for DBRX on A100 (#3679) Woosuk Kwon 2024-03-27 22:22:25 -07:00
• 3492859b68 [CI/Build] update default number of jobs and nvcc threads to avoid overloading the system (#3675) youkaichao 2024-03-27 21:18:54 -07:00
• 098e1776ba [Model] Add support for xverse (#3610) 3 people 2024-03-28 09:12:54 +08:00
• 10e6322283 [Model] Fix and clean commandr (#3671) Roy 2024-03-28 08:20:00 +08:00
• 6d9aa00fc4 [Docs] Add Command-R to supported models (#3669) Woosuk Kwon 2024-03-27 15:20:00 -07:00
• 1182607e18 Add support for Cohere's Command-R model (#3433) 3 people 2024-03-27 21:19:32 +00:00
• 45b6ef6513 feat(benchmarks): Add Prefix Caching Benchmark to Serving Benchmark (#3277) Roger Wang 2024-03-27 13:39:26 -07:00
• 1956931436 [Misc] add the "download-dir" option to the latency/throughput benchmarks (#3621) AmadeusChan 2024-03-27 16:39:05 -04:00
• e24336b5a7 [Model] Add support for DBRX (#3660) Megha Agarwal 2024-03-27 13:01:46 -07:00
• d18f4e73f3 [Bugfix] [Hotfix] fix nccl library name (#3661) youkaichao 2024-03-27 10:23:54 -07:00
• 82c540bebf [Bugfix] More faithful implementation of Gemma (#3653) Woosuk Kwon 2024-03-27 09:37:18 -07:00
• 8f44facddd [Core] remove cupy dependency (#3625) youkaichao 2024-03-27 00:33:26 -07:00
• e66b629c04 [Misc] Minor fix in KVCache type (#3652) Woosuk Kwon 2024-03-26 23:14:06 -07:00
• 76879342a3 [Doc]add lora support (#3649) Jee Li 2024-03-27 10:06:46 +08:00
• 566b57c5c4 [Kernel] support non-zero cuda devices in punica kernels (#3636) Jee Li 2024-03-27 08:37:42 +08:00
• 0dc72273b8 [BugFix] Fix ipv4 address parsing regression (#3645) Nick Hill 2024-03-26 14:39:44 -07:00
• a979d9771e [Bugfix] Fix ipv6 address parsing bug (#3641) liiliiliil 2024-03-27 02:58:20 +08:00
• 8af890a865 Enable more models to inference based on LoRA (#3382) Jee LiandAntoni Baum 2024-03-26 09:09:31 +08:00