Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 044793d8df [BugFix] Prevent LLM.encode for non-generation Models (#5184) Robert Shawandmgoin 2024-06-01 19:35:41 -04:00
• c2d6d2f960 [Bugfix]: Fix issues related to prefix caching example (#5177) (#5180) Daniil Arapov 2024-06-02 01:53:52 +03:00
• 8279078e21 [Bugfix] Remove deprecated @abstractproperty (#5174) Zhuohan Li 2024-06-01 15:40:25 -07:00
• b9c0605a8e [Feature][Kernel] Support bitsandbytes quantization and QLoRA (#4776) chenqianfzh 2024-06-01 13:51:10 -07:00
• 37464a0f74 [Bugfix] Fix call to init_logger in openai server (#4765) Nadav Shmayovits 2024-06-01 20:18:50 +03:00
• c354072828 [Minor] Fix the path typo in loader.py: save_sharded_states.py -> save_sharded_state.py (#5151) Ye Cao 2024-06-02 01:11:22 +08:00
• f081c3ce4b [Kernel] Update Cutlass fp8 configs (#5144) 3 people 2024-06-01 14:16:07 +05:30
• 260d119e86 [Kernel] Refactor CUTLASS kernels to always take scales that reside on the GPU (#5137) Tyler Michael Smith 2024-06-01 02:45:32 -04:00
• a360ff80bb [CI/Build] CMakeLists: build all extensions' cmake targets at the same time (#5034) Daniele 2024-06-01 06:06:45 +02:00
• 1197e02141 [Build] Guard against older CUDA versions when building CUTLASS 3.x kernels (#5168) Tyler Michael Smith 2024-05-31 20:21:38 -04:00
• 657579113f [Doc] Add checkmark for GPTBigCodeForCausalLM LoRA support (#5171) Nick Hill 2024-05-31 17:20:19 -07:00
• e9899fb7a4 [Model] Enable FP8 QKV in MoE and refine kernel tuning script (#5039) Cody Yu 2024-05-31 14:29:19 -07:00
• a377f0bd5e [Misc]: optimize eager mode host time (#4196) functionxu123andxuhao 2024-05-31 13:14:50 +08:00
• e9d3aa04f6 Revert "[Kernel] Marlin_24: Ensure the mma.sp instruction is using the ::ordered_metadata modifier (introduced with PTX 8.5)" (#5149) Simon Mo 2024-05-31 00:00:26 -05:00
• a22dea54d3 [Model] Support MAP-NEO model (#5081) SnowDistandZhuohan Li 2024-05-31 10:24:41 +08:00
• 533c217792 Fix cutlass sm_90a vesrion in CMakeList simon-mo 2024-05-31 02:13:01 +00:00
• 6d21fa1cad [Kernel] Marlin_24: Ensure the mma.sp instruction is using the ::ordered_metadata modifier (introduced with PTX 8.5) (#5136) Alexander Matveev 2024-05-30 22:02:11 -04:00
• b35be5403f [Bugfix] Avoid Warnings in SparseML Activation Quantization (#5120) Robert Shaw 2024-05-30 17:04:37 -07:00
• 45a1a69b98 [Build] Disable sm_90a in cu11 (#5141) Simon Mo 2024-05-30 16:37:16 -05:00
• 87a658c812 Bump version to v0.4.3 (#5046) Simon Mo 2024-05-30 13:13:46 -05:00
• 429d89720e add doc about serving option on dstack (#3074) Chansung ParkandRoger Wang 2024-05-31 02:11:07 +09:00
• a9bcc7afb2 [Doc] Use intersphinx and update entrypoints docs (#5125) Cyrus Leung 2024-05-31 00:59:23 +08:00
• d79d9eaaff [Misc] remove duplicate definition of seq_lens_tensor in model_runner.py (#5129) Hyunsung Lee 2024-05-30 22:56:19 +09:00
• f758505c73 [CI/Build] increase wheel size limit to 200 MB (#5130) youkaichao 2024-05-30 06:29:48 -07:00
• d910816c73 [Bugfix] Automatically Detect SparseML models (#5119) Robert Shaw 2024-05-30 05:58:37 -07:00
• 87d41c849d [BUGFIX] [FRONTEND] Correct chat logprobs (#5029) Breno FariaandBreno Faria 2024-05-30 11:52:14 +02:00
• e07aff9e52 [CI/Build] Docker cleanup functionality for amd servers (#5112) 5 people 2024-05-29 22:27:39 -05:00
• 5bf185a1c4 [Bugfix] gptq_marlin: Ensure g_idx_sort_indices is not a Parameter (#5108) Alexander Matveev 2024-05-29 20:30:18 -04:00
• 4fbcb0f27e [Doc][Build] update after removing vllm-nccl (#5103) youkaichaoandRoger Wang 2024-05-29 16:51:18 -07:00
• 7c3604fb68 [Bugfix] logprobs is not compatible with the OpenAI spec #4795 (#5031) Itay Etelis 2024-05-30 02:13:22 +03:00
• b1c255630d [Core] Avoid the need to pass None values to Sequence.inputs (#5099) Cyrus Leung 2024-05-30 07:05:01 +08:00
• eb6c50cdc2 [Bugfix][CI/Build] Fix codespell failing to skip files in git diff (#5097) Cyrus Leung 2024-05-30 07:02:54 +08:00
• eecd864388 [Bugfix][CI/Build] Fix test and improve code for merge_async_iterators (#5096) Cyrus Leung 2024-05-30 07:02:25 +08:00
• ae495c74ea [Doc]Replace deprecated flag in readme (#4526) Ronen Schaffer 2024-05-30 01:26:33 +03:00
• 4238bc82f2 [Core] Cross-attention KV caching and memory-management (towards eventual encoder/decoder model support) (#4837) afeldman-nm 2024-05-29 12:09:13 -04:00
• 594392d27a [Core][Distributed] improve p2p access check (#4992) youkaichao 2024-05-29 04:29:07 -07:00
• 18c1f16d86 [Bugfix] Fix arguments passed to Sequence in stop checker test (#5092) Cyrus Leung 2024-05-29 15:16:41 +08:00
• 5bd3c65072 [Core][Optimization] remove vllm-nccl (#5091) youkaichao 2024-05-28 22:13:52 -07:00
• 616e600e0b [Misc] add gpu_memory_utilization arg (#5079) Marut Pandya 2024-05-28 17:16:18 -07:00
• dfba529b40 [Bugfix] Remove the last EOS token unless explicitly specified (#5077) Junichi Sato 2024-05-29 09:15:35 +09:00
• 5ae5ed1e60 [Core] Consolidate prompt arguments to LLM engines (#4328) Cyrus LeungandRoger Wang 2024-05-29 04:29:31 +08:00
• 290f4ada2b [Docs] Add Dropbox as sponsors (#5089) Simon Mo 2024-05-28 12:29:09 -05:00
• dd8de11f0a [Kernel][ROCm][AMD] Add fused_moe Triton configs for MI300X (#4951) Divakar Verma 2024-05-28 11:03:23 -05:00
• 9ba415588a [BugFix] Fix Embedding Models with TP>1 (#5075) Robert Shaw 2024-05-28 08:32:42 -07:00
• d4f3985907 [Core] Sliding window for block manager v2 (#4545) Michał MoskalandRuth Evans 2024-05-27 19:07:07 -07:00
• 890aa93d27 [Model] Add support for falcon-11B (#5069) Isotr0py 2024-05-28 07:41:43 +08:00
• fbdb7b3ee2 [Core] Allow AQLM on Pascal (#5058) sasha0552 2024-05-27 22:26:14 +00:00
• 1102bef219 [Bugfix / Core] Prefix Caching Guards (merged with main) (#4846) 3 people 2024-05-27 15:18:17 -07:00
• f17a1a8f96 [Misc] Make Serving Benchmark More User-friendly (#5044) Roger Wang 2024-05-25 10:28:16 -07:00
• d5a1697772 [Dynamic Spec Decoding] Minor fix for disabling speculative decoding (#5000) Lily Liu 2024-05-25 10:00:14 -07:00
• 325c119961 [Misc] add logging level env var (#5045) youkaichao 2024-05-24 23:49:49 -07:00
• 8e192ff967 [Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799) 5 people 2024-05-25 01:00:52 -04:00
• e64fde4b01 [Core][Bugfix]: fix prefix caching for blockv2 (#4764) leiwen83andLei Wen 2024-05-25 01:07:09 +08:00
• 919770957f [Bugfix] Fix Mistral v0.3 Weight Loading (#5005) Robert ShawandCody Yu 2024-05-24 14:28:27 +02:00
• 6a50f4cafa [Doc] add ccache guide in doc (#5012) youkaichaoandMichael Goin 2024-05-23 16:21:54 -07:00
• e3470f8753 [Core]: Option To Use Prompt Token Ids Inside Logits Processor (#4985) Elisei SmirnovandElisei Smirnov 2024-05-24 01:04:24 +03:00
• a1242324c9 [Kernel] Initial Activation Quantization Support (#4525) 3 people 2024-05-23 17:29:18 -04:00
• 5eda2ea02a [Core][1/N] Support send/recv in PyNCCL Groups (#4988) Murali Andoorveedu 2024-05-23 09:54:48 -07:00
• 2ba80bed27 [Bugfix] Update Dockerfile.cpu to fix NameError: name 'vllm_ops' is not defined (#5009) Letian Li 2024-05-23 17:08:58 +01:00
• 6066253296 Marlin 24 prefill performance improvement (about 25% better on average) (#4983) Alexander Matveev 2024-05-23 02:39:27 -04:00
• ee3eea0a1b [Misc] Take user preference in attention selector (#4960) Cody Yu 2024-05-22 15:55:56 -07:00
• a36de682d4 [Minor] Fix small typo in llama.py: QKVParallelLinear -> QuantizationConfig (#4991) Philipp Moritz 2024-05-22 15:26:56 -07:00
• eb6d3c264d [Core] Eliminate parallel worker per-step task scheduling overhead (#4894) Nick Hill 2024-05-22 14:17:27 -07:00
• 97b030005c [Model] LoRA gptbigcode implementation (#3949) raywanb 2024-05-23 04:58:59 +08:00
• a3a73ab069 [Misc] Load FP8 kv-cache scaling factors from checkpoints (#4893) Cody Yu 2024-05-22 13:28:20 -07:00
• 8674f9880e [Kernel] Fixup for CUTLASS kernels in CUDA graphs (#4954) Tyler Michael Smith 2024-05-22 10:10:43 -04:00
• c74c913bfb [misc] remove comments that were supposed to be removed (#4977) SangBin Cho 2024-05-22 22:02:58 +09:00
• 5f6d10c14c [CI/Build] Enforce style for C++ and CUDA code with clang-format (#4722) Michael Goin 2024-05-22 03:18:41 -04:00
• 9b9a10d6cb [Frontend] Dynamic RoPE scaling (#4638) sasha0552 2024-05-22 05:32:35 +00:00
• 99eff67ba9 [Bugfix][Kernel] Add head size check for attention backend selection (#4944) Isotr0py 2024-05-22 03:33:25 +08:00
• 14772eeb8e [Bugfix] Fix flag name for max_seq_len_to_capture (#4935) Kante Yin 2024-05-22 00:30:52 +08:00
• 757b62c495 [CI/Build] Codespell ignore build/ directory (#4945) Michael Goin 2024-05-21 12:06:10 -04:00
• e941f88584 [Docs] Add acknowledgment for sponsors (#4925) Simon Mo 2024-05-21 00:17:25 -07:00
• f12c3b5b3d [Model] Add Phi-2 LoRA support (#4886) Isotr0py 2024-05-21 13:24:17 +08:00
• d130b573a0 [Model] add rope_scaling support for qwen2 (#4930) HUANG Fei 2024-05-21 13:22:22 +08:00
• 65ae8c2c8f [Core] Fix scheduler considering "no LoRA" as "LoRA" (#4897) Antoni Baum 2024-05-20 17:48:32 -07:00
• c3af44722c [Doc]Add documentation to benchmarking script when running TGI (#4920) Kuntai Du 2024-05-20 13:16:57 -07:00
• 1937e29848 [Core] Sharded State Loader download from HF (#4889) Aurick Qiao 2024-05-20 14:46:12 -04:00
• f0eecee610 [Bugfix] Fix dummy weight for fp8 (#4916) Mor ZusmanandMor Zusman 2024-05-20 21:44:25 +03:00
• 943e72ca56 [Build/CI] Enabling AMD Entrypoints Test (#4834) Alexei-V-Ivanov-AMDandAlexey Kondratiev 2024-05-20 13:29:28 -05:00
• 546a97ef69 [Misc]: allow user to specify port in distributed setting (#4914) Wenwei Zhang 2024-05-21 01:45:06 +08:00
• da5a0b539d Remove marlin warning (#4918) Alexander Matveev 2024-05-20 10:55:34 -04:00
• 6287537a0c [Model] LLaVA model refactor (#4910) Cyrus Leung 2024-05-20 16:11:25 +08:00
• b57e6c5949 [Kernel] Add flash-attn back (#4907) Woosuk Kwon 2024-05-19 18:11:30 -07:00
• 27ce85476e [Kernel] Add marlin_24 unit tests (#4901) Alexander Matveev 2024-05-19 11:37:34 -04:00
• f68470e803 [Bugfix][Model] Add base class for vision-language models (#4809) Cyrus Leung 2024-05-19 15:13:33 +08:00
• 2e9a2227ec [Lora] Support long context lora (#4787) SangBin Cho 2024-05-18 16:05:23 +09:00
• c0724fc915 [ROCm][Hardware][AMD] Adding Navi21 to fallback to naive attention if Triton is not used (#4658) alexeykondrat 2024-05-18 01:09:11 -04:00
• 86b45ae065 [Bugfix] Relax tiktoken to >= 0.6.0 (#4890) Michael Goin 2024-05-17 14:58:52 -04:00
• c5711ef985 [Doc] Update Ray Data distributed offline inference example (#4871) Antoni Baum 2024-05-17 10:52:11 -07:00
• 48d5985a08 Sync huggingface modifications of qwen Moe model (#4774) eigenLiu 2024-05-18 00:43:19 +08:00
• 33e0823de5 [Bugfix] fix rope error when load models with different dtypes (#4835) Jinzhen Lin 2024-05-17 17:43:34 +08:00
• 26148120b3 [Build/CI] Extending the set of AMD tests with Regression, Basic Correctness, Distributed, Engine, Llava Tests (#4797) Alexei-V-Ivanov-AMD 2024-05-16 22:58:25 -05:00
• 0150a10630 [Frontend] OpenAI API server: Do not add bos token by default when encoding (#4688) bofeng huang 2024-05-17 03:47:22 +02:00
• 8e7fb5d43a Support to serve vLLM on Kubernetes with LWS (#4829) Kante Yin 2024-05-17 07:37:29 +08:00
• 9a31a817a8 [Bugfix] Fix FP8 KV cache support (#4869) Woosuk Kwon 2024-05-16 15:42:29 -07:00
• 2060e93659 [Kernel] Add w8a8 CUTLASS kernels (#4749) Tyler Michael Smith 2024-05-16 18:32:50 -04:00
• 8435b207af [Kernel] Add punica dimension for Qwen1.5-32B LoRA (#4850) SilencioandSilencio 2024-05-17 02:16:09 +08:00
• 10fa9eea21 [Misc] remove old comments (#4866) youkaichao 2024-05-16 11:07:41 -07:00
• e08188081b [Core][Distributed] remove graph mode function (#4818) youkaichao 2024-05-16 10:59:52 -07:00