Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 786b7f18a5 Add code-revision config argument for Hugging Face Hub (#2892) Mark Mozolewski 2024-02-17 22:36:53 -08:00
• 8f36444c4f multi-LoRA as extra models in OpenAI server (#2775) jvmncs 2024-02-17 15:00:48 -05:00
• 185b2c29e2 Defensively copy sampling_params (#2881) Nick Hill 2024-02-17 11:18:04 -08:00
• 5f08050d8d Bump up to v0.3.1 (#2887) Woosuk Kwon 2024-02-16 15:05:18 -08:00
• 64da65b322 Prefix Caching- fix t4 triton error (#2517) shiyi.c_98 2024-02-16 14:17:55 -08:00
• 5255d99dc5 [ROCm] Dockerfile fix for flash-attention build (#2885) Hongxia Yang 2024-02-15 13:22:39 -05:00
• 4f2ad11135 Fix DeciLM (#2883) Philipp Moritz 2024-02-14 22:29:57 -08:00
• d7afab6d3a [BugFix] Fix GC bug for LLM class (#2882) Woosuk Kwon 2024-02-14 22:17:44 -08:00
• 31348dff03 Align LoRA code between Mistral and Mixtral (fixes #2875) (#2880) Philipp MoritzandPernekhan Utemuratov 2024-02-14 16:00:43 -08:00
• 25e86b6a61 Don't use cupy NCCL for AMD backends (#2855) Woosuk Kwon 2024-02-14 12:30:44 -08:00
• 4efbac6d35 Migrate AquilaForCausalLM to LlamaForCausalLM (#2867) Roy 2024-02-15 04:30:24 +08:00
• 87069ccf68 Fix docker python version (#2845) Nikola Borisov 2024-02-14 10:17:57 -08:00
• 7e45107f51 [Fix] Fix memory profiling when GPU is used by multiple processes (#2863) Woosuk Kwon 2024-02-13 19:52:34 -08:00
• 0c48b37c31 Fix internlm after https://github.com/vllm-project/vllm/pull/2860 (#2861) Philipp Moritz 2024-02-13 18:01:15 -08:00
• 7eacffd951 Migrate InternLMForCausalLM to LlamaForCausalLM (#2860) Philipp MoritzandRoy 2024-02-13 17:12:05 -08:00
• 2a543d6efe Add LoRA support for Mixtral (#2831) Terry 2024-02-13 15:55:45 -08:00
• 317b29de0f Remove Yi model definition, please use LlamaForCausalLM instead (#2854) Philipp MoritzandRoy 2024-02-13 14:22:22 -08:00
• a463c333dd Use CuPy for CUDA graphs (#2811) Woosuk Kwon 2024-02-13 11:32:06 -08:00
• ea356004d4 Revert "Refactor llama family models (#2637)" (#2851) Philipp Moritz 2024-02-13 09:24:59 -08:00
• 5c976a7e1a Refactor llama family models (#2637) Roy 2024-02-13 16:09:23 +08:00
• f964493274 [CI] Ensure documentation build is checked in CI (#2842) Simon Mo 2024-02-12 22:53:07 -08:00
• a4211a4dc3 Serving Benchmark Refactoring (#2433) Roger Wang 2024-02-12 22:53:00 -08:00
• 563836496a Refactor 2 awq gemm kernels into m16nXk32 (#2723) RexandChunan Zeng 2024-02-12 11:02:17 -08:00
• 4ca2c358b1 Add documentation section about LoRA (#2834) Philipp Moritz 2024-02-12 08:24:45 -08:00
• 0580aab02f [ROCm] support Radeon™ 7900 series (gfx1100) without using flash-attention (#2768) Hongxia Yang 2024-02-11 02:14:37 -05:00
• 3711811b1d Disable custom all reduce by default (#2808) Woosuk Kwon 2024-02-08 09:58:03 -08:00
• 65b89d16ee [Ray] Integration compiled DAG off by default (#2471) SangBin Cho 2024-02-09 02:57:25 +09:00
• 931746bc6d Add documentation on how to do incremental builds (#2796) Philipp Moritz 2024-02-07 14:42:02 -08:00
• c81dddb45c [ROCm] Fix build problem resulted from previous commit related to FP8 kv-cache support (#2790) Hongxia Yang 2024-02-07 01:36:59 -05:00
• fe6d09ae61 [Minor] More fix of test_cache.py CI test failure (#2750) Lily Liu 2024-02-06 11:38:38 -08:00
• ed70c70ea3 modelscope: fix issue when model parameter is not a model id but path of the model. (#2489) liuyhwangyh 2024-02-07 01:57:15 +08:00
• f0d4e14557 Add fused top-K softmax kernel for MoE (#2769) Woosuk Kwon 2024-02-05 17:38:02 -08:00
• 2ccee3def6 [ROCm] Fixup arch checks for ROCM (#2627) Douglas Lehr 2024-02-05 16:59:09 -06:00
• b92adec8e8 Set local logging level via env variable (#2774) Lukas 2024-02-05 23:26:50 +01:00
• 56f738ae9b [ROCm] Fix some kernels failed unit tests (#2498) Hongxia Yang 2024-02-05 17:25:36 -05:00
• 72d3a30c63 [Minor] Fix benchmark_latency script (#2765) Woosuk Kwon 2024-02-05 12:45:37 -08:00
• c9b45adeeb Require triton >= 2.1.0 (#2746) whyiugandyangrui1 2024-02-05 15:07:36 +08:00
• 5a6c81b051 Remove eos tokens from output by default (#2611) Rex 2024-02-04 14:32:42 -08:00
• 51cd22ce56 set&get llm internal tokenizer instead of the TokenizerGroup (#2741) dancingpipiandshujunhua1 2024-02-05 06:25:36 +08:00
• 5ed704ec8c docs: fix langchain (#2736) Massimiliano Pronesti 2024-02-04 03:17:55 +01:00
• 4abf6336ec Add one example to run batch inference distributed on Ray (#2696) Cheng Su 2024-02-02 15:41:42 -08:00
• 0e163fce18 Fix default length_penalty to 1.0 (#2667) zspo 2024-02-02 07:59:39 +08:00
• 96b6f475dd Remove hardcoded device="cuda" to support more devices (#2503) Kunshang JiandJiang Li 2024-02-02 07:46:39 +08:00
• c410f5d020 Use revision when downloading the quantization config file (#2697) Pernekhan UtemuratovandPernekhan Utemuratov 2024-02-01 15:41:58 -08:00
• bb8c697ee0 Update README for meetup slides (#2718) Simon Mo 2024-02-01 14:56:53 -08:00
• b9e96b17de fix python 3.8 syntax (#2716) Simon Mo 2024-02-01 14:00:58 -08:00
• 923797fea4 Fix compile error when using rocm (#2648) zhaoyang-star 2024-02-02 01:35:09 +08:00
• cd9e60c76c Add Internlm2 (#2666) Fengzhe Zhou 2024-02-02 01:27:40 +08:00
• 93b38bea5d Refactor Prometheus and Add Request Level Metrics (#2316) Robert Shaw 2024-01-31 14:58:07 -08:00
• d0d93b92b1 Add unit test for Mixtral MoE layer (#2677) Philipp Moritz 2024-01-31 14:34:17 -08:00
• 89efcf1ce5 [Minor] Fix test_cache.py CI test failure (#2684) Philipp Moritz 2024-01-31 10:12:11 -08:00
• c664b0e683 fix some bugs (#2689) zspo 2024-02-01 02:09:23 +08:00
• d69ff0cbbb Fixes assertion failure in prefix caching: the lora index mapping should respect prefix_len (#2688) Tao He 2024-02-01 01:00:13 +08:00
• 1af090b57d Bump up version to v0.3.0 (#2656) Zhuohan Li 2024-01-31 00:07:07 -08:00
• 3dad944485 Add quantized mixtral support (#2673) Woosuk Kwon 2024-01-30 16:34:10 -08:00
• 105a40f53a [Minor] Fix false warning when TP=1 (#2674) Woosuk Kwon 2024-01-30 14:39:40 -08:00
• bbe9bd9684 [Minor] Fix a small typo (#2672) Philipp Moritz 2024-01-30 13:40:37 -08:00
• 4f65af0e25 Add swap_blocks unit tests (#2616) Vladimir 2024-01-30 18:30:50 +01:00
• d79ced3292 Fix 'Actor methods cannot be called directly' when using --engine-use-ray (#2664) Wen Sun 2024-01-31 00:17:05 +08:00
• ab40644669 Fused MOE for Mixtral (#2542) Philipp Moritzandchen shen 2024-01-29 22:43:37 -08:00
• 5d60def02c DeepseekMoE support with Fused MoE kernel (#2453) wangding zengandroy 2024-01-30 13:19:48 +08:00
• ea8489fce2 ROCm: Allow setting compilation target (#2581) Rasmus Larsen 2024-01-29 19:52:31 +01:00
• 1b20639a43 No repeated IPC open (#2642) Hanzhi Zhou 2024-01-30 02:46:29 +08:00
• b72af8f1ed Fix error when tp > 1 (#2644) zhaoyang-starandzhaoyang-star 2024-01-29 14:47:39 +08:00
• 9090bf02e7 Support FP8-E5M2 KV Cache (#2279) 3 people 2024-01-29 08:43:54 +08:00
• 7d648418b8 Update Ray version requirements (#2636) Simon Mo 2024-01-28 14:27:22 -08:00
• 89be30fa7d Small async_llm_engine refactor (#2618) Murali Andoorveedu 2024-01-27 23:28:37 -08:00
• f8ecb84c02 Speed up Punica compilation (#2632) Woosuk Kwon 2024-01-27 17:46:56 -08:00
• 5f036d2bcc [Minor] Fix warning on Ray dependencies (#2630) Woosuk Kwon 2024-01-27 15:43:40 -08:00
• 380170038e Implement custom all reduce kernels (#2192) Hanzhi Zhou 2024-01-28 04:46:35 +08:00
• 220a47627b Use head_dim in config if exists (#2622) Xiang Xu 2024-01-27 10:30:49 -08:00
• beb89f68b4 AWQ: Up to 2.66x higher throughput (#2566) Casper 2024-01-27 08:53:17 +01:00
• 390b495ff3 Don't build punica kernels by default (#2605) Philipp Moritz 2024-01-26 15:19:19 -08:00
• 3a0e1fc070 Support for Stable LM 2 (#2598) dakotamahan-stabilityandZhuohan Li 2024-01-26 14:45:19 -06:00
• 6b7de1a030 [ROCm] add support to ROCm 6.0 and MI300 (#2274) Hongxia Yang 2024-01-26 15:41:10 -05:00
• 5265631d15 use a correct device when creating OptionalCUDAGuard (#2583) Vladimir 2024-01-26 08:48:17 +01:00
• 2832e7b9f9 fix names and license for Qwen2 (#2589) Junyang Lin 2024-01-25 14:37:51 +08:00
• 3a7dd7e367 Support Batch Completion in Server (#2529) Simon Mo 2024-01-24 17:11:07 -08:00
• 223c19224b Fix the syntax error in the doc of supported_models (#2584) LastWhisper 2024-01-25 03:22:51 +08:00
• f1f6cc10c7 Added include_stop_str_in_output and length_penalty parameters to OpenAI API (#2562) Federico Galatolo 2024-01-24 19:21:56 +01:00
• 3209b49033 [Bugfix] fix crash if max_tokens=None (#2570) Nikola Borisov 2024-01-23 22:38:55 -08:00
• 1e4277d2d1 lint: format all python file instead of just source code (#2567) Simon Mo 2024-01-23 15:53:06 -08:00
• 9b945daaf1 [Experimental] Add multi-LoRA support (#1804) 4 people 2024-01-24 00:26:37 +01:00
• 9c1352eb57 [Feature] Simple API token authentication and pluggable middlewares (#1106) Erfan Al-Hossami 2024-01-23 18:13:00 -05:00
• 7a0b011dd5 Add a 1-line docstring to explain why calling context_attention_fwd twice in test_prefix_prefill.py (#2553) Jason Zhu 2024-01-22 14:47:25 -08:00
• 63e835cbcc Fix progress bar and allow HTTPS in benchmark_serving.py (#2552) Harry Mellor 2024-01-22 22:40:31 +00:00
• 94b5edeb53 Add qwen2 (#2495) Junyang Lin 2024-01-23 06:34:21 +08:00
• ab7e6006d6 Fix https://github.com/vllm-project/vllm/issues/2540 (#2545) Philipp Moritz 2024-01-22 10:02:38 -08:00
• 18bfcdd05c [Speculative decoding 2/9] Multi-step worker for draft model (#2424) Cade Daniel 2024-01-21 16:31:47 -08:00
• 71d63ed72e migrate pydantic from v1 to v2 (#2531) Jannis Schönleber 2024-01-22 01:05:56 +01:00
• d75c40734a [Fix] Keep scheduler.running as deque (#2523) Nick Hill 2024-01-20 22:36:09 -08:00
• 5b23c3f26f Add group as an argument in broadcast ops (#2522) Junda Chen 2024-01-20 16:00:26 -08:00
• 00efdc84ba Add benchmark serving to CI (#2505) Simon Mo 2024-01-19 20:20:19 -08:00
• 91a61da9b1 [Bugfix] fix load local safetensors model (#2512) Roy 2024-01-20 08:26:16 +08:00
• ef9b636e2d Simplify broadcast logic for control messages (#2501) Zhuohan Li 2024-01-19 11:23:30 -08:00
• 2709c0009a Support OpenAI API server in benchmark_serving.py (#2172) Harry Mellor 2024-01-19 04:34:08 +00:00
• dd7e8f5f64 refactor complemention api for readability (#2499) Simon Mo 2024-01-18 16:45:14 -08:00
• d2a68364c4 [BugFix] Fix abort_seq_group (#2463) ljss 2024-01-19 07:10:42 +08:00
• 7e1081139d Don't download both safetensor and bin files. (#2480) Nikola Borisov 2024-01-18 11:05:53 -08:00
• 18473cf498 [Neuron] Add an option to build with neuron (#2065) Liangfu Chen 2024-01-18 10:58:50 -08:00