Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 4dd4b5c538 Bump up to v0.2.4 (#2034) Woosuk Kwon 2023-12-11 11:49:39 -08:00
• 6120e5aaea Fix import error msg for megablocks (#2038) Woosuk Kwon 2023-12-11 11:40:56 -08:00
• 2eaa81b236 Update README.md to add megablocks requirement for mixtral (#2033) Ram 2023-12-12 01:07:34 +05:30
• 81ce2a4b26 [Minor] Fix type annotation in Mixtral (#2036) Woosuk Kwon 2023-12-11 11:32:39 -08:00
• 5dd80d3777 Fix latency benchmark script (#2035) Woosuk Kwon 2023-12-11 11:19:08 -08:00
• beeee69bc9 Revert adding Megablocks (#2030) Woosuk Kwon 2023-12-11 10:49:00 -08:00
• 9bf28d0b69 Update requirements.txt for mixtral (#2029) Ram 2023-12-12 00:09:29 +05:30
• c0ce15dfb2 Update run_on_sky.rst (#2025) Ikko Eltociear Ashimine 2023-12-12 03:32:58 +09:00
• b9bcdc7158 Change the load format to pt for Mixtral (#2028) Woosuk Kwon 2023-12-11 10:32:17 -08:00
• 4ff0203987 Minor fixes for Mixtral (#2015) Woosuk Kwon 2023-12-11 09:16:15 -08:00
• b5f882cc98 Mixtral 8x7B support (#2011) 3 people 2023-12-11 10:09:15 +01:00
• 2e8fc0d4c3 Fix completion API echo and logprob combo (#1992) Simon Mo 2023-12-10 13:20:30 -08:00
• dacaf5a400 Replace head_mapping params with num_kv_heads to attention kernel. (#1997) 3 people 2023-12-11 02:12:53 +08:00
• 24cde76a15 [Minor] Add comment on skipping rope caches (#2004) Woosuk Kwon 2023-12-10 10:04:12 -08:00
• 1aa1361510 Fix OpenAI server completion_tokens referenced before assignment (#1996) Jin Shang 2023-12-10 13:01:21 +08:00
• fe470ae5ad [Minor] Fix code style for baichuan (#2003) Woosuk Kwon 2023-12-09 19:24:29 -08:00
• 3a8c2381f7 Fix for KeyError on Loading LLaMA (#1978) Jun Gao 2023-12-10 07:59:57 +08:00
• c85b80c2b6 [Docker] Add cuda arch list as build option (#1950) Simon Mo 2023-12-08 09:53:47 -08:00
• 2b981012a6 Fix Baichuan2-7B-Chat (#1987) firebook 2023-12-09 01:38:36 +08:00
• 6ccc0bfffb Merge EmbeddedLLM/vllm-rocm into vLLM main (#1836) 7 people 2023-12-08 15:16:52 +08:00
• c8e7eb1eb3 fix typo in getenv call (#1972) Daya Khudia 2023-12-07 16:04:41 -08:00
• 24f60a54f4 [Docker] Adding number of nvcc_threads during build as envar (#1893) AguirreNicolas 2023-12-07 16:00:32 -03:00
• 42c02f5892 Fix quickstart.rst typo jinja (#1964) gottlike 2023-12-07 17:34:44 +01:00
• ebede26ebf Make InternLM follow rope_scaling in config.json (#1956) Jie Liandlijie8 2023-12-08 00:32:08 +08:00
• d940ce497e Fix typo in adding_model.rst (#1947) Peter Götz 2023-12-06 19:04:26 +01:00
• 05ff90b692 Save pytorch profiler output for latency benchmark (#1871) Antoni Baum 2023-12-05 20:55:55 -08:00
• 1d9b737e05 Support ChatGLMForConditionalGeneration (#1932) dancingpipiandshujunhua1 2023-12-06 02:52:48 +08:00
• 60dc62dc9e add custom server params (#1868) Roy 2023-12-04 04:59:18 +08:00
• 0f90effc66 Bump up to v0.2.3 (#1903) Woosuk Kwon 2023-12-03 12:27:47 -08:00
• 464dd985e3 Fix num_gpus when TP > 1 (#1852) Woosuk Kwon 2023-12-03 12:24:30 -08:00
• c07a442854 chore(examples-docs): upgrade to OpenAI V1 (#1785) Massimiliano Pronesti 2023-12-03 10:11:22 +01:00
• cd3aa153a4 Fix broken worker test (#1900) Woosuk Kwon 2023-12-02 22:17:33 -08:00
• 9b294976a2 Add PyTorch-native implementation of custom layers (#1898) Woosuk Kwon 2023-12-02 21:18:40 -08:00
• 5313c2cb8b Add Production Metrics in Prometheus format (#1890) Simon Mo 2023-12-02 16:37:44 -08:00
• 5f09cbdb63 Fix broken sampler tests (#1896) Woosuk KwonandAntoni Baum 2023-12-02 16:06:17 -08:00
• 4cefa9b49b [Docs] Update the AWQ documentation to highlight performance issue (#1883) Simon Mo 2023-12-02 15:52:47 -08:00
• f86bd6190a Fix the typo in SamplingParams' docstring (#1886) Jerry 2023-12-01 18:06:36 +08:00
• e5452ddfd6 Normalize head weights for Baichuan 2 (#1876) Woosuk Kwon 2023-11-30 20:03:58 -08:00
• d06980dfa7 Fix Baichuan tokenizer error (#1874) Woosuk Kwon 2023-11-30 18:35:50 -08:00
• 66785cc05c Support chat template and echo for chat API (#1756) Adam Brusselback 2023-11-30 19:43:13 -05:00
• 05a38612b0 docs: add instruction for langchain (#1162) Massimiliano Pronesti 2023-11-30 19:57:44 +01:00
• d27f4bae39 Fix rope cache key error (#1867) Roy 2023-12-01 00:29:28 +08:00
• 8d8c2f6ffe Support max-model-len argument for throughput benchmark (#1858) aisensiy 2023-12-01 00:10:24 +08:00
• 51d3cb951d Remove max_num_seqs in latency benchmark script (#1855) Woosuk Kwon 2023-11-30 00:00:32 -08:00
• e74b1736a1 Add profile option to latency benchmark script (#1839) Woosuk Kwon 2023-11-29 23:42:52 -08:00
• f07c1ceaa5 [FIX] Fix docker build error (#1831) (#1832) AllenandAntoni Baum 2023-11-30 15:06:50 +08:00
• 63b2206ad0 Avoid multiple instantiations of the RoPE class (#1828) Jee Li 2023-11-30 15:06:27 +08:00
• 27feead2f8 Refactor Worker & InputMetadata (#1843) Woosuk Kwon 2023-11-29 22:16:37 -08:00
• c782195662 Disable Logs Requests should Disable Logging of requests. (#1779) Michael McCullochandMichael McCulloch 2023-11-29 22:50:02 -07:00
• 0f621c2c7d [Docs] Add information about using shared memory in docker (#1845) Simon Mo 2023-11-29 18:33:56 -08:00
• a9e4574261 Refactor Attention (#1840) Woosuk Kwon 2023-11-29 15:37:31 -08:00
• 0229c386c5 Better integration with Ray Serve (#1821) FlorianJoncourandFlorianJoncour 2023-11-29 21:25:43 +00:00
• a7b3e33078 [Fix] Fix RoPE in ChatGLM-32K (#1841) Woosuk Kwon 2023-11-29 13:01:19 -08:00
• e19a64c7ef [FIX] Fix formatting error in main branch (#1822) Zhuohan Li 2023-11-28 16:56:43 -08:00
• 1cb4ad8de9 [FIX] Fix formatting error Zhuohan Li 2023-11-29 00:40:19 +00:00
• 6ed068a71a Use the type BlockTable (#1791) explainerauthors 2023-11-28 16:34:05 -08:00
• 708e6c18b0 [FIX] Fix class naming (#1803) Zhuohan Li 2023-11-28 14:08:01 -08:00
• b943890484 Fix OPT param names (#1819) Woosuk Kwon 2023-11-28 11:22:44 -08:00
• a1125ad4df Correct comments in parallel_state.py (#1818) explainerauthors 2023-11-28 10:19:35 -08:00
• a8b150c595 Init model on GPU to reduce CPU memory footprint (#1796) ljss 2023-11-28 03:18:26 +08:00
• 665cbcec4b Added echo function to OpenAI API server. (#1504) Yunmo Chen 2023-11-27 13:29:17 +08:00
• 7c600440f7 Fix model docstrings (#1764) Woosuk Kwon 2023-11-23 23:04:44 -08:00
• e0c6f556e8 [Build] Avoid building too many extensions (#1624) Yanming W 2023-11-23 16:31:19 -08:00
• de23687d16 Fix repetition penalty aligned with huggingface (#1577) ljss 2023-11-23 06:41:44 +08:00
• 4cea74c73b Set top_p=0 and top_k=-1 in greedy sampling (#1748) ljss 2023-11-23 04:51:09 +08:00
• a921d8be9d [DOCS] Add engine args documentation (#1741) Casper 2023-11-22 21:31:27 +01:00
• 094f716bf2 Add stop_token_ids in SamplingParams.__repr__ (#1745) 陈序 2023-11-22 12:13:53 +08:00
• 7d761fe3c1 [FIX] Fix the case when input_is_parallel=False for ScaledActivation (#1737) Zhuohan Li 2023-11-20 23:56:48 -08:00
• cf35d8f3d7 [BugFix] Fix TP support for AWQ (#1731) Woosuk Kwon 2023-11-20 21:42:45 -08:00
• 4bb6b67188 fix RAM OOM when load large models in tensor parallel mode. (#1395) boydfdandran_lin 2023-11-21 11:02:42 +08:00
• 819b18e7ba Rewrite torch.repeat_interleave to remove cpu synchronization (#1599) ljss 2023-11-21 09:46:32 +08:00
• 19849db573 [Fix] Fix bugs in scheduler (#1727) Zhuofan 2023-11-21 08:10:50 +08:00
• 3d4ceb292c Fix hanging in the scheduler caused by long prompts (#1534) 陈序 2023-11-21 08:06:49 +08:00
• f5a37c6c6c [BugFix] Fix a bug in loading safetensors (#1732) Woosuk Kwon 2023-11-20 15:51:18 -08:00
• 32c927b53f [FIX] Update the doc link in README.md (#1730) Zhuohan Li 2023-11-20 12:46:24 -08:00
• 5ffc0d13a2 Migrate linter from pylint to ruff (#1665) Simon Mo 2023-11-20 11:58:01 -08:00
• 112627e8b2 [Docs] Fix the code block's format in deploying_with_docker page (#1722) Wen Sun 2023-11-20 17:22:39 +08:00
• 37c1e3c218 Documentation about official docker image (#1709) Simon Mo 2023-11-19 20:56:26 -08:00
• 06e9ebebd5 Add instructions to install vLLM+cu118 (#1717) Woosuk Kwon 2023-11-18 23:48:58 -08:00
• c5f7740d89 Bump up to v0.2.2 (#1689) Woosuk Kwon 2023-11-18 21:57:07 -08:00
• be66d9b125 Fix warning msg on quantization (#1715) Woosuk Kwon 2023-11-18 21:49:55 -08:00
• e1054247ba [Optimization] Implement fused add rmsnorm (#1667) ljss 2023-11-19 10:18:02 +08:00
• 8d17774f92 Add AWQ support for all models (#1714) Woosuk Kwon 2023-11-18 17:56:47 -08:00
• e946260cf3 use get_tensor in safe_open (#1696) twaka 2023-11-19 09:45:18 +09:00
• edb305584b Support download models from www.modelscope.cn (#1588) liuyhwangyh 2023-11-18 12:38:31 +08:00
• bb00f66e19 Use quantization_config in hf config (#1695) Woosuk Kwon 2023-11-17 16:23:49 -08:00
• e87557b069 Support Min P Sampler (#1642) Roy 2023-11-18 08:20:49 +08:00
• dcc543a298 [Minor] Fix comment (#1704) Zhuofan 2023-11-18 01:42:49 +08:00
• 0fc280b06c Update the adding-model doc according to the new refactor (#1692) Zhuohan Li 2023-11-16 18:46:26 -08:00
• 20d0699d49 [Fix] Fix comm test (#1691) Zhuohan Li 2023-11-16 16:28:39 -08:00
• 686f5e3210 Return usage for openai streaming requests (#1663) Iskren Ivov Chernev 2023-11-17 01:28:36 +02:00
• 415d109527 [Fix] Update Supported Models List (#1690) Zhuohan Li 2023-11-16 14:47:26 -08:00
• 521b35f799 Support Microsoft Phi 1.5 (#1664) maximzubkov 2023-11-16 23:28:39 +01:00
• cb08cd0d75 [Minor] Fix duplication of ignored seq group in engine step (#1666) Simon Mo 2023-11-16 13:11:41 -08:00
• 2a2c135b41 Fix loading error when safetensors contains empty tensor (#1687) twaka 2023-11-17 03:38:10 +09:00
• 65ea2ddf17 feat(config): support parsing torch.dtype (#1641) Aaron Pham 2023-11-16 04:31:06 -05:00
• b514d3c496 Revert MptConfig to MPTConfig (#1668) Megha Agarwal 2023-11-16 01:19:39 -08:00
• 7076fa1c9f TP/quantization/weight loading refactor part 2 - Refactor quantized linear logic and extend quantization support to all models (#1622) Zhuohan Li 2023-11-15 22:50:41 -08:00
• 660a7fcfa4 Add DeepSpeed MII backend to benchmark script (#1649) Woosuk Kwon 2023-11-14 12:35:30 -08:00
• 054072bee5 [Minor] Move RoPE selection logic to get_rope (#1633) Woosuk Kwon 2023-11-12 16:04:50 -08:00