Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 5b8a7c1cb0 [Misc] centralize all usage of environment variables (#4548) youkaichao 2024-05-02 11:13:25 -07:00
• 1ff0c73a79 [BugFix] Include target-device specific requirements.txt in sdist (#4559) Mark McLoughlin 2024-05-02 18:52:51 +01:00
• 5ad60b0cbd [Misc] Exclude the tests directory from being packaged (#4552) Hu Dong 2024-05-03 01:50:25 +08:00
• fb087af52e [mypy][7/N] Cover all directories (#4555) SangBin Cho 2024-05-03 02:47:41 +09:00
• 7038e8b803 [Kernel] Support running GPTQ 8-bit models in Marlin (#4533) alexm-nm 2024-05-02 12:56:22 -04:00
• 2a85f93007 [Core][Distributed] enable multiple tp group (#4512) youkaichaoandZhuohan Li 2024-05-01 21:28:21 -07:00
• cf8cac8c70 [mypy][6/N] Fix all the core subdirectory typing (#4450) SangBin ChoandCade Daniel 2024-05-02 12:01:00 +09:00
• 5e401bce17 [CI]Add regression tests to ensure the async engine generates metrics (#4524) Ronen Schaffer 2024-05-02 05:57:12 +03:00
• 0d62fe58db [Bug fix][Core] assert num_new_tokens == 1 fails when SamplingParams.n is not 1 and max_tokens is large & Add tests for preemption (#4451) SangBin Cho 2024-05-02 11:24:13 +09:00
• b8afa8b95a [MISC] Rework logger to enable pythonic custom logging configuration to be provided (#4273) Danny Guinther 2024-05-01 20:34:40 -04:00
• 826b82a260 [Misc] Fix expert_ids shape in MoE (#4517) Woosuk Kwon 2024-05-01 16:47:59 -07:00
• c9d852d601 [Misc] Remove Mixtral device="cuda" declarations (#4543) Philipp Moritz 2024-05-01 16:30:52 -07:00
• 6ef09b08f8 [Core][Distributed] fix pynccl del error (#4508) youkaichao 2024-05-01 15:23:06 -07:00
• 3a922c1e7e [Bugfix][Core] Fix and refactor logging stats (#4336) Roy 2024-05-02 04:08:14 +08:00
• c47ba4aaa9 [Bugfix] Add validation for seed (#4529) sasha0552 2024-05-01 19:31:22 +00:00
• 24bb4fe432 [Kernel] Update fused_moe tuning script for FP8 (#4457) Philipp Moritz 2024-05-01 11:47:38 -07:00
• a657bfc48a [Core] Add multiproc_worker_utils for multiprocessing-based workers (#4357) Nick Hill 2024-05-01 11:41:59 -07:00
• 24750f4cad [Core] Enable prefix caching with block manager v2 enabled (#4142) 3 people 2024-05-02 02:20:32 +08:00
• b38e42fbca [Speculative decoding] Add ngram prompt lookup decoding (#4237) leiwen83andLei Wen 2024-05-02 02:13:03 +08:00
• 8b798eec75 [CI/Build][Bugfix] VLLM_USE_PRECOMPILED should skip compilation (#4534) Travis Johnson 2024-05-01 12:01:50 -06:00
• 69909126a7 [Bugfix] Use random seed if seed is -1 (#4531) sasha0552 2024-05-01 17:41:17 +00:00
• e491c7e053 [Doc] update(example model): for OpenAI compatible serving (#4503) Frαnçois 2024-05-01 19:14:16 +02:00
• 4dc8026d86 [Bugfix] Fix 307 Redirect for /metrics (#4523) Robert Shaw 2024-05-01 12:14:13 -04:00
• a88bb9b032 [Bugfix] Fix the fp8 kv_cache check error that occurs when failing to obtain the CUDA version. (#4173) AnyISalIn 2024-05-02 00:11:03 +08:00
• 6f1df80436 [Test] Add ignore_eos test (#4519) SangBin Cho 2024-05-01 21:45:42 +09:00
• d6f4bd7cdd [Misc]Add customized information for models (#4132) Jee Li 2024-05-01 12:18:14 +08:00
• c3845d82dc Allow user to define whitespace pattern for outlines (#4305) Robert Caulk 2024-05-01 05:48:39 +02:00
• a822eb3413 [Misc] fix typo in block manager (#4453) Pastel！ 2024-05-01 11:41:32 +08:00
• f458112e8a [Misc][Typo] type annotation fix (#4495) harrywu 2024-05-01 11:21:39 +08:00
• 2e240c69a9 [Core] Centralize GPU Worker construction (#4419) Nick Hill 2024-04-30 18:06:34 -07:00
• ee37328da0 Unable to find Punica extension issue during source code installation (#4494) fuchen.ljlandSimon Mo 2024-05-01 08:42:09 +08:00
• 6ad58f42c5 fix_tokenizer_snapshot_download_bug (#4493) fuchen.ljl 2024-05-01 07:38:50 +08:00
• dd1a50a8bc [Bugfix][Minor] Make ignore_eos effective (#4468) Li, Jiang 2024-05-01 07:33:33 +08:00
• 715c2d854d [Frontend] [Core] Tensorizer: support dynamic num_readers, update version (#4467) Alpay Ariyak 2024-04-30 19:32:13 -04:00
• a494140433 [Frontend] Support complex message content for chat completions endpoint (#3467) 3 people 2024-05-01 01:28:46 +02:00
• 111815d482 [Kernel] Support Fp8 Checkpoints (Dynamic + Static) (#4332) 6 people 2024-04-30 17:46:12 -04:00
• b31a1fb63c [Doc] add visualization for multi-stage dockerfile (#4456) Prashant GuptaandRoger Wang 2024-04-30 10:41:59 -07:00
• 4bb53e2dde [BugFix] fix num_lookahead_slots missing in async executor (#4165) leiwen83andLei Wen 2024-05-01 01:12:59 +08:00
• 26f2fb5113 [Core]Refactor gptq_marlin ops (#4466) Kunshang Ji 2024-04-30 12:14:47 +00:00
• fa32207842 [Bugfix][Kernel] Fix compute_type for MoE kernel (#4463) Woosuk Kwon 2024-04-29 22:05:40 -07:00
• d627a3d837 [Misc] Upgrade to torch==2.3.0 (#4454) Michael Goin 2024-04-29 20:05:47 -04:00
• f4f921b7f1 [Core][Distributed] use cpu group to broadcast metadata in cpu (#4444) youkaichao 2024-04-29 13:52:22 -07:00
• ac5ccf0156 [CI] hotfix: soft fail neuron test (#4458) Simon Mo 2024-04-29 12:50:01 -07:00
• 73c8d677e5 [Kernel] Marlin Expansion: Support AutoGPTQ Models with Marlin (#3922) 3 people 2024-04-29 12:35:34 -04:00
• df29793dc7 [mypy][5/N] Support all typing on model executor (#4427) SangBin Cho 2024-04-29 11:01:26 +09:00
• 03dd7d52bf [CI] clean docker cache for neuron (#4441) Simon Mo 2024-04-28 16:32:07 -07:00
• bf480c5302 Add more Prometheus metrics (#2764) 3 people 2024-04-29 01:59:33 +03:00
• 9c7306ac11 [Misc] fix typo in llm_engine init logging (#4428) DefTruth 2024-04-28 18:58:30 +08:00
• 4ea1f9678d [BugFix] Resolved Issues For LinearMethod --> QuantConfig (#4418) Robert Shaw 2024-04-27 14:35:33 -04:00
• ba4be44c32 [BugFix] Fix return type of executor execute_model methods (#4402) Nick Hill 2024-04-27 11:17:45 -07:00
• d6e520e170 [Core] Support offline use of local cache for models (#4374) Prashant GuptaandTravis Johnson 2024-04-27 09:59:55 -07:00
• 81661da7b2 [BugFix] Fix min_tokens when eos_token_id is None (#4389) Nick HillandDefTruth 2024-04-27 09:52:46 -07:00
• dfea173148 [Bugfix] Abort requests when the connection to /v1/completions is interrupted (#4363) Ruoyu Qin 2024-04-28 00:48:37 +08:00
• 7134303cbb [Bugfix][Core] Fix get decoding config from ray (#4335) Roy 2024-04-27 19:30:08 +08:00
• 3da24c2df7 [Model] Phi-3 4k sliding window temp. fix (#4380) Caio Mendes 2024-04-27 07:08:15 -03:00
• eefeb16464 [Kernel] Full Tensor Parallelism for LoRA Layers (#3524) Austin VeselkaandAntoni Baum 2024-04-27 02:03:48 -05:00
• 18d23f642a [ROCm][Hardware][AMD] Enable group query attention for triton FA (#4406) Hongxia Yang 2024-04-27 02:37:40 -04:00
• 87f545ba6f [Misc] Fix logger format typo (#4396) Roy 2024-04-27 13:45:02 +08:00
• 8947bc3c15 [Frontend][Bugfix] Disallow extra fields in OpenAI API (#4355) Cyrus Leung 2024-04-27 13:08:24 +08:00
• 12628d3c78 [Kernel] Optimize FP8 support for MoE kernel / Mixtral via static scales (#4343) Philipp MoritzandWoosuk Kwon 2024-04-26 21:49:59 -07:00
• 258a2c58d0 [Core] Introduce DistributedGPUExecutor abstract class (#4348) Nick Hill 2024-04-26 21:14:26 -07:00
• aba47be3fe [Misc] add RFC issue template (#4401) youkaichaoandSimon Mo 2024-04-26 15:47:45 -07:00
• a62aaf1df5 [Misc][Refactor] Generalize linear_method to be quant_method (#4373) Cody Yu 2024-04-26 13:41:14 -07:00
• 603ad84815 [Core] Refactoring sampler and support prompt logprob for chunked prefill (#4309) SangBin Cho 2024-04-26 22:02:02 +09:00
• a88081bf76 [CI] Disable non-lazy string operation on logging (#4326) SangBin ChoandDanny Guinther 2024-04-26 16:16:58 +09:00
• 2f30e7c72f [Frontend] Add --log-level option to api server (#4377) Norman Mu 2024-04-25 22:36:01 -07:00
• a74dee9b62 [Bugfix] Fix parameter name in get_tokenizer (#4107) Cyrus Leung 2024-04-26 10:10:48 +08:00
• cf29b7eda4 [ROCm][Hardware][AMD][Doc] Documentation update for ROCm (#4376) Hongxia YangandWoosukKwon 2024-04-25 21:12:25 -04:00
• efffb63f58 [Core] Move function tracing setup to util function (#4352) Nick Hill 2024-04-25 16:45:12 -07:00
• 15e7c675b0 [Core] Add shutdown() method to ExecutorBase (#4349) Nick Hill 2024-04-25 16:32:48 -07:00
• b6dcb4d442 [Misc] Fix flash attention backend log (#4368) Roy 2024-04-26 03:43:32 +08:00
• b5b4a398a7 [Mypy] Typing lora folder (#4337) SangBin Cho 2024-04-26 04:13:50 +09:00
• f4bc4de1b1 [Core]refactor aqlm quant ops (#4351) Kunshang Ji 2024-04-25 19:03:56 +00:00
• bd7a8eef25 [Doc] README Phi-3 name fix. (#4372) Caio MendesandCaio Mendes 2024-04-25 14:32:00 -03:00
• 7ee82bef1e [CI/Build] Adding functionality to reset the node's GPUs before processing. (#4213) Alexei-V-Ivanov-AMD 2024-04-25 11:37:20 -05:00
• fbf152d976 [Bugfix][Model] Refactor OLMo model to support new HF format in transformers 4.40.0 (#4324) Isotr0pyandWoosuk Kwon 2024-04-26 00:35:56 +08:00
• 479d69fad0 [Core] Move ray_utils.py from engine to executor package (#4347) Nick Hill 2024-04-24 23:52:22 -07:00
• 96e90fdeb3 [Model] Adds Phi-3 support (#4298) Caio Mendes 2024-04-25 00:06:57 -03:00
• a395a638c2 [Misc] Use public API in benchmark_throughput (#4300) zifeitong 2024-04-24 14:10:24 -07:00
• 2768884ac4 [Doc] Add note for docker user (#4340) youkaichaoandSimon Mo 2024-04-24 14:09:44 -07:00
• aae08249ac [Bugfix] Fix marlin kernel crash on H100 (#4218) alexm-nm 2024-04-24 13:35:01 -04:00
• 7923dcad12 [Misc] Update ShareGPT Dataset Sampling in Serving Benchmark (#4279) Roger Wang 2024-04-24 09:49:13 -07:00
• 3cd9b5bb2d [Core][Distributed] use existing torch.cuda.device (#4318) youkaichao 2024-04-24 09:00:20 -07:00
• 468d761b32 [Misc] Reduce supported Punica dtypes (#4304) Woosuk Kwon 2024-04-23 18:54:33 -07:00
• e4bf860a54 [CI][Build] change pynvml to nvidia-ml-py (#4302) youkaichao 2024-04-23 18:33:12 -07:00
• 91f50a6fe2 [Core][Distributed] use cpu/gloo to initialize pynccl (#4248) youkaichao 2024-04-23 18:32:19 -07:00
• 79a268c4ab [BUG] fixed fp8 conflict with aqlm (#4307) Robert Shaw 2024-04-23 21:26:33 -04:00
• eace8bf0b9 [Kernel] FP8 support for MoE kernel / Mixtral (#4244) Philipp Moritz 2024-04-23 18:18:23 -07:00
• 1e8f4252aa [Bugfix][Frontend] Raise exception when file-like chat template fails to be opened (#4292) Cyrus Leung 2024-04-24 02:19:03 +08:00
• 2b7949c1c2 AQLM CUDA support (#3287) James Flemingandmgoin 2024-04-23 13:59:33 -04:00
• 62b5166bd4 [CI] Add ccache for wheel builds job (#4281) Simon Mo 2024-04-23 09:51:41 -07:00
• d86285a4a4 [Core][Logging] Add last frame information for better debugging (#4278) youkaichao 2024-04-23 09:45:52 -07:00
• d87f39e9a9 [Bugfix] Add init_cached_hf_modules to RayWorkerWrapper (#4286) DefTruth 2024-04-24 00:28:35 +08:00
• d3c8180ac4 [Bugfix] Fixing max token error message for openai compatible server (#4016) Jack Gordley 2024-04-23 12:06:29 +01:00
• 62b8aebc6f [Speculative decoding 7/9] Speculative decoding end-to-end correctness tests. (#3951) Cade Daniel 2024-04-23 01:02:36 -07:00
• 050f285ff6 [Core] Scheduling optimization 2 (#4280) SangBin Cho 2024-04-23 17:02:11 +09:00
• 8f2ea22bde [Core] Some simplification of WorkerWrapper changes (#4183) Nick Hill 2024-04-23 00:49:08 -07:00
• 0ae11f78ab [Mypy] Part 3 fix typing for nested directories for most of directory (#4161) SangBin Cho 2024-04-23 13:32:44 +09:00
• 34128a697e Fix autodoc directives (#4272) Harry MellorandHarry Mellor 2024-04-23 02:53:01 +01:00
• c1b4e4157c [Core][Distributed] use absolute path for library file (#4271) youkaichao 2024-04-22 17:21:48 -07:00