Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• e72ae80b06 [Bugfix] Support 2D input shape in MoE layer (#6287) Woosuk Kwon 2024-07-10 06:03:16 -07:00
• 8a924d2248 [Doc] Guide for adding multi-modal plugins (#6205) Cyrus Leung 2024-07-10 14:55:34 +08:00
• 5ed3505d82 [Bugfix][TPU] Add prompt adapter methods to TPUExecutor (#6279) Woosuk Kwon 2024-07-09 19:30:56 -07:00
• da78caecfa [core][distributed] zmq fallback for broadcasting large objects (#6183) youkaichao 2024-07-09 18:49:11 -07:00
• 2416b26e11 [Speculative Decoding] Medusa Implementation with Top-1 proposer (#4978) Abhinav Goyal 2024-07-10 07:04:02 +05:30
• d3a245138a [Bugfix]fix and needs_scalar_to_array logic check (#6238) Baoyuan QiandRobert Shaw 2024-07-10 07:43:24 +08:00
• 673dd4cae9 [Docs] Docs update for Pipeline Parallel (#6222) Murali AndoorveeduandSimon Mo 2024-07-09 16:24:58 -07:00
• 4d6ada947c [CORE] Adding support for insertion of soft-tuned prompts (#4645) 4 people 2024-07-09 16:26:36 -04:00
• a0550cbc80 Add support for multi-node on CI (#5955) Kevin H. Luu 2024-07-09 12:56:56 -07:00
• 08c5bdecae [Bugfix][TPU] Fix outlines installation in TPU Dockerfile (#6256) Woosuk Kwon 2024-07-09 02:56:06 -07:00
• 5d5b4c5fe5 [Bugfix][TPU] Add missing None to model input (#6245) Woosuk Kwon 2024-07-09 00:21:37 -07:00
• 70c232f85a [core][distributed] fix ray worker rank assignment (#6235) youkaichao 2024-07-08 21:31:44 -07:00
• a3c9435d93 [hardware][cuda] use device id under CUDA_VISIBLE_DEVICES for get_device_capability (#6216) youkaichao 2024-07-08 20:02:15 -07:00
• 4f0e0ea131 Add FlashInfer to default Dockerfile (#6172) Simon Mo 2024-07-08 13:38:03 -07:00
• ddc369fba1 [Bugfix] Mamba cache Cuda Graph padding (#6214) tomeras91 2024-07-08 21:25:51 +03:00
• 185ad31f37 [Bugfix] use diskcache in outlines _get_guide #5436 (#6203) Eric 2024-07-09 02:23:24 +08:00
• 543aa48573 [Kernel] Correctly invoke prefill & decode kernels for cross-attention (towards eventual encoder/decoder model support) (#4888) afeldman-nmandWoosuk Kwon 2024-07-08 13:12:15 -04:00
• f7a8fa39d8 [Kernel] reloading fused_moe config on the last chunk (#6210) Avshalom Manevich 2024-07-08 18:00:38 +03:00
• 717f4bcea0 Feature/add benchmark testing (#5947) HaichuanandRoger Wang 2024-07-08 15:52:06 +08:00
• 16620f439d do not exclude object field in CompletionStreamResponse (#6196) kczimm 2024-07-07 21:32:57 -05:00
• 3b08fe2b13 [misc][frontend] log all available endpoints (#6195) youkaichaoandCody Yu 2024-07-07 15:11:12 -07:00
• abfe705a02 [ Misc ] Support Fp8 via llm-compressor (#6110) Robert ShawandRobert Shaw 2024-07-07 16:42:11 -04:00
• 333306a252 add benchmark for fix length input and output (#5857) HaichuanandRoger Wang 2024-07-07 15:42:13 +08:00
• 6206dcb29e [Model] Add PaliGemma (#5189) Roger WangandWoosuk Kwon 2024-07-06 18:25:50 -07:00
• 9389380015 [Doc] Move guide for multimodal model and other improvements (#6168) Cyrus Leung 2024-07-06 17:18:59 +08:00
• 175c43eca4 [Doc] Reorganize Supported Models by Type (#6167) Roger Wang 2024-07-05 22:59:36 -07:00
• bc96d5c330 Move release wheel env var to Dockerfile instead (#6163) Simon Mo 2024-07-05 17:19:53 -07:00
• f0250620dd Fix release wheel build env var (#6162) Simon Mo 2024-07-05 16:24:31 -07:00
• 2de490d60f Update wheel builds to strip debug (#6161) Simon Mo 2024-07-05 14:51:25 -07:00
• 79d406e918 [Docs] Fix readthedocs for tag build (#6158) Simon Mo 2024-07-05 12:44:40 -07:00
• abad5746a7 bump version to v0.5.1 (#6157) Simon Mo 2024-07-05 12:04:51 -07:00
• e58294ddf2 [Bugfix] Add verbose error if scipy is missing for blocksparse attention (#5695) JGSweets 2024-07-05 12:41:01 -05:00
• f1e15da6fe [Frontend] Continuous usage stats in OpenAI completion API (#5742) jvlunteren 2024-07-05 19:37:09 +02:00
• 0097bb1829 [Bugfix] Use templated datasource in grafana.json to allow automatic imports (#6136) Christian Rohmann 2024-07-05 18:49:47 +02:00
• ea4b570483 [VLM] Cleanup validation and update docs (#6149) Cyrus Leung 2024-07-05 13:49:38 +08:00
• a41357e941 [VLM] Improve consistency between feature size calculation and dummy data for profiling (#6146) Roger Wang 2024-07-04 18:29:47 -07:00
• ae96ef8fbd [VLM] Calculate maximum number of multi-modal tokens by model (#6121) Cyrus Leung 2024-07-05 07:37:23 +08:00
• 69ec3ca14c [Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051) Lily LiuandSimon Mo 2024-07-04 16:35:51 -07:00
• 81d7a50f24 [Hardware][Intel CPU] Adding intel openmp tunings in Docker file (#6008) Yuan 2024-07-05 06:22:12 +08:00
• 27902d42be [misc][doc] try to add warning for latest html (#5979) youkaichao 2024-07-04 09:57:09 -07:00
• 56b325e977 [ROCm][AMD][Model]Adding alibi slopes support in ROCm triton flash attention and naive flash attention (#6043) Gregory ShtrasbergandHongxia Yang 2024-07-04 01:19:38 -04:00
• 3dd507083f [CI/Build] Cleanup VLM tests (#6107) Cyrus Leung 2024-07-04 09:58:18 +08:00
• 0ed646b7aa [Distributed][Core] Support Py39 and Py38 for PP (#6120) Murali Andoorveedu 2024-07-03 17:52:29 -07:00
• 1dab9bc8a9 [Bugfix] set OMP_NUM_THREADS to 1 by default for multiprocessing (#6109) Travis JohnsonandNick Hill 2024-07-03 17:56:59 -06:00
• 3de6e6a30e [core][distributed] support n layers % pp size != 0 (#6115) youkaichao 2024-07-03 16:40:31 -07:00
• 966fe72141 [doc][misc] bump up py version in installation doc (#6119) youkaichao 2024-07-03 15:52:04 -07:00
• 62963d129e [ Misc ] Clean Up CompressedTensorsW8A8 (#6113) Robert Shaw 2024-07-03 18:50:08 -04:00
• d9e98f42e4 [vlm] Remove vision language config. (#6089) xwjiang2010andRoger Wang 2024-07-03 15:14:16 -07:00
• 3c6325f0fc [core][distributed] custom allreduce when pp size > 1 (#6117) youkaichao 2024-07-03 14:41:32 -07:00
• 47f0954af0 [Kernel] Expand FP8 support to Ampere GPUs using FP8 Marlin (#5975) Michael Goin 2024-07-03 13:38:00 -04:00
• 7cd2ebb025 [Bugfix] Fix compute_logits in Jamba (#6093) Roger Wang 2024-07-03 00:32:35 -07:00
• f1c78138aa [Doc] Fix Mock Import (#6094) Roger Wang 2024-07-03 00:13:56 -07:00
• 3a86b54fb0 [VLM][Frontend] Proper Image Prompt Formatting from OpenAI API (#6091) Roger WangandCyrus Leung 2024-07-02 23:41:23 -07:00
• f666207161 [misc][distributed] error on invalid state (#6092) youkaichao 2024-07-02 23:37:29 -07:00
• d830656a97 [BugFix] Avoid unnecessary Ray import warnings (#6079) Nick Hill 2024-07-02 23:09:40 -07:00
• d18bab3587 [CI] Fix base url doesn't strip "/" (#6087) SangBin Cho 2024-07-03 13:31:25 +09:00
• 9831aec49f [Core] Dynamic image size support for VLMs (#5276) 5 people 2024-07-03 11:34:00 +08:00
• 482045ee77 [hardware][misc] introduce platform abstraction (#6080) youkaichao 2024-07-02 20:12:22 -07:00
• 9d6a8daa87 [Model] Jamba support (#4115) 7 people 2024-07-03 02:11:29 +03:00
• ee93f4f92a [CORE] Quantized lm-head Framework (#4442) 3 people 2024-07-03 06:25:17 +08:00
• 7c008c51a9 [ Misc ] Refactor MoE to isolate Fp8 From Mixtral (#5970) 3 people 2024-07-02 17:54:35 -04:00
• 4d26d806e1 Update conftest.py (#6076) Robert Shaw 2024-07-02 16:14:22 -04:00
• c5832d2ae9 [Core] Pipeline Parallel Support (#4412) Murali Andoorveedu 2024-07-02 10:58:08 -07:00
• 15aba081f3 [Speculative Decoding] MLPSpeculator Tensor Parallel support (1/2) (#6050) 3 people 2024-07-02 07:20:29 -07:00
• 31354e563f [Doc] Reinstate doc dependencies (#6061) Cyrus Leung 2024-07-02 18:53:16 +08:00
• 98d6682cd1 [VLM] Remove image_input_type from VLM config (#5852) 3 people 2024-07-02 00:57:09 -07:00
• 2c37540aa6 [Frontend] Add template related params to request (#5709) danieljannai21 2024-07-02 09:01:57 +03:00
• 3476ed0809 [Core] Optimize block_manager_v2 vs block_manager_v1 (to make V2 default) (#5602) Alexander Matveev 2024-07-01 23:10:37 -04:00
• 54600709b6 [Model] Changes to MLPSpeculator to support tie_weights and input_scale (#5965) Thomas ParnellandJoshua Rosenkranz 2024-07-02 01:40:02 +02:00
• e373853e12 [Frontend] Relax api url assertion for openai benchmarking (#6046) James Whedbee 2024-07-01 18:39:10 -05:00
• c87ebc3ef9 [BugFix] Ensure worker model loop is always stopped at the right time (#5987) Nick Hill 2024-07-01 16:17:58 -07:00
• c4059ea54f [Bugfix] Add explicit end_forward calls to flashinfer (#6044) Antoni Baum 2024-07-01 16:08:58 -07:00
• 8e0817c262 [Bugfix][Doc] Fix Doc Formatting (#6048) Roger Wang 2024-07-01 15:09:11 -07:00
• 83bdcb6ac3 add FAQ doc under 'serving' (#5946) ning.zhang 2024-07-01 14:11:36 -07:00
• 12a59959ed [Bugfix] adding chunking mechanism to fused_moe to handle large inputs (#6029) Avshalom Manevich 2024-07-02 00:08:29 +03:00
• dec6fc6f3b [Bugfix] Use RayActorError for older versions of Ray in RayTokenizerGroupPool (#6039) Antoni Baum 2024-07-01 13:12:40 -07:00
• 8893130b63 [doc][misc] further lower visibility of simple api server (#6041) youkaichaoandSimon Mo 2024-07-01 10:50:56 -07:00
• bb60326836 [Misc] update benchmark backend for scalellm (#6018) zhyncs 2024-07-02 01:20:33 +08:00
• 4050d646e5 [doc][misc] remove deprecated api server in doc (#6037) youkaichao 2024-07-01 09:52:43 -07:00
• d76084c12f [ CI ] Re-enable Large Model LM Eval (#6031) Robert Shaw 2024-07-01 12:40:45 -04:00
• 80ca1e6a3a [Speculative Decoding 2/2 ] Integrate typical acceptance sampler into Spec Decode Worker (#5348) sroy745 2024-07-01 00:33:05 -07:00
• 614aa51203 [misc][cuda] use nvml to avoid accidentally cuda initialization (#6007) youkaichao 2024-06-30 20:07:34 -07:00
• af9ad46fca [ Misc ] Refactor w8a8 to use process_weights_after_load (Simplify Weight Loading) (#5940) Robert ShawandRobert Shaw 2024-06-30 19:06:27 -04:00
• 7836fdcc11 [Misc] Fix get_min_capability (#5971) Dipika Sikka 2024-06-30 16:15:16 -04:00
• deacb7ec44 [ CI ] Temporarily Disable Large LM-Eval Tests (#6005) Robert Shawandrshaw@neuralmagic.com <rshaw@neuralmagic> 2024-06-30 14:56:56 -04:00
• f5e73c9f1b [Lora] Use safetensor keys instead of adapter_config.json to find unexpected modules. (#5909) SangBin Choandsang 2024-07-01 02:11:15 +09:00
• c6c240aa0a [Frontend]: Support base64 embedding (#5935) llmprosandCyrus Leung 2024-06-30 08:53:00 -07:00
• 2be6955a3f [ci][distributed] fix device count call youkaichao 2024-06-30 01:06:13 -07:00
• 9d47f64eb6 [CI/Build] [3/3] Reorganize entrypoints tests (#5966) Cyrus Leung 2024-06-30 12:58:49 +08:00
• cff6a1fec1 [CI/Build] Reuse code for checking output consistency (#5988) Cyrus Leung 2024-06-30 11:44:25 +08:00
• bcc6a09b63 [CI/Build] Temporarily Remove Phi3-Vision from TP Test (#5989) Roger Wang 2024-06-29 18:18:31 -07:00
• 9def10664e [Bugfix][CI/Build][Hardware][AMD] Install matching torchvision to fix AMD tests (#5949) Matt Wong 2024-06-29 14:47:58 -05:00
• 75aa1442db [ CI/Build ] LM Eval Harness Based CI Testing (#5838) Robert ShawandRobert Shaw 2024-06-29 13:04:30 -04:00
• 99397da534 [CI/Build] Add TP test for vision models (#5892) Cyrus Leung 2024-06-29 23:45:54 +08:00
• 8dbfcd35bf [ CI/Build ] Added E2E Test For Compressed Tensors (#5839) 3 people 2024-06-29 09:12:58 -04:00
• f7dac83d95 [Kernel] Raise an exception in MoE kernel if the batch size is larger then 65k (#5939) Cody Yu 2024-06-29 06:04:20 -07:00
• 7c01f70641 [Core] Optimize SequenceStatus.is_finished by switching to IntEnum (#5974) Antoni Baum 2024-06-29 05:47:53 -07:00
• 51e971d39e [Bugfix] Support eos_token_id from config.json (#5954) Cyrus Leung 2024-06-29 19:19:02 +08:00
• 329df38f1a [Misc] Update Phi-3-Vision Example (#5981) Roger WangandCyrus Leung 2024-06-28 23:34:29 -07:00
• 580353da93 [Bugfix] Fix precisions in Gemma 1 (#5913) Woosuk Kwon 2024-06-28 20:10:21 -07:00