Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 2dd34371a6 [Bugfix] Fix RMSNorm forward in InternViT attention qk_layernorm (#6992) Isotr0py 2024-08-02 03:00:28 +08:00
• 7e0861bd0b [CI/Build] Update PyTorch to 2.4.0 (#6951) Sage MooreandMichael Goin 2024-08-01 11:11:24 -07:00
• a72a424b3e [Build/CI] Fixing Docker Hub quota issue. (#7043) Alexei-V-Ivanov-AMD 2024-08-01 13:07:37 -05:00
• c8a7e93273 [core][scheduler] simplify and improve scheduler (#6867) youkaichao 2024-07-31 23:51:09 -07:00
• 3c10591ef2 [Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user (#6954) zifeitong 2024-07-31 21:13:34 -07:00
• 0437492ea9 PP comm optimization: replace send with partial send + allgather (#6695) Aurick QiaoandAurick Qiao 2024-07-31 20:15:42 -07:00
• 630dd9e0ae [Bugfix][Model] Skip loading lm_head weights if using tie_word_embeddings (#6758) Travis Johnson 2024-07-31 20:49:11 -06:00
• 23993a7997 [Bugfix][TPU] Do not use torch.Generator for TPUs (#6981) Woosuk Kwon 2024-07-31 18:50:28 -07:00
• 1d2e7fb73f [Model] Pipeline parallel support for Qwen2 (#6924) xuyi 2024-08-01 09:49:51 +08:00
• 7ecee34321 [Kernel][RFC] Refactor the punica kernel based on Triton (#5036) Jee Jee Li 2024-08-01 08:12:24 +08:00
• 7eb0cb4a14 Revert "[Frontend] Factor out code for running uvicorn" (#7012) Simon MoandRobert Shaw 2024-07-31 16:34:26 -07:00
• a0dce9383a [Misc] Add compressed-tensors to optimized quant list (#7006) Michael Goin 2024-07-31 17:40:44 -04:00
• 35e9c12bfa [Kernel] Tuned int8 Cutlass Kernels for SM75 (T4) (#6996) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-07-31 17:40:32 -04:00
• 93548eb37e [Kernel] Enable FP8 Cutlass for Ada Lovelace (#6950) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-07-31 17:40:22 -04:00
• 460c1884e3 [Bugfix] Support cpu offloading with fp8 quantization (#6960) Michael Goin 2024-07-31 15:47:46 -04:00
• bd70013407 [MISC] Introduce pipeline parallelism partition strategies (#6920) Cody Yuandyoukaichao 2024-07-31 12:02:17 -07:00
• 2ee8d3ba55 [Model] use FusedMoE layer in Jamba (#6935) Avshalom Manevich 2024-07-31 22:00:24 +03:00
• daed30c4a9 [Bugfix] Fix feature size calculation for LLaVA-NeXT (#6982) Cyrus Leung 2024-07-31 23:46:17 +08:00
• 2f4e108f75 [Bugfix] Clean up MiniCPM-V (#6939) 3 people 2024-07-31 22:39:19 +08:00
• 6512937de1 Support W4A8 quantization for vllm (#5218) HandH1998 2024-07-31 21:55:21 +08:00
• c0644cf9ce [Bugfix] fix logit processor excceed vocab size issue (#6927) Fei 2024-07-31 01:16:01 -07:00
• 533d1932d2 [Bugfix][TPU] Set readonly=True for non-root devices (#6980) Woosuk Kwon 2024-07-31 00:19:28 -07:00
• 9f0e69b653 [CI/Build] Fix mypy errors (#6968) Cyrus Leung 2024-07-31 10:49:48 +08:00
• f230cc2ca6 [Bugfix] Fix broadcasting logic for multi_modal_kwargs (#6836) Cyrus Leung 2024-07-31 10:38:45 +08:00
• da1f7cc12a [mypy] Enable following imports for some directories (#6681) Cyrus Leung 2024-07-31 10:38:03 +08:00
• c32ab8be1a [Speculative decoding] Add serving benchmark for llama3 70b + speculative decoding (#6964) Cade Daniel 2024-07-30 17:53:21 -07:00
• fb4f530bf5 [CI] [nightly benchmark] Do not re-download sharegpt dataset if exists (#6706) Cade Daniel 2024-07-30 16:28:49 -07:00
• 79319cedfa [Nightly benchmarking suite] Remove pkill python from run benchmark suite (#6965) Cade Daniel 2024-07-30 16:28:05 -07:00
• 40c27a7cbb [Build] Temporarily Disable Kernels and LoRA tests (#6961) Simon Mo 2024-07-30 14:59:48 -07:00
• 6ca8031e71 [core][misc] improve free_finished_seq_groups (#6865) youkaichaoandWoosuk Kwon 2024-07-30 14:32:12 -07:00
• d7a299edaa [Kernel] Remove scaled_fp8_quant kernel padding footgun (#6842) Tyler Michael Smith 2024-07-30 16:37:01 -04:00
• 052b6f8ca4 [Bugfix] Fix tensorizer memory profiling bug during testing (#6881) Sanger Steel 2024-07-30 14:48:50 -04:00
• 5895b24677 [OpenVINO] Updated OpenVINO requirements and build docs (#6948) Ilya Lavrenov 2024-07-30 22:33:01 +04:00
• cbbc904470 [Kernel] Squash a few more warnings (#6914) Tyler Michael Smith 2024-07-30 13:50:42 -04:00
• 5cf9254a9c [BugFix] Fix use of per-request seed with pipeline parallel (#6698) Nick Hill 2024-07-30 10:40:08 -07:00
• f058403683 [Doc] Super tiny fix doc typo (#6949) fzyzcjy 2024-07-31 00:14:03 +08:00
• c66c7f86ac [Bugfix] Fix PaliGemma MMP (#6930) Roger Wang 2024-07-30 02:20:57 -07:00
• 6e063ea35b [TPU] Fix greedy decoding (#6933) Woosuk Kwon 2024-07-30 02:06:29 -07:00
• af647fb8b3 [Kernel] Tuned int8 kernels for Ada Lovelace (#6848) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-07-29 22:24:58 -04:00
• 61a97c32f6 [Kernel] Fix marlin divide-by-zero warnings (#6904) Tyler Michael Smith 2024-07-29 21:26:07 -04:00
• 4fbf4aa128 [ci] GHA workflow to remove ready label upon "/notready" comment (#6921) Kevin H. Luu 2024-07-29 17:03:45 -07:00
• aae6d36f7e [Kernel] Remove unused variables in awq/gemm_kernels.cu (#6908) Tyler Michael Smith 2024-07-29 20:01:17 -04:00
• 9f69d8245a [Frontend] New allowed_token_ids decoding request parameter (#6753) Nick Hill 2024-07-29 16:37:27 -07:00
• 9a7e2d0534 [Bugfix] Allow vllm to still work if triton is not installed. (#6786) Thomas Parnell 2024-07-29 23:51:27 +02:00
• 7f8d612d24 [TPU] Support tensor parallelism in async llm engine (#6891) Earthwalker 2024-07-30 03:42:21 +08:00
• 60d1c6e584 [Kernel] Fix deprecation function warnings squeezellm quant_cuda_kernel (#6901) Tyler Michael Smith 2024-07-29 12:59:02 -04:00
• db9e5708a9 [Core] Reduce unnecessary compute when logprobs=None (#6532) Peng Guanwen 2024-07-30 00:47:31 +08:00
• 766435e660 [Kernel] Tuned FP8 Kernels for Ada Lovelace (#6677) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-07-29 11:42:35 -04:00
• 7cbd9ec7a9 [Model] Initialize support for InternVL2 series models (#6514) Isotr0pyandRoger Wang 2024-07-29 18:16:30 +08:00
• 3eeb148f46 [Misc] Pass cutlass_fp8_supported correctly in fbgemm_fp8 (#6871) Elsa Granger 2024-07-28 23:13:49 +08:00
• b1366a9534 Add Nemotron to PP_SUPPORTED_MODELS (#6863) Michael Goin 2024-07-27 18:05:17 -04:00
• 75acdaa4b6 [Kernel] Increase precision of GPTQ/AWQ Marlin kernel (#6795) Alexander Matveev 2024-07-27 17:52:33 -04:00
• fad5576c58 [TPU] Reduce compilation time & Upgrade PyTorch XLA version (#6856) Woosuk Kwon 2024-07-27 10:28:33 -07:00
• f954d0715c [Docs] Add RunLLM chat widget (#6857) Chenggang Wu 2024-07-27 09:24:46 -07:00
• 1ad86acf17 [Model] Initial support for BLIP-2 (#5920) Cyrus Leungandywang96 2024-07-27 19:53:07 +08:00
• ecb33a28cb [CI/Build][Doc] Update CI and Doc for VLM example changes (#6860) Roger Wang 2024-07-27 02:54:14 -07:00
• a57d75821c [bugfix] make args.stream work (#6831) Wang Ran (汪然) 2024-07-27 17:07:02 +08:00
• 925de97e05 [Bugfix] Fix VLM example typo (#6859) Roger Wang 2024-07-26 23:24:08 -07:00
• aa46953a20 [Misc][VLM][Doc] Consolidate offline examples for vision language models (#6858) Roger WangandCyrus Leung 2024-07-26 22:44:13 -07:00
• 593e79e733 [Bugfix] torch.set_num_threads() in multiproc_gpu_executor (#6802) Travis Johnson 2024-07-26 23:15:20 -06:00
• c53041ae3b [Doc] Add missing mock import to docs conf.py (#6834) Harry Mellor 2024-07-27 05:47:33 +01:00
• 52f07e3dec [Hardware][TPU] Implement tensor parallelism with Ray (#5871) Woosuk Kwon 2024-07-26 20:54:27 -07:00
• 14dbd5a767 [Model] H2O Danube3-4b (#6451) Joe 2024-07-26 20:47:50 -07:00
• ed94e4f427 [Bugfix][Model] Jamba assertions and no chunked prefill by default for Jamba (#6784) tomeras91 2024-07-27 06:45:31 +03:00
• 3c3012398e [Doc] add VLLM_TARGET_DEVICE=neuron to documentation for neuron (#6844) omrishiv 2024-07-26 20:20:16 -07:00
• ced36cd89b [ROCm] Upgrade PyTorch nightly version (#6845) Woosuk Kwon 2024-07-26 20:16:13 -07:00
• 969d032265 [Bugfix]: Fix Tensorizer test failures (#6835) Sanger Steel 2024-07-26 23:02:25 -04:00
• 55712941e5 [Bug Fix] Illegal memory access, FP8 Llama 3.1 405b (#6852) Lucas Wilkinson 2024-07-26 22:27:44 -04:00
• 981b0d5673 [Frontend] Factor out code for running uvicorn (#6828) Cyrus Leung 2024-07-27 09:58:25 +08:00
• d09b94ca58 [TPU] Support collective communications in XLA devices (#6813) Woosuk Kwon 2024-07-26 18:45:57 -07:00
• bb5494676f enforce eager mode with bnb quantization temporarily (#6846) chenqianfzh 2024-07-26 18:32:20 -07:00
• b5f49ee55b Update README.md (#6847) Gurpreet Singh Dhami 2024-07-26 20:26:45 -04:00
• 150a1ffbfd [Doc] Update SkyPilot doc for wrong indents and instructions for update service (#4283) Zhanghao Wu 2024-07-26 17:39:10 -04:00
• 281977bd6e [Doc] Add Nemotron to supported model docs (#6843) Michael Goin 2024-07-26 17:32:44 -04:00
• 3bbb4936dc [Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend and update documentation (#6125) Li, Jiang 2024-07-27 04:50:10 +08:00
• aa4867791e [Misc][TPU] Support TPU in initialize_ray_cluster (#6812) Woosuk Kwon 2024-07-26 12:39:49 -07:00
• 71734f1bf2 [Build/CI][ROCm] Minor simplification to Dockerfile.rocm (#6811) Woosuk Kwon 2024-07-26 12:28:32 -07:00
• 50704f52c4 [Bugfix][Kernel] Promote another index to int64_t (#6838) Tyler Michael Smith 2024-07-26 14:41:04 -04:00
• 07278c37dd [Model] Support Nemotron models (Nemotron-3, Nemotron-4, Minitron) (#6611) Michael Goin 2024-07-26 14:33:42 -04:00
• 85ad7e2d01 [doc][debugging] add known issues for hangs (#6816) youkaichao 2024-07-25 21:48:05 -07:00
• 89a84b0bb7 [Core] Use array to speedup padding (#6779) Peng Guanwen 2024-07-26 12:31:31 +08:00
• 084a01fd35 [Bugfix] [Easy] Fixed a bug in the multiprocessing GPU executor. (#6770) Anthony Platanios 2024-07-26 00:25:35 -04:00
• 062a1d0fab Fix ReplicatedLinear weight loading (#6793) QQSong 2024-07-25 19:24:58 -07:00
• 2eb9f4ff26 [ci] Mark tensorizer as soft fail and separate from grouped test (#6810) Kevin H. Luu 2024-07-25 18:08:33 -07:00
• 443c7cf4cf [ci][distributed] fix flaky tests (#6806) youkaichao 2024-07-25 17:44:09 -07:00
• 1adddb14bf [Core] Fix ray forward_dag error mssg (#6792) SangBin Cho 2024-07-25 16:53:25 -07:00
• b7215de2c5 [Docs] Publish 5th meetup slides (#6799) Woosuk Kwon 2024-07-25 16:47:55 -07:00
• f3ff63c3f4 [doc][distributed] improve multinode serving doc (#6804) youkaichao 2024-07-25 15:38:32 -07:00
• cd7edc4e87 [Bugfix] Fix empty (nullptr) channelwise scales when loading wNa16 using compressed tensors (#6798) Lucas Wilkinson 2024-07-25 18:05:09 -04:00
• 6a1e25b151 [Doc] Add documentations for nightly benchmarks (#6412) Kuntai Du 2024-07-25 11:57:16 -07:00
• 95db75de64 [Bugfix] Add synchronize to prevent possible data race (#6788) Tyler Michael SmithandLucas Wilkinson 2024-07-25 13:40:01 -04:00
• 65b1f121c8 [Bugfix] Fix kv_cache_dtype=fp8 without scales for FP8 checkpoints (#6761) Michael Goin 2024-07-25 12:46:15 -04:00
• 889da130e7 [ Misc ] fp8-marlin channelwise via compressed-tensors (#6524) Robert Shawandmgoin 2024-07-25 09:46:04 -07:00
• b75e314fff [Bugfix] Add image placeholder for OpenAI Compatible Server of MiniCPM-V (#6787) 3 people 2024-07-26 00:42:49 +08:00
• 316a41ac1d [Bugfix] Fix encoding_format in examples/openai_embedding_client.py (#6755) Chang Su 2024-07-24 22:48:07 -07:00
• 0310029a2f [Bugfix] Fix awq_marlin and gptq_marlin flags (#6745) Alexander Matveev 2024-07-25 01:34:11 -04:00
• 309aaef825 [Bugfix] Fix decode tokens w. CUDA graph (#6757) Cody Yu 2024-07-24 22:33:56 -07:00
• 9e169a4c61 [Model] Adding support for MiniCPM-V (#4087) Alphi 2024-07-25 11:59:30 +08:00
• 5689e256ba [Frontend] Represent tokens with identifiable strings (#6626) Evan Z. Liu 2024-07-24 18:51:00 -07:00
• 740374d456 [core][distributed] fix zmq hang (#6759) youkaichao 2024-07-24 17:37:12 -07:00