Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 3ee5c4bca5 [ci] Add A100 queue into AWS CI template (#5648) Kevin H. Luu 2024-06-19 07:42:13 -07:00
• e9c2732b97 [CI/Build] Add tqdm to dependencies (#5680) Cyrus Leung 2024-06-19 22:37:33 +08:00
• d8714530d1 [Misc]Add param max-model-len in benchmark_latency.py (#5629) DearPlanet 2024-06-19 18:19:08 +08:00
• 7d46c8d378 [Bugfix] Fix sampling_params passed incorrectly in Phi3v example (#5684) Isotr0py 2024-06-19 17:58:32 +08:00
• da971ec7a5 [Model] Add FP8 kv cache for Qwen2 (#5656) Michael Goin 2024-06-19 05:38:26 -04:00
• 3eea74889f [misc][distributed] use 127.0.0.1 for single-node (#5619) youkaichao 2024-06-19 01:05:00 -07:00
• f758aed0e8 [Bugfix][CI/Build][AMD][ROCm]Fixed the cmake build bug which generate garbage on certain devices (#5641) Hongxia Yang 2024-06-19 02:21:29 -04:00
• e5150f2c28 [Bugfix] Added test for sampling repetition penalty bug. (#5659) Thomas Parnell 2024-06-19 08:03:55 +02:00
• 59a1eb59c9 [Bugfix] Fix Phi-3 Long RoPE scaling implementation (#5628) Shukant Pal 2024-06-18 18:46:38 -07:00
• 6820724e51 [Bugfix] Fix w8a8 benchmarks for int8 case (#5643) Tyler Michael Smith 2024-06-18 20:33:25 -04:00
• b23ce92032 [Bugfix] Fix CUDA version check for mma warning suppression (#5642) Tyler Michael Smith 2024-06-18 19:48:49 -04:00
• 2bd231a7b7 [Doc] Added cerebrium as Integration option (#5553) milo157 2024-06-18 18:56:59 -04:00
• 8a173382c8 [Bugfix] Fix for inconsistent behaviour related to sampling and repetition penalties (#5639) Thomas Parnell 2024-06-18 23:18:37 +02:00
• 07feecde1a [Model] LoRA support added for command-r (#5178) sergey-tinkoff 2024-06-18 21:01:21 +03:00
• 19091efc44 [ci] Setup Release pipeline and build release wheels with cache (#5610) Kevin H. Luu 2024-06-18 11:00:36 -07:00
• 95db455e7f [Misc] Add channel-wise quantization support for w8a8 dynamic per token activation quantization (#5542) Dipika Sikka 2024-06-18 12:45:05 -04:00
• 7879f24dcc [Misc] Add OpenTelemetry support (#4687) Ronen Schaffer 2024-06-18 19:17:03 +03:00
• 13db4369d9 [ci] Deprecate original CI template (#5624) Kevin H. Luu 2024-06-18 07:26:20 -07:00
• 4ad7b53e59 [CI/Build][Misc] Update Pytest Marker for VLMs (#5623) Roger Wang 2024-06-18 06:10:04 -07:00
• f0cc0e68e3 [Misc] Remove import from transformers logging (#5625) Chang Su 2024-06-18 05:12:19 -07:00
• db5ec52ad7 [bugfix][distributed] improve p2p capability test (#5612) youkaichao 2024-06-18 00:21:05 -07:00
• 114d7270ff [CI] Avoid naming different metrics with the same name in performance benchmark (#5615) Kuntai Du 2024-06-17 21:37:18 -07:00
• 32c86e494a [Misc] Fix typo (#5618) Cyrus Leung 2024-06-18 11:58:30 +08:00
• 8eadcf0b90 [misc][typo] fix typo (#5620) youkaichao 2024-06-17 20:54:57 -07:00
• 5002175e80 [Kernel] Add punica dimensions for Granite 13b (#5559) Joe Runde 2024-06-17 21:54:11 -06:00
• daef218b55 [Model] Initialize Phi-3-vision support (#4986) Isotr0py 2024-06-18 10:34:33 +08:00
• fa9e385229 [Speculative Decoding 1/2 ] Add typical acceptance sampling as one of the sampling techniques in the verifier (#5131) sroy745 2024-06-17 19:29:09 -07:00
• 26e1188e51 [Fix] Use utf-8 encoding in entrypoints/openai/run_batch.py (#5606) zifeitong 2024-06-17 16:16:10 -07:00
• a3e8a05d4c [Bugfix] Fix KV head calculation for MPT models when using GQA (#5142) Bruce Fontaine 2024-06-17 15:26:41 -07:00
• e441bad674 [Optimization] use a pool to reuse LogicalTokenBlock.token_ids (#5584) youkaichao 2024-06-17 15:08:05 -07:00
• 1b44aaf4e3 [bugfix][distributed] fix 16 gpus local rank arrangement (#5604) youkaichao 2024-06-17 14:35:04 -07:00
• 9e4e6fe207 [CI] the readability of benchmarking and prepare for dashboard (#5571) Kuntai Du 2024-06-17 11:41:08 -07:00
• ab66536dbf [CI/BUILD] Support non-AVX512 vLLM building and testing (#5574) Jie Fu (傅杰) 2024-06-18 02:36:10 +08:00
• 728c4c8a06 [Hardware][Intel GPU] Add Intel GPU(XPU) inference backend (#3814) 4 people 2024-06-18 02:01:25 +08:00
• 1f12122b17 [Misc] use AutoTokenizer for benchmark serving when vLLM not installed (#5588) zhyncs 2024-06-18 00:40:35 +08:00
• 890d8d960b [Kernel] compressed-tensors marlin 24 support (#5435) Dipika Sikka 2024-06-17 12:32:48 -04:00
• 9e74d9d003 Correct alignment in the seq_len diagram. (#5592) Charles RigginsandLiqian Chen 2024-06-18 00:05:33 +08:00
• 9333fb8eb9 [Model] Rename Phi3 rope scaling type (#5595) Amit Garg 2024-06-17 09:04:14 -07:00
• e2b85cf86a Fix w8a8 benchmark and add Llama-3-8B (#5562) Cody Yu 2024-06-16 23:48:06 -07:00
• 845a3f26f9 [Doc] add debugging tips for crash and multi-node debugging (#5581) youkaichao 2024-06-16 19:08:01 -07:00
• f07d513320 [build][misc] limit numpy version (#5582) youkaichao 2024-06-16 16:07:01 -07:00
• 4a6769053a [CI][BugFix] Flip is_quant_method_supported condition (#5577) Michael Goin 2024-06-16 10:07:34 -04:00
• f31c1f90e3 Add basic correctness 2 GPU tests to 4 GPU pipeline (#5518) Antoni Baum 2024-06-16 00:48:02 -07:00
• 3ce2c050dd [Fix] Correct OpenAI batch response format (#5554) zifeitong 2024-06-15 16:57:54 -07:00
• 1c0afa13c5 [BugFix] Don't start a Ray cluster when not using Ray (#5570) Nick Hill 2024-06-15 16:30:51 -07:00
• d919ecc771 add gptq_marlin test for bug report https://github.com/vllm-project/vllm/issues/5088 (#5145) Alexander Matveev 2024-06-15 13:38:16 -04:00
• e691918e3b [misc] Do not allow to use lora with chunked prefill. (#5538) SangBin ChoandCyrus Leung 2024-06-15 23:59:36 +09:00
• 81fbb3655f [CI/Build] Test both text and token IDs in batched OpenAI Completions API (#5568) Cyrus Leung 2024-06-15 19:29:42 +08:00
• 0e9164b40a [mypy] Enable type checking for test directory (#5017) Cyrus Leung 2024-06-15 12:45:31 +08:00
• 1b8a0d71cf [Core][Bugfix]: fix prefix caching for blockv2 (#5364) leiwen83andLei Wen 2024-06-15 08:23:56 +08:00
• bd7efe95d0 Add ccache to amd (#5555) Simon Mo 2024-06-14 19:18:22 -05:00
• f5bb85b435 [Core][Distributed] improve p2p cache generation (#5528) youkaichao 2024-06-14 14:47:45 -07:00
• 28c145eb57 [Bugfix] Fix typo in Pallas backend (#5558) Woosuk Kwon 2024-06-14 14:40:09 -07:00
• e2afb03c92 [Bugfix] Enable loading FP8 checkpoints for gpt_bigcode models (#5460) Thomas Parnell 2024-06-14 22:28:11 +02:00
• 6e2527a7cb [Doc] Update documentation on Tensorizer (#5471) Sanger Steel 2024-06-14 14:27:57 -04:00
• cdab68dcdb [Docs] Add ZhenFund as a Sponsor (#5548) Simon Mo 2024-06-14 13:17:21 -05:00
• d1c3d7d139 [misc][distributed] fix benign error in is_in_the_same_node (#5512) youkaichao 2024-06-14 10:59:28 -07:00
• 77490c6f2f [Core] Remove duplicate processing in async engine (#5525) Cyrus Leung 2024-06-15 01:04:42 +08:00
• 48f589e18b [mis] fix flaky test of test_cuda_device_count_stateless (#5546) youkaichao 2024-06-14 10:02:23 -07:00
• 348616ac4b [Kernel] Suppress mma.sp warning on CUDA 12.5 and later (#5401) Tyler Michael Smith 2024-06-14 13:02:00 -04:00
• 15985680e2 [ Misc ] Rs/compressed tensors cleanup (#5432) 3 people 2024-06-14 13:01:46 -04:00
• d74674bbd9 [Misc] Fix arg names (#5524) Allen.Dou 2024-06-15 00:47:44 +08:00
• 703475f6c2 [Kernel] Fix CUTLASS 3.x custom broadcast load epilogue (#5516) Tyler Michael Smith 2024-06-14 12:30:15 -04:00
• d47af2bc02 [CI/Build] Disable LLaVA-NeXT CPU test (#5529) Cyrus Leung 2024-06-15 00:27:30 +08:00
• 319ad7f1d3 [CI/Build][Misc] Add CI that benchmarks vllm performance on those PRs with perf-benchmarks label (#5073) Kuntai Duandsimon-mo 2024-06-13 22:36:20 -07:00
• 0f0d8bc065 bump version to v0.5.0.post1 (#5522) Simon Mo 2024-06-13 21:42:06 -05:00
• 55d6361b13 [Misc] Fix arg names in quantizer script (#5507) Allen.Dou 2024-06-14 10:02:53 +08:00
• cd9c0d65d9 [Hardware][Intel] Support CPU inference with AVX2 ISA (#5452) Jie Fu (傅杰) 2024-06-14 07:22:24 +08:00
• 50eed24d25 Add cuda_device_count_stateless (#5473) Antoni Baum 2024-06-13 16:06:49 -07:00
• e38042d4af [Kernel] Disable CUTLASS kernels for fp8 (#5505) Tyler Michael Smith 2024-06-13 16:38:05 -04:00
• 33e3b37242 [CI/Build] Disable test_fp8.py (#5508) Tyler Michael Smith 2024-06-13 16:37:48 -04:00
• 1696efe6c9 [misc] fix format.sh (#5511) youkaichao 2024-06-13 12:09:16 -07:00
• 6b0511a57b Revert "[Core] Remove unnecessary copies in flash attn backend" (#5478) Antoni Baum 2024-06-13 11:22:50 -07:00
• a8fda4f661 Seperate dev requirements into lint and test (#5474) Antoni Baum 2024-06-13 11:22:41 -07:00
• 30299a41fa [MISC] Remove FP8 warning (#5472) Cody YuandPhilipp Moritz 2024-06-13 11:22:30 -07:00
• 85657b5607 [Kernel] Factor out epilogues from cutlass kernels (#5391) 5 people 2024-06-13 14:22:19 -04:00
• 0ce7b952f8 [Doc] Update LLaVA docs (#5437) Cyrus LeungandRoger Wang 2024-06-14 02:22:07 +08:00
• 39873476f8 [CI/Build] Simplify OpenAI server setup in tests (#5100) Cyrus Leung 2024-06-14 02:21:53 +08:00
• 03dccc886e [Misc] Add vLLM version getter to utils (#5098) Cyrus Leung 2024-06-14 02:21:39 +08:00
• a65634d3ae [Docs] Add 4th meetup slides (#5509) Woosuk Kwon 2024-06-13 10:18:26 -07:00
• 80aa7e91fc [Hardware][Intel] Optimize CPU backend and add more performance tips (#4971) Li, JiangandJianan Gu 2024-06-14 00:33:14 +08:00
• bd43973522 [Kernel] Tune Qwen2MoE kernel configurations with tp2,4 (#5497) wenyujin333 2024-06-14 00:01:10 +08:00
• 23ec72fa03 [CI/Build][REDO] Add is_quant_method_supported to control quantization test configurations (#5466) Michael Goin 2024-06-13 11:18:08 -04:00
• c2637a613b [Kernel] w4a16 support for compressed-tensors (#5385) Dipika SikkaandRobert Shaw 2024-06-13 10:19:56 -04:00
• 88407532e7 [Bugfix]if the content is started with ":"(response of ping), client should i… (#5303) Wang, YiandRoger Wang 2024-06-13 11:16:41 +08:00
• 916d219d62 [ci] Use sccache to build images (#5419) Kevin H. Luu 2024-06-12 17:58:12 -07:00
• ea3890a5f0 [Core][Distributed] code deduplication in tp&pp with coordinator(#5293) youkaichao 2024-06-12 17:27:08 -07:00
• 2135cacb45 [Bugfix] Fix wrong multi_modal_input format for CPU runner (#5451) Isotr0py 2024-06-13 07:20:18 +08:00
• 7d19de2e9c [Frontend] Add "input speed" to tqdm postfix alongside output speed (#5425) Michael Goin 2024-06-12 18:42:12 -04:00
• 94a07bbdd8 [Bugfix] Fix typo in scheduler.py (requeset -> request) (#5470) Michael Goin 2024-06-12 17:59:44 -04:00
• b8d4dfff9c [Doc] Update debug docs (#5438) Cyrus Leung 2024-06-13 05:49:31 +08:00
• 622d45128c [misc] add hint for AttributeError (#5462) youkaichao 2024-06-12 14:46:35 -07:00
• 51602eefd3 [Frontend] [Core] Support for sharded tensorized models (#4990) 3 people 2024-06-12 15:13:52 -06:00
• 5cc50a531f [Bugfix] TYPE_CHECKING for MultiModalData (#5444) Arthur Kim 2024-06-13 06:08:52 +09:00
• 5985e3427d [Kernel] Vectorized FP8 quantize kernel (#5396) Cody Yu 2024-06-12 14:07:26 -07:00
• 8b82a89997 [ci] Add AMD, Neuron, Intel tests for AWS CI and turn off default soft fail for GPU tests (#5464) Kevin H. Luu 2024-06-12 14:00:18 -07:00
• c3c2903e72 [Bugfix] Add device assertion to TorchSDPA (#5402) Li, Jiang 2024-06-13 03:58:53 +08:00
• 1a8bfd92d5 [Hardware] Initial TPU integration (#5292) Woosuk Kwon 2024-06-12 11:53:03 -07:00
• 847cdcca1c [CI] Upgrade codespell version. (#5381) SangBin Cho 2024-06-13 02:06:14 +09:00
• e3c12bf6d2 Revert "[CI/Build] Add is_quant_method_supported to control quantization test configurations" (#5463) Simon Mo 2024-06-12 12:03:24 -05:00