Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 32e7db2536 Bump version to v0.6.0 (#8166) Simon Mo 2024-09-04 16:34:27 -07:00
• 008cf886c9 [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) Harsha vardhan manoj BikkiandHarsha Bikki 2024-09-04 16:33:43 -07:00
• 77d9e514a2 [MISC] Replace input token throughput with total token throughput (#8164) Cody YuandMichael Goin 2024-09-04 13:23:22 -07:00
• e02ce498be [Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models (#5649) 3 people 2024-09-04 15:18:13 -05:00
• 561d6f8077 [CI] Change test input in Gemma LoRA test (#8163) Woosuk Kwon 2024-09-04 13:05:50 -07:00
• d1dec64243 [CI/Build][ROCm] Enabling LoRA tests on ROCm (#7369) alexeykondratandSimon Mo 2024-09-04 14:57:54 -04:00
• 2ad2e5608e [MISC] Consolidate FP8 kv-cache tests (#8131) Cody Yu 2024-09-04 11:53:25 -07:00
• d3311562fb [Bugfix] remove post_layernorm in siglip (#8106) wnma 2024-09-04 18:55:37 +08:00
• ccd7207191 chore: Update check-wheel-size.py to read MAX_SIZE_MB from env (#8103) TimWang 2024-09-04 14:17:05 +08:00
• 855c262a6b [Frontend] Multimodal support in offline chat (#8098) Cyrus Leung 2024-09-04 13:22:17 +08:00
• 2be8ec6e71 [Model] Add Ultravox support for multiple audio chunks (#7963) Peter Salas 2024-09-03 21:38:21 -07:00
• e16fa99a6a [Misc] Update fbgemmfp8 to use vLLMParameters (#7972) Dipika SikkaandMichael Goin 2024-09-03 22:12:41 -04:00
• 61f4a93d14 [TPU][Bugfix] Use XLA rank for persistent cache path (#8137) Woosuk Kwon 2024-09-03 18:35:33 -07:00
• d4db9f53c8 [Benchmark] Add --async-engine option to benchmark_throughput.py (#7964) Nick Hill 2024-09-03 17:57:41 -07:00
• 2188a60c7e [Misc] Update GPTQ to use vLLMParameters (#7976) Dipika Sikka 2024-09-03 17:21:44 -04:00
• dc0b6066ab [CI] Change PR remainder to avoid at-mentions (#8134) Simon Mo 2024-09-03 14:11:42 -07:00
• 0af3abe3d3 [TPU][Bugfix] Fix next_token_ids shape (#8128) Woosuk Kwon 2024-09-03 13:29:24 -07:00
• f1575dc99f [ci] Fix GHA workflow (#8129) Kevin H. Luu 2024-09-03 13:25:09 -07:00
• c02638efb3 [CI/Build] make pip install vllm work in macos (for import only) (#8118) tomeras91 2024-09-03 22:37:08 +03:00
• 652c83b697 [Misc] Raise a more informative exception in add/remove_logger (#7750) Antoni Baum 2024-09-03 12:28:25 -07:00
• 6d646d08a2 [Core] Optimize Async + Multi-step (#8050) Alexander Matveev 2024-09-03 14:50:29 -04:00
• 95a178f861 [CI] Only PR reviewers/committers can trigger CI on PR (#8124) Kevin H. Luu 2024-09-03 11:32:27 -07:00
• bd852f2a8b [Performance] Enable chunked prefill and prefix caching together (#8120) 3 people 2024-09-03 10:49:18 -07:00
• ec266536b7 [Bugfix][VLM] Add fallback to SDPA for ViT model running on CPU backend (#8061) Isotr0py 2024-09-03 21:37:52 +08:00
• 0fbc6696c2 [Bugfix] Fix single output condition in output processor (#7881) Woosuk Kwon 2024-09-02 20:35:42 -07:00
• 6e36f4fa6c improve chunked prefill performance wang.yuqi 2024-09-03 05:20:12 +08:00
• dd2a6a82e3 [Bugfix] Fix internlm2 tensor parallel inference (#8055) Isotr0py 2024-09-02 23:48:56 +08:00
• 4ca65a9763 [Core][Bugfix] Accept GGUF model without .gguf extension (#8056) Isotr0py 2024-09-02 20:43:26 +08:00
• e2b2aa5a0f [TPU] Align worker index with node boundary (#7932) Woosuk Kwon 2024-09-01 23:09:46 -07:00
• e6a26ed037 [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) Lily Liu 2024-09-01 21:23:29 -07:00
• f8d60145b4 [Model] Add Granite model (#7436) Shawn TanandNick Hill 2024-09-01 21:37:18 -04:00
• 5b86b19954 [Misc] Optional installation of audio related packages (#8063) Roger Wang 2024-09-01 14:46:57 -07:00
• 5231f0898e [Frontend][VLM] Add support for multiple multi-modal items (#8049) Roger Wang 2024-08-31 16:35:53 -07:00
• 8423aef4c8 [BugFix][Core] Multistep Fix Crash on Request Cancellation (#8059) Robert Shaw 2024-08-31 15:44:03 -04:00
• 4f5d8446ed [Bugfix] Fix ModelScope models in v0.5.5 (#8037) Nicolò Lucchesi 2024-08-31 09:27:58 +02:00
• d05f0a9db2 [Bugfix] Fix import error in Phi-3.5-MoE (#8052) Cyrus Leung 2024-08-31 13:26:55 +08:00
• 622f8abff8 [Bugfix] bugfix and add model test for flashinfer fp8 kv cache. (#8013) Pavani Majety 2024-08-30 22:18:50 -07:00
• 1248e8506a [Model] Adding support for MSFT Phi-3.5-MoE (#7729) 4 people 2024-08-31 03:42:57 +08:00
• 2684efc467 [TPU][Bugfix] Fix tpu type api (#8035) Woosuk Kwon 2024-08-30 09:01:26 -07:00
• 058344f89a [Frontend]-config-cli-args (#7737) 3 people 2024-08-30 08:21:02 -07:00
• 98cef6a227 [Core] Increase default max_num_batched_tokens for multimodal models (#8028) Cyrus Leung 2024-08-30 23:20:34 +08:00
• f97be32d1d [VLM][Model] TP support for ViTs (#7186) 3 people 2024-08-31 00:19:27 +09:00
• afd39a4511 [Bugfix] Fix import error in Exaone model (#8034) Cyrus Leung 2024-08-30 23:03:28 +08:00
• 2148441fd3 [TPU] Support single and multi-host TPUs on GKE (#7613) Richard Liu 2024-08-30 00:27:40 -07:00
• dc13e99348 [MODEL] add Exaone model support (#7819) Yohan Na 2024-08-30 15:34:20 +09:00
• 34a0e96d46 [Kernel] changing fused moe kernel chunk size default to 32k (#7995) Avshalom Manevich 2024-08-30 11:11:39 +07:00
• 80c7b089b1 [TPU] Async output processing for TPU (#8011) Woosuk Kwon 2024-08-29 19:35:29 -07:00
• 428dd1445e [Core] Logprobs support in Multi-step (#7652) afeldman-nm 2024-08-29 22:19:08 -04:00
• 4abed65c58 [VLM] Disallow overflowing max_model_len for multimodal models (#7998) Cyrus Leung 2024-08-30 08:49:04 +08:00
• 0c785d344d Add more percentiles and latencies (#7759) Wei-Sheng Chin 2024-08-29 16:48:11 -07:00
• 4664ceaad6 support bitsandbytes 8-bit and FP4 quantized models (#7445) chenqianfzh 2024-08-29 16:09:08 -07:00
• 257afc37c5 [Neuron] Adding support for context-lenght, token-gen buckets. (#7885) Harsha vardhan manoj BikkiandHarsha Bikki 2024-08-29 13:58:14 -07:00
• 86a677de42 [misc] update tpu int8 to use new vLLM Parameters (#7973) Dipika Sikka 2024-08-29 16:46:55 -04:00
• d78789ac16 [Bugfix] Fix incorrect vocal embedding shards for GGUF model in tensor parallelism (#7954) Isotr0py 2024-08-30 03:54:49 +08:00
• c334b1898b extend cuda graph size for H200 (#7894) kushanamandyoukaichao 2024-08-29 12:15:04 -07:00
• 6b3421567d [Core][Kernels] Enable FP8 KV Cache with Flashinfer backend. + BugFix for kv_cache_dtype=auto (#7985) 3 people 2024-08-29 11:53:11 -07:00
• 3f60f2244e [Core] Combine async postprocessor and multi-step (#7921) Alexander Matveev 2024-08-29 14:18:26 -04:00
• f205c09854 [Bugfix] Unify rank computation across regular decoding and speculative decoding (#7899) Jonas M. Kübler 2024-08-29 07:18:13 +02:00
• ef99a78760 Revert "[Core][Kernels] Use FlashInfer backend for FP8 KV Cache when available." (#7982) youkaichao 2024-08-28 21:27:06 -07:00
• 74d5543ec5 [VLM][Core] Fix exceptions on ragged NestedTensors (#7974) Peter Salas 2024-08-28 20:24:31 -07:00
• a7f65c2be9 [torch.compile] remove reset (#7975) youkaichao 2024-08-28 17:32:26 -07:00
• 4289cad37f [Frontend] Minor optimizations to zmq decoupled front-end (#7957) Nick HillandRobert Shaw 2024-08-28 17:22:43 -07:00
• af59df0a10 Remove faulty Meta-Llama-3-8B-Instruct-FP8.yaml lm-eval test (#7961) Michael Goin 2024-08-28 19:19:17 -04:00
• ce6bf3a2cf [torch.compile] avoid Dynamo guard evaluation overhead (#7898) youkaichaoandWoosuk Kwon 2024-08-28 16:10:12 -07:00
• 3cdfe1f38b [Bugfix] Make torch registration of punica ops optional (#7970) bnellnm 2024-08-28 18:11:49 -04:00
• fdd9daafa3 [Kernel/Model] Migrate mamba_ssm and causal_conv1d kernels to vLLM (#7651) Mor Zusman 2024-08-29 01:06:52 +03:00
• 8c56e57def [Doc] fix 404 link (#7966) Stas Bekman 2024-08-28 13:54:23 -07:00
• eeffde1ac0 [TPU] Upgrade PyTorch XLA nightly (#7967) Woosuk Kwon 2024-08-28 13:10:21 -07:00
• e5697d161c [Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and awq_gemm to support AWQ (#7386) rasmith 2024-08-28 14:37:47 -05:00
• b98cc28f91 [Core][Kernels] Use FlashInfer backend for FP8 KV Cache when available. (#7798) Pavani MajetyandSimon Mo 2024-08-28 10:01:22 -07:00
• ef9baee3c5 [Bugfix][VLM] Fix incompatibility between #7902 and #7230 (#7948) Cyrus Leung 2024-08-28 23:11:18 +08:00
• 98c12cffe5 [Doc] fix the autoAWQ example (#7937) Stas Bekman 2024-08-28 05:12:32 -07:00
• f52a43a8b9 [ci][test] fix pp test failure (#7945) youkaichao 2024-08-28 01:27:07 -07:00
• e3580537a4 [Performance] Enable chunked prefill and prefix caching together (#7753) Cody Yu 2024-08-28 00:36:31 -07:00
• f508e03e7f [Core] Async_output_proc: Add virtual engine support (towards pipeline parallel) (#7911) Alexander Matveev 2024-08-28 03:02:30 -04:00
• 51f86bf487 [mypy][CI/Build] Fix mypy errors (#7929) Cyrus Leung 2024-08-28 14:47:44 +08:00
• c166e7e43e [Bugfix] Allow ScalarType to be compiled with pytorch 2.3 and add checks for registering FakeScalarType and dynamo support. (#7886) bnellnm 2024-08-27 23:13:45 -04:00
• bc6e42a9b1 [hardware][rocm] allow rocm to override default env var (#7926) youkaichao 2024-08-27 19:50:06 -07:00
• fab5f53e2d [Core][VLM] Stack multimodal tensors to represent multiple images within each prompt (#7902) Peter Salas 2024-08-27 18:53:56 -07:00
• 9c71c97ae2 [mypy] Enable mypy type checking for vllm/core (#7229) Jonathan Berkhahn 2024-08-27 16:11:14 -07:00
• 5340a2dccf [Model] Add multi-image input support for LLaVA-Next offline inference (#7230) zifeitong 2024-08-27 16:09:02 -07:00
• 345be0e244 [benchmark] Update TGI version (#7917) Philipp Schmid 2024-08-28 00:07:53 +02:00
• fc911880cc [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766) Dipika SikkaandElizaWszola 2024-08-27 18:07:09 -04:00
• ed6f002d33 [cuda][misc] error on empty CUDA_VISIBLE_DEVICES (#7924) youkaichao 2024-08-27 12:06:11 -07:00
• b09c755be8 [Bugfix] Fix phi3v incorrect image_idx when using async engine (#7916) Isotr0py 2024-08-28 01:36:09 +08:00
• 42e932c7d4 [CI/Build][ROCm] Enabling tensorizer tests for ROCm (#7237) alexeykondrat 2024-08-27 13:09:13 -04:00
• 076169f603 [Hardware][Intel GPU] Add intel GPU pipeline parallel support. (#7810) Kunshang Ji 2024-08-28 01:07:02 +08:00
• 9db642138b [CI/Build][VLM] Cleanup multiple images inputs model test (#7897) Isotr0py 2024-08-27 23:28:30 +08:00
• 6fc4e6e07a [Model] Add Mistral Tokenization to improve robustness and chat encoding (#7739) Patrick von Platen 2024-08-27 14:40:02 +02:00
• 9606c7197d Revert #7509 (#7887) Cody Yu 2024-08-27 00:16:31 -07:00
• 64cc644425 [core][torch.compile] discard the compile for profiling (#7796) youkaichao 2024-08-26 21:33:58 -07:00
• 39178c7fbc [Tests] Disable retries and use context manager for openai client (#7565) Nick Hill 2024-08-26 21:33:17 -07:00
• 2eedede875 [Core] Asynchronous Output Processor (#7049) Megha AgarwalandAlexander Matveev 2024-08-26 20:53:20 -07:00
• 015e6cc252 [Misc] Update compressed tensors lifecycle to remove prefix from create_weights (#7825) Dipika Sikka 2024-08-26 20:09:34 -04:00
• 760e9f71a8 [Bugfix] neuron: enable tensor parallelism (#7562) omrishiv 2024-08-26 15:13:13 -07:00
• 05826c887b [misc] fix custom allreduce p2p cache file generation (#7853) youkaichao 2024-08-26 15:02:25 -07:00
• dd9857f5fa [Misc] Update gptq_marlin_24 to use vLLMParameters (#7762) Dipika SikkaandMichael Goin 2024-08-26 17:44:54 -04:00
• 665304092d [Misc] Update qqq to use vLLMParameters (#7805) Dipika Sikka 2024-08-26 15:16:15 -04:00
• 2deb029d11 [Performance][BlockManagerV2] Mark prefix cache block as computed after schedule (#7822) Cody Yu 2024-08-26 11:24:53 -07:00
• 029c71de11 [CI/Build] Avoid downloading all HF files in RemoteOpenAIServer (#7836) Cyrus Leung 2024-08-26 13:31:10 +08:00