Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• ba4994443a [Kernel] Add punica dimensions for Granite 3b and 8b (#5930) Joe Runde 2024-06-28 20:48:25 -06:00
• 906a19cdb0 [Misc] Extend vLLM Metrics logging API (#5925) William LinandAntoni Baum 2024-06-28 19:36:06 -07:00
• c4bca740e8 [Bugfix] fix missing last itl in openai completions benchmark (#5926) mcalman 2024-06-28 22:34:42 -04:00
• 7f83f40dee [Bugfix][TPU] Fix pad slot id (#5977) Woosuk Kwon 2024-06-28 18:55:17 -07:00
• 54814fd85b [Bugfix][TPU] Fix TPU sampler output (#5978) Woosuk Kwon 2024-06-28 18:14:16 -07:00
• 7041de4384 [Kernel] Flashinfer for prefill & decode, with Cudagraph support for decode (#4628) Lily LiuandLiuXiaoxuanPKU <llilyliupku@gmail.com>, bong-furiosa <bongwon.jang@furiosa.ai> 2024-06-28 15:28:49 -07:00
• 6a62cb82cc [Bugfix] Fix Engine Failing After Invalid Request - AsyncEngineDeadError (#5963) Robert ShawandRobert Shaw 2024-06-28 17:46:30 -04:00
• 5d2a1a9cf0 Unmark more files as executable (#5962) Tyler Michael Smith 2024-06-28 17:34:56 -04:00
• 4bf35ed9ae [Bugfix] Only add Attention.kv_scale if kv cache quantization is enabled (#5936) Michael Goin 2024-06-28 17:12:40 -04:00
• be0b3af9e0 Support Deepseek-V2 (#4650) wangding zengandPhilipp Moritz 2024-06-29 04:24:57 +08:00
• 2cd402e169 [ Bugfix ] Enabling Loading Models With Fused QKV/MLP on Disk with FP8 (#5921) Robert ShawandRobert Shaw 2024-06-28 14:43:49 -04:00
• b185230744 [ Misc ] Remove fp8_shard_indexer from Col/Row Parallel Linear (Simplify Weight Loading) (#5928) Robert ShawandRobert Shaw 2024-06-28 13:49:57 -04:00
• 6a2d659d28 [Bugfix] Fix compute datatype for cutlass 3.x epilogues (#5931) Tyler Michael Smith 2024-06-28 13:10:34 -04:00
• b2c620230a [Spec Decode] Introduce DraftModelRunner (#5799) Cody Yu 2024-06-28 09:17:51 -07:00
• b90d8cd832 [Distributed] Make it clear that % should not be in tensor dict keys. (#5927) xwjiang2010 2024-06-28 08:20:22 -07:00
• 3b752a6555 [CI/Build] [2/3] Reorganize entrypoints tests (#5904) Cyrus Leung 2024-06-28 22:59:18 +08:00
• ec1ad0046c [Bugfix] Better error message for MLPSpeculator when num_speculative_tokens is set too high (#5894) Thomas Parnell 2024-06-28 16:42:17 +02:00
• 57f09a419c [Hardware][Intel] OpenVINO vLLM backend (#5379) Ilya Lavrenov 2024-06-28 17:50:16 +04:00
• 5932634409 Unmark fused_moe config json file as executable (#5960) Tyler Michael Smith 2024-06-28 09:36:12 -04:00
• 5cbe8d155c [Core] Registry for processing model inputs (#5214) Cyrus Leungandywang96 2024-06-28 20:09:56 +08:00
• 0d0e3a42ac [Bugfix][Hardware][Intel CPU] Fix unpassed multi_modal_kwargs for CPU runner (#5956) Isotr0py 2024-06-28 20:03:41 +08:00
• 74d55c065b [VLM][BugFix] Make sure that multi_modal_kwargs can broadcast properly with ring buffer. (#5905) xwjiang2010andRoger Wang 2024-06-28 00:29:13 -07:00
• f136da15e1 [Hardware][TPU] Optimize KV cache swapping (#5878) Woosuk Kwon 2024-06-27 21:12:13 -07:00
• c3dde367f1 [Kernel][ROCm][AMD] fused_moe Triton configs v2 for mi300X (#5932) Divakar Verma 2024-06-27 15:41:08 -05:00
• 64e8d2a783 [core][misc] remove logical block (#5882) youkaichao 2024-06-27 13:34:55 -07:00
• 79c92c7c8a [Model] Add Gemma 2 (#5908) Woosuk Kwon 2024-06-27 13:33:56 -07:00
• 736ed38849 [CI/Build] Fix Args for _get_logits_warper in Sampler Test (#5922) Roger Wang 2024-06-27 11:43:04 -07:00
• 365791ff81 [BugFix] Fix min_tokens behaviour for multiple eos tokens (#5849) Nick Hill 2024-06-27 11:31:11 -07:00
• 691e29ecf3 [BugFix] Fix MLPSpeculator handling of num_speculative_tokens (#5876) Nick Hill 2024-06-27 10:59:33 -07:00
• 3fd02bda51 [doc][misc] add note for Kubernetes users (#5916) youkaichao 2024-06-27 10:07:07 -07:00
• 98cf2ed678 [Model][Bugfix] Implicit model flags and reenable Phi-3-Vision (#5896) Cyrus Leung 2024-06-28 00:08:10 +08:00
• e9d32d077d [CI/Build] [1/3] Reorganize entrypoints tests (#5526) Cyrus Leung 2024-06-27 20:43:17 +08:00
• 2061f0b8a7 [Bugfix] Fix img_sizes Parsing in Phi3-Vision (#5888) Roger Wang 2024-06-27 01:29:24 -07:00
• 96354d6a29 [Model] Add base class for LoRA-supported models (#5018) Cyrus Leung 2024-06-27 16:03:04 +08:00
• d12af207d2 [VLM][Bugfix] Make sure that multi_modal_kwargs is broadcasted properly (#5880) xwjiang2010 2024-06-27 00:15:24 -07:00
• 6eabc6cb0e [Doc] Add note about context length in Phi-3-Vision example (#5887) Cyrus Leung 2024-06-27 14:20:01 +08:00
• 2110557dab [BugFix] Fix cuda graph for MLPSpeculator (#5875) Nick HillandAbhinav Goyal 2024-06-26 21:12:10 -07:00
• b9e84259e9 [Misc] Add example for LLaVA-NeXT (#5879) Roger Wang 2024-06-26 17:57:16 -07:00
• 294104c3f9 [doc] update usage of env var to avoid conflict (#5873) youkaichao 2024-06-26 14:57:12 -07:00
• 38a1674abb Support CPU inference with VSX PowerPC ISA (#5652) Chip Kerchner 2024-06-26 17:53:04 -04:00
• f5c8628fdc [Bugfix][TPU] Fix CPU cache allocation (#5869) Woosuk Kwon 2024-06-26 13:42:40 -07:00
• cbc53b6b8d [Hardware][TPU] Support parallel sampling & Swapping (#5855) Woosuk Kwon 2024-06-26 11:07:49 -07:00
• c54269d967 [Frontend] Add tokenize/detokenize endpoints (#5054) sasha0552 2024-06-26 16:54:22 +00:00
• 5bfd1bbc98 [Kernel] Adding bias epilogue support for cutlass_scaled_mm (#5560) 3 people 2024-06-26 11:16:00 -04:00
• 6984c02a27 [CI/Build] Refactor image test assets (#5821) Cyrus Leung 2024-06-26 16:02:34 +08:00
• 3439c5a8e3 [Bugfix][TPU] Fix KV cache size calculation (#5860) Woosuk Kwon 2024-06-26 00:58:23 -07:00
• 6806998bf9 [Bugfix] Fix embedding to support 2D inputs (#5829) Woosuk Kwon 2024-06-26 00:15:22 -07:00
• 515080ad2f [bugfix][distributed] fix shm broadcast when the queue size is full (#5801) youkaichao 2024-06-25 21:56:02 -07:00
• 3aa7b6cf66 [Misc][Doc] Add Example of using OpenAI Server with VLM (#5832) Roger Wang 2024-06-25 20:34:25 -07:00
• dda4811591 [Core] Refactor Worker and ModelRunner to consolidate control plane communication (#5408) Stephanie WangandStephanie 2024-06-25 20:30:03 -07:00
• 82079729cc [Bugfix] Fix assertion in NeuronExecutor (#5841) aws-patlange 2024-06-25 19:52:10 -07:00
• c2a8ac75e0 [CI/Build] Add E2E tests for MLPSpeculator (#5791) Thomas Parnell 2024-06-26 01:04:08 +01:00
• f178e56c68 [Hardware][TPU] Raise errors for unsupported sampling params (#5850) Woosuk Kwon 2024-06-25 16:58:23 -07:00
• dd793d1de5 [Hardware][AMD][CI/Build][Doc] Upgrade to ROCm 6.1, Dockerfile improvements, test fixes (#5422) Matt Wong 2024-06-25 17:56:15 -05:00
• bc34937d68 [Hardware][TPU] Refactor TPU backend (#5831) Woosuk Kwon 2024-06-25 15:25:52 -07:00
• dd248f7675 [Misc] Update w4a16 compressed-tensors support to include w8a16 (#5794) Dipika Sikka 2024-06-25 15:23:35 -04:00
• d9b34baedd [CI/Build] Add unit testing for FlexibleArgumentParser (#5798) Michael Goin 2024-06-25 15:18:03 -04:00
• c18ebfdd71 [doc][distributed] add both gloo and nccl tests (#5834) youkaichao 2024-06-25 12:10:28 -07:00
• 67882dbb44 [Core] Add fault tolerance for RayTokenizerGroupPool (#5748) Antoni Baum 2024-06-25 10:15:10 -07:00
• 7b99314301 [Misc] Remove useless code in cpu_worker (#5824) Jie Fu (傅杰) 2024-06-26 00:41:36 +08:00
• 2ce5d6688b [Speculative Decoding] Support draft model on different tensor-parallel size than target model (#5414) Woo-Yeon Lee 2024-06-25 18:56:06 +09:00
• f23871e9ee [Doc] Add notice about breaking changes to VLMs (#5818) Cyrus Leung 2024-06-25 16:25:03 +08:00
• e9de9dd551 [ci] Remove aws template (#5757) Kevin H. Luu 2024-06-24 21:09:02 -07:00
• ba991d5c84 [Bugfix] Fix FlexibleArgumentParser replaces _ with - for actual args (#5795) Chang Su 2024-06-24 16:01:19 -07:00
• 1744cc99ba [Doc] Add Phi-3-medium to list of supported models (#5788) Michael Goin 2024-06-24 13:48:55 -04:00
• e72dc6cb35 [Doc] Add "Suggest edit" button to doc pages (#5789) Michael Goin 2024-06-24 13:26:17 -04:00
• c246212952 [doc][faq] add warning to download models for every nodes (#5783) youkaichao 2024-06-24 00:37:42 -07:00
• edd5fe5fa2 [Bugfix] Add phi3v resize for dynamic shape and fix torchvision requirement (#5772) Isotr0py 2024-06-24 12:11:53 +08:00
• 5d4d90536f [Distributed] Add send and recv helpers (#5719) Murali Andoorveedu 2024-06-23 17:42:28 -04:00
• 6c916ac8a8 [BugFix] [Kernel] Add Cutlass2x fallback kernels (#5744) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-06-24 02:37:11 +05:30
• 832ea88fcb [core][distributed] improve shared memory broadcast (#5754) youkaichao 2024-06-22 10:00:43 -07:00
• 8c00f9c15d [Docs][TPU] Add installation tip for TPU (#5761) Woosuk Kwon 2024-06-21 23:09:40 -07:00
• 0cbc1d2b4f [Bugfix] Fix pin_lora error in TPU executor (#5760) Woosuk Kwon 2024-06-21 22:25:14 -07:00
• ff9ddbceee [Misc] Remove #4789 workaround left in vllm/entrypoints/openai/run_batch.py (#5756) zifeitong 2024-06-21 20:33:12 -07:00
• 9c62db07ed [Model] Support Qwen-VL and Qwen-VL-Chat models with text-only inputs (#5710) Jie Fu (傅杰)andRoger Wang 2024-06-22 10:07:08 +08:00
• cf90ae0123 [CI][Hardware][Intel GPU] add Intel GPU(XPU) ci pipeline (#5616) Kunshang Ji 2024-06-22 08:09:34 +08:00
• f5dda63eb5 [LoRA] Add support for pinning lora adapters in the LRU cache (#5603) rohithkrn 2024-06-21 15:42:46 -07:00
• 7187507301 [ci][test] fix ca test in main (#5746) youkaichao 2024-06-21 14:04:26 -07:00
• f1e72cc19a [BugFix] exclude version 1.15.0 for modelscope (#5668) zhyncs 2024-06-22 03:15:48 +08:00
• 5b15bde539 [Doc] Documentation on supported hardware for quantization methods (#5745) Michael Goin 2024-06-21 12:44:29 -04:00
• bd620b01fb [Kernel][CPU] Add Quick gelu to CPU (#5717) Roger Wang 2024-06-20 23:39:40 -07:00
• d9a252bc8e [Core][Distributed] add shm broadcast (#5399) youkaichaoandCody Yu 2024-06-20 22:12:35 -07:00
• 67005a07bc [Bugfix] Add fully sharded layer for QKVParallelLinearWithLora (#5665) Jee LiandAntoni Baum 2024-06-21 12:46:28 +08:00
• c35e4a3dd7 [BugFix] Fix test_phi3v.py (#5725) Chang Su 2024-06-20 21:45:34 -07:00
• 1f5674218f [Kernel] Add punica dimension for Qwen2 LoRA (#5441) Jinzhen Lin 2024-06-21 08:55:41 +08:00
• b12518d3cf [Model] MLPSpeculator speculative decoding support (#4947) 4 people 2024-06-20 20:23:12 -04:00
• 6c5b7af152 [distributed][misc] use fork by default for mp (#5669) youkaichao 2024-06-20 17:06:34 -07:00
• 8065a7e220 [Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718) Michael Goin 2024-06-20 19:00:13 -04:00
• 3f3b6b2150 [Bugfix] Fix the CUDA version check for FP8 support in the CUTLASS kernels (#5715) Tyler Michael Smith 2024-06-20 14:36:10 -04:00
• a7dcc62086 [Kernel] Update Cutlass int8 kernel configs for SM80 (#5275) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-06-20 19:03:21 +05:30
• ad137cd111 [Model] Port over CLIPVisionModel for VLMs (#5591) Roger Wang 2024-06-20 04:52:09 -07:00
• 111af1fa2c [Kernel] Update Cutlass int8 kernel configs for SM90 (#5514) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-06-20 12:07:08 +05:30
• 1b2eaac316 [Bugfix][Doc] FIx Duplicate Explicit Target Name Errors (#5703) Roger Wang 2024-06-19 23:10:47 -07:00
• 3730a1c832 [Misc] Improve conftest (#5681) Cyrus Leung 2024-06-20 10:09:21 +08:00
• 949e49a685 [ci] Limit num gpus if specified for A100 (#5694) Kevin H. Luu 2024-06-19 16:30:03 -07:00
• 4a30d7e3cc [Misc] Add per channel support for static activation quantization; update w8a8 schemes to share base classes (#5650) Dipika Sikka 2024-06-19 18:06:44 -04:00
• e83db9e7e3 [Doc] Update docker references (#5614) Rafael Vasquez 2024-06-19 18:01:45 -04:00
• 78687504f7 [Bugfix] AsyncLLMEngine hangs with asyncio.run (#5654) zifeitong 2024-06-19 13:57:12 -07:00
• d571ca0108 [ci][distributed] add tests for custom allreduce (#5689) youkaichao 2024-06-19 13:16:04 -07:00
• afed90a034 [Frontend][Bugfix] Fix preemption_mode -> preemption-mode for CLI arg in arg_utils.py (#5688) Michael Goin 2024-06-19 14:41:42 -04:00