Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• cf349c4a97 [Bugfix][CPU] Fix CPU embedding runner with tensor parallel (#10394) Isotr0py 2024-11-17 15:12:04 +08:00
• 905d0f0af4 [CI/Build] Fix IDC hpu [Device not found] issue (#10384) Chendi.Xue 2024-11-17 00:58:22 -06:00
• 643ecf7b11 [V1] Refactor model executable interface for all text-only language models (#10374) Roger Wang 2024-11-16 21:18:46 -08:00
• 4fd9375028 [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) youkaichao 2024-11-16 18:02:14 -08:00
• 661a34fd4f [V1] Add code owners for V1 (#10397) Woosuk Kwon 2024-11-16 10:45:26 -08:00
• 361c29e174 [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) 电脑星人 2024-11-17 02:10:00 +08:00
• b98d89efd4 [Misc] Medusa supports custom bias (#10361) Sky Lee 2024-11-17 00:33:01 +08:00
• 8b6725b0cf [Misc] Update benchmark to support image_url file or http (#10287) Jaehyun An 2024-11-16 19:15:40 +09:00
• 1d75472626 [BugFix] [Kernel] Fix GPU SEGV occuring in fused_moe kernel (#10385) rasmith 2024-11-16 03:55:05 -06:00
• 2f427c2d16 [misc][plugin] improve log messages (#10386) youkaichao 2024-11-16 01:23:20 -08:00
• 755b85359b [doc] add doc for the plugin system (#10372) youkaichao 2024-11-15 21:46:27 -08:00
• 32e46e000f [Frontend] Automatic detection of chat content format from AST (#9919) Cyrus Leung 2024-11-16 13:35:40 +08:00
• 4f168f69a3 [Docs] Misc updates to TPU installation instructions (#10165) Michael Green 2024-11-15 21:26:17 +00:00
• 3e8d14d8a1 [Doc] Move PR template content to docs (#10159) Russell Bryant 2024-11-15 16:20:20 -05:00
• a067f85e08 [Frontend] Add --version flag to CLI (#10369) Russell Bryant 2024-11-15 16:13:53 -05:00
• c76ac49d26 [Docs] Add Nebius as sponsors (#10371) Simon Mo 2024-11-15 12:47:40 -08:00
• a6221a144a [Misc] bump mistral common version (#10367) Simon Mo 2024-11-15 09:48:07 -08:00
• 79ee45b428 [Misc] Bump up test_fused_moe tolerance (#10364) ElizaWszola 2024-11-15 17:31:18 +01:00
• 691a3ec047 [Bugfix] Ensure special tokens are properly filtered out for guided structured output with MistralTokenizer (#10363) Guillaume Calmettes 2024-11-15 15:50:40 +01:00
• 3a763ba0c3 [core][misc] keep compatibility for old-style classes (#10356) youkaichao 2024-11-15 05:55:51 -08:00
• f2056f726d [Misc] Fix some help info of arg_utils to improve readability (#10362) shangmingc 2024-11-15 20:40:30 +08:00
• 1d65ec7eeb [Bugfix] Fix fully sharded LoRA bug (#10352) Jee Jee Li 2024-11-15 18:34:58 +08:00
• 26908554b2 [Doc] Remove float32 choice from --lora-dtype (#10348) Xin Yang 2024-11-15 02:22:57 -08:00
• b311efd0bd [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) Cyrus Leung 2024-11-15 17:34:17 +08:00
• 3d158cdc8d Add default value to avoid Falcon crash (#5363) (#10347) wchen61 2024-11-15 16:52:20 +08:00
• 02dbf30e9a [Build] skip renaming files for release wheels pipeline (#9671) Simon Mo 2024-11-14 23:31:52 -08:00
• 2ac6d0e75b [Misc] Consolidate pooler config overrides (#10351) Cyrus Leung 2024-11-15 14:59:00 +08:00
• 2ec8827288 [Bugfix] Qwen-vl output is inconsistent in speculative decoding (#10350) Sky Lee 2024-11-15 13:40:10 +08:00
• b40cf6402e [Model] Support Qwen2 embeddings and use tags to select model tests (#10184) Cyrus Leung 2024-11-15 12:23:09 +08:00
• 2885ba0e24 [Misc] Change RedundantReshapesPass and FusionPass logging from info to debug (#10308) Tyler Michael Smith 2024-11-14 21:44:26 -05:00
• bf2ddc6610 [bugfix] Fix static asymmetric quantization case (#10334) Luka GovedičandDaniël de Kok 2024-11-14 20:35:11 -05:00
• 972112d82f [Bugfix] Fix unable to load some models (#10312) Cyrus Leung 2024-11-15 08:55:54 +08:00
• 11cd1ae6ad [Tool parsing] Improve / correct mistral tool parsing (#10333) Patrick von Platen 2024-11-15 01:42:49 +01:00
• 554af9228d [Bugfix] use AF_INET6 for OpenAI Compatible Server with ipv6 (#9583) Zijin Xiao 2024-11-15 08:38:53 +08:00
• b2e0ad3b59 [Perf] Reduce peak memory usage of llama (#10339) Murali Andoorveedu 2024-11-14 16:38:20 -08:00
• 4a18fd14ba Support Roberta embedding models (#9387) Maximilien de BayserandFlavia Beo 2024-11-14 18:23:29 -03:00
• 1dbae0329c [Docs] Publish meetup slides (#10331) Woosuk Kwon 2024-11-14 08:19:38 -08:00
• 675d603400 [CI/Build] Make shellcheck happy (#10285) Cyrus Leung 2024-11-14 17:47:53 +08:00
• 03025c023f [CI/Build] Fix CPU CI online inference timeout (#10314) Isotr0py 2024-11-14 16:45:32 +08:00
• 29f3ef26a3 [ci][distributed] disable hanging tests (#10317) youkaichao 2024-11-14 00:23:39 -08:00
• 294bf467ba [Model] Add BNB quantization support for Idefics3 (#10310) B-201andJee Jee Li 2024-11-14 14:31:44 +08:00
• 52b48c1ead [BugFix]: properly deserialize tool_calls iterator before processing by mistral-common when MistralTokenizer is used (#9951) Guillaume Calmettes 2024-11-14 05:48:16 +01:00
• f67ce05d0b [Frontend] Pythonic tool parser (#9859) Mike Depinet 2024-11-13 20:14:34 -08:00
• e0853b6508 [Misc] format.sh: Simplify tool_version_check (#10305) Russell Bryant 2024-11-13 22:12:35 -05:00
• 504ac53d18 [misc] error early for old-style class (#10304) youkaichao 2024-11-13 18:55:39 -08:00
• 15bb8330aa [Bugfix] Fix tensor parallel for qwen2 classification model (#10297) Isotr0py 2024-11-14 10:54:59 +08:00
• ac49b59d8b [Bugfix] bitsandbytes models fail to run pipeline parallel (#10200) HoangCongDuc 2024-11-14 00:56:39 +08:00
• 0b8bb86bf1 [1/N] Initial prototype for multi-modal processor (#10044) Cyrus Leung 2024-11-13 20:39:03 +08:00
• bb7991aa29 [V1] Add missing tokenizer options for Detokenizer (#10288) Roger Wang 2024-11-13 03:02:56 -08:00
• d909acf9fe [Model][LoRA]LoRA support added for idefics3 (#10281) B-201 2024-11-13 17:25:59 +08:00
• b6dde33019 [Core] Flashinfer - Remove advance step size restriction (#10282) Pavani Majety 2024-11-13 00:29:32 -08:00
• 1b886aa104 [Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944) Austin VeselkaandFurtherAI 2024-11-13 02:28:13 -06:00
• 3945c82346 [Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (#10221) 电脑星人 2024-11-13 15:07:22 +08:00
• 032fcf16ae [Doc] Fix typo in arg_utils.py (#10264) Xin Yang 2024-11-12 21:54:52 -08:00
• 56a955e774 Bump to compressed-tensors v0.8.0 (#10279) Dipika Sikka 2024-11-13 00:54:10 -05:00
• bbd3e86926 [V1] Support VLMs with fine-grained scheduling (#9871) Woosuk KwonandRoger Wang 2024-11-12 20:53:13 -08:00
• 0d4ea3fb5c [core][distributed] use tcp store directly (#10275) youkaichao 2024-11-12 17:36:08 -08:00
• 112fa0bbe5 [V1] Fix CI tests on V1 engine (#10272) Woosuk Kwon 2024-11-12 16:17:20 -08:00
• 377b74fe87 Revert "[ci][build] limit cmake version" (#10271) youkaichao 2024-11-12 15:06:48 -08:00
• 18081451f9 [doc] improve debugging doc (#10270) youkaichao 2024-11-12 14:43:52 -08:00
• 96ae0eaeb2 [doc] fix location of runllm widget (#10266) youkaichao 2024-11-12 14:34:39 -08:00
• 1f55e05713 [V1] Enable Inductor when using piecewise CUDA graphs (#10268) Woosuk Kwon 2024-11-12 13:39:56 -08:00
• 8a06428c70 [LoRA] Adds support for bias in LoRA (#5733) UmeshandUmesh Deshpande 2024-11-12 11:08:40 -08:00
• b41fb9d3b1 [Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982) sroy745 2024-11-12 10:53:57 -08:00
• 7c65527918 [V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to EngineCoreRequest (#10245) Woosuk Kwon 2024-11-12 08:57:14 -08:00
• 47db6ec831 [Frontend] Add per-request number of cached token stats (#10174) zifeitong 2024-11-12 08:42:28 -08:00
• 176fcb1c71 [Bugfix] Fix QwenModel argument (#10262) Jie Fu (傅杰) 2024-11-13 00:36:51 +08:00
• a838ba7254 [Misc]Fix Idefics3Model argument (#10255) Jee Jee Li 2024-11-12 21:07:11 +08:00
• 36c513a076 [BugFix] Do not raise a ValueError when tool_choice is set to the supported none option and tools are not defined. (#10000) Guillaume Calmettes 2024-11-12 12:13:46 +01:00
• d201d41973 [CI][CPU]refactor CPU tests to allow to bind with different cores (#10222) Yuan 2024-11-12 18:07:32 +08:00
• 3a28f18b0b [doc] explain the class hierarchy in vLLM (#10240) youkaichao 2024-11-11 22:56:44 -08:00
• 812c981fa0 Splitting attention kernel file (#10091) Aleksandr MalyshevandAleksandr Malyshev 2024-11-11 22:55:07 -08:00
• 7f5edb5900 [Misc][LoRA] Replace hardcoded cuda device with configurable argument (#10223) Jee Jee Li 2024-11-12 11:10:15 +08:00
• eea55cca5b [1/N] torch.compile user interface design (#10237) youkaichao 2024-11-11 18:01:06 -08:00
• 9cdba9669c [Doc] Update help text for --distributed-executor-backend (#10231) Russell Bryant 2024-11-11 20:55:09 -05:00
• d1c6799b88 [doc] update debugging guide (#10236) youkaichao 2024-11-11 15:21:12 -08:00
• 6ace6fba2c [V1] AsyncLLM Implementation (#9826) 5 people 2024-11-11 18:05:38 -05:00
• 08f93e7439 Make shutil rename in python_only_dev (#10233) Nikolai Shcheglov 2024-11-11 16:29:19 -06:00
• 9d5b4e4dea [V1] Enable custom ops with piecewise CUDA graphs (#10228) Woosuk Kwon 2024-11-11 11:58:07 -08:00
• 8a7fe47d32 [misc][distributed] auto port selection and disable tests (#10226) youkaichao 2024-11-11 11:54:59 -08:00
• 4800339c62 Add docs on serving with Llama Stack (#10183) Yuan TangandRussell Bryant 2024-11-11 14:28:55 -05:00
• fe15729a2b [V1] Use custom ops for piecewise CUDA graphs (#10227) Woosuk Kwon 2024-11-11 11:26:48 -08:00
• 330e82d34a [v1][torch.compile] support managing cudagraph buffer (#10203) youkaichaoandWoosuk Kwon 2024-11-11 11:10:27 -08:00
• d7a4f2207b [V1] Do not use inductor for piecewise CUDA graphs (#10225) Woosuk Kwon 2024-11-11 11:05:57 -08:00
• f9dadfbee3 [V1] Fix detokenizer ports (#10224) Woosuk Kwon 2024-11-11 10:42:07 -08:00
• 25144ceed0 Bump actions/setup-python from 5.2.0 to 5.3.0 (#10209) dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 2024-11-11 17:24:10 +00:00
• e6de9784d2 [core][distributed] add stateless process group (#10216) youkaichao 2024-11-11 09:02:14 -08:00
• 36fc439de0 [Doc] fix doc string typo in block_manager swap_out function (#10212) Yangcheng Li 2024-11-12 00:53:07 +08:00
• 874f551b36 [Metrics] add more metrics (#4464) 3 people 2024-11-12 00:17:38 +08:00
• 2cebda42bb [Bugfix][Hardware][CPU] Fix broken encoder-decoder CPU runner (#10218) Isotr0py 2024-11-11 20:37:58 +08:00
• 5fb1f935b0 [V1] Allow tokenizer_mode and trust_remote_code for Detokenizer (#10211) Roger Wang 2024-11-11 02:01:18 -08:00
• 36e4acd02a [LoRA][Kernel] Remove the unused libentry module (#10214) Jee Jee Li 2024-11-11 17:43:23 +08:00
• 58170d6503 [Hardware][CPU] Add embedding models support for CPU backend (#10193) Isotr0py 2024-11-11 16:54:28 +08:00
• 9804ac7c7c Bump the patch-update group with 5 updates (#10210) dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 2024-11-11 07:22:40 +00:00
• f89d18ff74 [6/N] pass whole config to inner model (#10205) youkaichao 2024-11-10 22:41:46 -08:00
• f0f2e5638e [doc] improve debugging code (#10206) youkaichao 2024-11-10 17:49:40 -08:00
• ad9a78bf64 [Doc] Fix typo error in vllm/entrypoints/openai/cli_args.py (#10196) yansh97 2024-11-11 08:14:22 +08:00
• 73b9083e99 [misc] improve cloudpickle registration and tests (#10202) youkaichao 2024-11-10 16:10:53 -08:00
• 20cf2f553c [Misc] small fixes to function tracing file path (#9543) Shawn Duandyoukaichao 2024-11-11 07:21:06 +08:00
• bfb7d61a7c [doc] Polish the integration with huggingface doc (#10195) Yongzaoandyoukaichao 2024-11-11 02:22:04 +08:00