Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 0b769992ec [Bugfix]: Use float32 for base64 embedding (#7855) ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 2024-08-26 06:16:38 +03:00
• 1856aff4d6 [Spec Decoding] Streamline batch expansion tensor manipulation (#7851) Nick Hill 2024-08-25 15:45:14 -07:00
• 70c094ade6 [misc][cuda] improve pynvml warning (#7852) youkaichao 2024-08-25 14:30:09 -07:00
• 2059b8d9ca [Misc] Remove snapshot_download usage in InternVL2 test (#7835) Isotr0py 2024-08-25 23:53:09 +08:00
• 8aaf3d5347 [Model][VLM] Support multi-images inputs for Phi-3-vision models (#7783) Isotr0py 2024-08-25 19:51:20 +08:00
• 80162c44b1 [Bugfix] Fix Phi-3v crash when input images are of certain sizes (#7840) zifeitong 2024-08-24 18:16:24 -07:00
• aab0fcdb63 [ci][test] fix RemoteOpenAIServer (#7838) youkaichao 2024-08-24 10:31:28 -07:00
• ea9fa160e3 [ci][test] exclude model download time in server start time (#7834) youkaichao 2024-08-24 01:03:27 -07:00
• 7d9ffa2ae1 [misc][core] lazy import outlines (#7831) youkaichao 2024-08-24 00:51:38 -07:00
• d81abefd2e [Frontend] add json_schema support from OpenAI protocol (#7654) Tyler Rockwood 2024-08-24 01:07:24 -05:00
• 8da48e4d95 [Frontend] Publish Prometheus metrics in run_batch API (#7641) Pooya Davoodi 2024-08-23 23:04:22 -07:00
• 6885fde317 [Bugfix] Fix run_batch logger (#7640) Pooya Davoodi 2024-08-23 13:58:26 -07:00
• 9db93de20c [Core] Add multi-step support to LLMEngine (#7789) Alexander Matveev 2024-08-23 15:45:53 -04:00
• 09c7792610 Bump version to v0.5.5 (#7823) Simon Mo 2024-08-23 11:35:33 -07:00
• f1df5dbfd6 [Misc] Update marlin to use vLLMParameters (#7803) Dipika Sikka 2024-08-23 14:30:52 -04:00
• 35ee2ad6b9 [github][misc] promote asking llm first (#7809) youkaichao 2024-08-23 09:38:50 -07:00
• e25fee57c2 [BugFix] Fix server crash on empty prompt (#7746) Maximilien de Bayser 2024-08-23 10:12:44 -03:00
• faeddb565d [misc] Add Torch profiler support for CPU-only devices (#7806) Jie Fu (傅杰) 2024-08-23 13:46:25 +08:00
• fc5ebbd1d3 [Hardware][Intel GPU] refactor xpu_model_runner for tp (#7712) Kunshang Ji 2024-08-23 11:06:54 +08:00
• c01a6cb231 [Ray backend] Better error when pg topology is bad. (#7584) SangBin Choandyoukaichao 2024-08-22 17:44:25 -07:00
• b903e1ba7f [Frontend] error suppression cleanup (#7786) Joe Runde 2024-08-22 15:50:21 -06:00
• a152246428 [Misc] fix typo in triton import warning (#7794) Siyuan Liu 2024-08-22 13:51:23 -07:00
• 666ad0aa16 [ci] Cleanup & refactor Dockerfile to pass different Python versions and sccache bucket via build args (#7705) Kevin H. Luu 2024-08-22 13:10:55 -07:00
• 15310b5101 [Bugfix] Use LoadFormat values for vllm serve --load-format (#7784) Michael Goin 2024-08-22 14:37:08 -04:00
• 57792ed469 [Doc] Fix incorrect docs from #7615 (#7788) Peter Salas 2024-08-22 10:02:06 -07:00
• d3b5b98021 [Misc] Enhance prefix-caching benchmark tool (#6568) Jiaxin Shan 2024-08-23 00:32:02 +08:00
• cc0eaf12b1 [Bugfix] spec decode handle None entries in topk args in create_sequence_group_output (#7232) Travis Johnson 2024-08-22 07:33:48 -06:00
• 955b5191c9 [Misc] update fp8 to use vLLMParameter (#7437) Dipika Sikka 2024-08-22 08:36:18 -04:00
• 55d63b1211 [Bugfix] Don't build machete on cuda <12.0 (#7757) Lucas Wilkinson 2024-08-22 08:28:52 -04:00
• 4f419c00a6 Fix ShardedStateLoader for vllm fp8 quantization (#7708) Flex Wang 2024-08-22 05:25:04 -07:00
• a3fce56b88 [Speculative Decoding] EAGLE Implementation with Top-1 proposer (#6830) Abhinav Goyal 2024-08-22 15:12:24 +05:30
• b3856bef7d [Misc] Use torch.compile for GemmaRMSNorm (#7642) Woosuk Kwon 2024-08-22 01:14:13 -07:00
• 8c6f694a79 [ci] refine dependency for distributed tests (#7776) youkaichao 2024-08-22 00:54:15 -07:00
• eeee1c3b1a [TPU] Avoid initializing TPU runtime in is_tpu (#7763) Woosuk Kwon 2024-08-21 21:31:49 -07:00
• aae74ef95c Revert "[Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527)" (#7764) Michael Goin 2024-08-21 23:42:14 -04:00
• cde9183b40 [Bug][Frontend] Improve ZMQ client robustness (#7443) Joe Runde 2024-08-21 20:18:11 -06:00
• df1a21131d [Model] Fix Phi-3.5-vision-instruct 'num_crops' issue (#7710) zifeitong 2024-08-21 18:36:24 -07:00
• 7937009a7e [Kernel] Replaced blockReduce[...] functions with cub::BlockReduce (#7233) Luka GovedičandMichael Goin 2024-08-21 20:18:00 -04:00
• 9984605412 [AMD][CI/Build] Disambiguation of the function call for ROCm 6.2 headers compatibility (#7477) Gregory ShtrasbergandCharlie Fu 2024-08-21 19:47:36 -04:00
• 7eebe8ccaa [distributed][misc] error on same VLLM_HOST_IP setting (#7756) youkaichao 2024-08-21 16:25:34 -07:00
• 8678a69ab5 [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527) Dipika SikkaandElizaWszola 2024-08-21 19:17:10 -04:00
• 5844017285 [ci] [multi-step] narrow multi-step test dependency paths (#7760) William Lin 2024-08-21 15:52:40 -07:00
• 1ca0d4f86b [Model] Add UltravoxModel and UltravoxConfig (#7615) Peter Salas 2024-08-21 15:49:39 -07:00
• dd53c4b023 [misc] Add Torch profiler support (#7451) William LinandCody Yu 2024-08-21 15:39:26 -07:00
• 970dfdc01d [Frontend] Improve Startup Failure UX (#7716) Robert Shaw 2024-08-21 15:53:01 -04:00
• 91f4522cbf [multi-step] Raise error if not using async engine (#7703) William Lin 2024-08-21 11:49:19 -07:00
• 1b32e02648 [Bugfix] Pass PYTHONPATH from setup.py to CMake (#7730) sasha0552 2024-08-21 18:17:48 +00:00
• f7e3b0c5aa [Bugfix][Frontend] Fix Issues Under High Load With zeromq Frontend (#7394) Robert ShawandNick Hill 2024-08-21 13:34:14 -04:00
• d3c002eadc [Bugfix] chat method add_generation_prompt param (#7734) Brian Li 2024-08-22 01:33:35 +08:00
• 9b73a2f498 [Spec Decoding] Use target model max length as default for draft model (#7706) Nick Hill 2024-08-21 12:23:22 -04:00
• 6925cdbeea [Bugfix][Hardware][CPU] Fix mm_limits initialization for CPU backend (#7735) Isotr0py 2024-08-22 00:23:03 +08:00
• 53328d7536 [BUG] fix crash on flashinfer backend with cudagraph disabled, when attention group_size not in [1,2,4,8] (#7509) LI MOU 2024-08-21 23:54:31 +08:00
• c75363fbc0 [BugFix] Avoid premature async generator exit and raise all exception variations (#7698) Nick Hill 2024-08-21 11:45:55 -04:00
• dd3fa0e430 [Bugfix] Mirror jinja2 in pyproject.toml (#7723) sasha0552 2024-08-21 13:41:17 +00:00
• baaedfdb2d [mypy] Enable following imports for entrypoints (#7248) 3 people 2024-08-21 14:28:21 +08:00
• 4506641212 [Doc] Section for Multimodal Language Models (#7719) Roger Wang 2024-08-20 23:24:01 -07:00
• 12e1c65bc9 [Model] Add AWQ quantization support for InternVL2 model (#7187) Isotr0py 2024-08-21 14:18:57 +08:00
• b74a125800 [ci] try to log process using the port to debug the port usage (#7711) youkaichao 2024-08-20 17:41:12 -07:00
• 66a9e713a7 [Core] Pipe worker_class_fn argument in Executor (#7707) Antoni Baum 2024-08-20 17:37:39 -07:00
• 9e51b6a626 [ci][test] adjust max wait time for cpu offloading test (#7709) youkaichao 2024-08-20 17:12:44 -07:00
• 6e4658c7aa [Intel GPU] fix xpu not support punica kernel (which use torch.library.custom_op) (#7685) Kunshang Ji 2024-08-21 03:01:09 +08:00
• 3b682179dd [Core] Add AttentionState abstraction (#7663) Antoni Baum 2024-08-20 11:50:45 -07:00
• c6af027a35 [Misc] Add jinja2 as an explicit build requirement (#7695) Lucas Wilkinson 2024-08-20 13:17:47 -04:00
• 2aa00d59ad [CI/Build] Pin OpenTelemetry versions and make errors clearer (#7266) Ronen Schaffer 2024-08-20 20:02:21 +03:00
• c42590f97a [Hardware] [Intel GPU] refactor xpu worker/executor (#7686) Kunshang Ji 2024-08-21 00:54:10 +08:00
• aae6927be0 [VLM][Model] Add test for InternViT vision encoder (#7409) Isotr0py 2024-08-20 23:10:20 +08:00
• 398521ad19 [OpenVINO] Updated documentation (#7687) Ilya Lavrenov 2024-08-20 17:33:56 +04:00
• 5288c06aa0 [Kernel] (1/N) Machete - Hopper Optimized Mixed Precision Linear Kernel (#7174) Lucas Wilkinson 2024-08-20 09:09:33 -04:00
• b6f99a6ffe [Core] Refactor executor classes for easier inheritance (#7673) Kunshang Ji 2024-08-20 15:56:50 +08:00
• ad28a74beb [misc][cuda] add warning for pynvml user (#7675) youkaichao 2024-08-20 00:35:09 -07:00
• e6d811dd13 [XPU] fallback to native implementation for xpu custom op (#7670) jianyizh 2024-08-20 15:26:09 +08:00
• c4be16e1a7 [misc] add nvidia related library in collect env (#7674) youkaichao 2024-08-19 23:22:49 -07:00
• 3d8a5f063d [CI] Organizing performance benchmark files (#7616) Kuntai Du 2024-08-19 22:43:54 -07:00
• f4fc7337bf [Bugfix] support tie_word_embeddings for all models (#5724) Zijian Hu 2024-08-19 20:00:04 -07:00
• 0df7ec0b2d [ci] Install Buildkite test suite analysis (#7667) Kevin H. Luu 2024-08-19 19:55:04 -07:00
• 312f761232 [Speculative Decoding] Fixing hidden states handling in batch expansion (#7508) Abhinav Goyal 2024-08-20 06:28:14 +05:30
• e54ebc2f8f [doc] fix doc build error caused by msgspec (#7659) youkaichao 2024-08-19 17:50:59 -07:00
• 67e02fa8a4 [Bugfix] use StoreBoolean instead of type=bool for --disable-logprobs-during-spec-decoding (#7665) Travis Johnson 2024-08-19 18:43:09 -06:00
• 43735bf5e1 [TPU] Remove redundant input tensor cloning (#7660) Woosuk Kwon 2024-08-19 15:55:04 -07:00
• da115230fd [Bugfix] Don't disable existing loggers (#7664) Andrew Song 2024-08-19 15:11:58 -07:00
• 7601cb044d [Core] Support tensor parallelism for GGUF quantization (#7520) Isotr0py 2024-08-20 05:30:14 +08:00
• 47b65a5508 [core] Multi Step Scheduling (#7000) William Linandafeldman-nm 2024-08-19 13:52:13 -07:00
• dad961ef5c [Bugfix] fix lora_dtype value type in arg_utils.py - part 2 (#5428) Ali Panahi 2024-08-19 13:47:00 -07:00
• 3ac50b47d0 [MISC] Add prefix cache hit rate to metrics (#7606) Cody Yu 2024-08-19 11:52:07 -07:00
• df845b2b46 [Misc] Remove Gemma RoPE (#7638) Woosuk Kwon 2024-08-19 09:29:31 -07:00
• 1a36287b89 [Bugfix] Fix xpu build (#7644) Kunshang Ji 2024-08-19 13:00:09 +08:00
• f710fb5265 [Core] Use flashinfer sampling kernel when available (#7137) Peng GuanwenandMichael Goin 2024-08-19 11:24:03 +08:00
• ff7ec82c4d [Core] Optimize SPMD architecture with delta + serialization optimization (#7109) SangBin Cho 2024-08-18 17:57:20 -07:00
• 200a2ffa6b [Misc] Refactor Llama3 RoPE initialization (#7637) Woosuk Kwon 2024-08-18 17:18:12 -07:00
• 40e1360bb6 [CI/Build] Add text-only test for Qwen models (#7475) Alex Brooks 2024-08-18 17:43:46 -06:00
• e3b318216d [ Bugfix ] Fix Prometheus Metrics With zeromq Frontend (#7279) Robert ShawandNick Hill 2024-08-18 16:19:48 -04:00
• ab7165f2c7 [TPU] Optimize RoPE forward_native2 (#7636) Woosuk Kwon 2024-08-18 01:15:10 -07:00
• 0c2fa50b84 [TPU] Use mark_dynamic only for dummy run (#7634) Woosuk Kwon 2024-08-18 00:18:53 -07:00
• ce143353c6 [TPU] Skip creating empty tensor (#7630) Woosuk Kwon 2024-08-17 14:22:46 -07:00
• bbf55c4805 [VLM] Refactor MultiModalConfig initialization and profiling (#7530) Roger Wang 2024-08-17 13:30:55 -07:00
• 1ef13cf92f [Misc]Fix BitAndBytes exception messages (#7626) Jee Jee Li 2024-08-18 03:02:14 +08:00
• 832163b875 [ci][test] allow longer wait time for api server (#7629) youkaichao 2024-08-17 11:26:38 -07:00
• e73f76eec6 [Model] Pipeline parallel support for JAIS (#7603) Besher Alkurdi 2024-08-17 21:11:09 +03:00
• d95cc0a55c [core][misc] update libcudart finding (#7620) youkaichaoandcjackal 2024-08-16 23:01:35 -07:00
• 5bf45db7df [ci][test] fix engine/logger test (#7621) youkaichao 2024-08-16 23:00:59 -07:00