Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 4f341bd4bf [Doc] Update list of supported models (#8987) Cyrus Leung 2024-10-02 00:35:39 +08:00
• 35bd215168 [Core] [Frontend] Priority scheduling for embeddings and in the OpenAI-API (#8965) Sebastian Schoennenbeck 2024-10-01 11:58:06 +02:00
• 1fe0a4264a [Bugfix] Fix Token IDs Reference for MiniCPM-V When Images are Provided With No Placeholders (#8991) Alex Brooks 2024-10-01 03:52:44 -06:00
• bc4eb65b54 [Bugfix] Fix Fuyu tensor parallel inference (#8986) Isotr0py 2024-10-01 17:51:41 +08:00
• 82f3937e59 [Misc] add process_weights_after_loading for DummyLoader (#8969) Divakar Verma 2024-09-30 22:46:41 -05:00
• 7da2487591 [torch.compile] fix tensor alias (#8982) youkaichao 2024-09-30 20:40:48 -07:00
• aaccca2b4d [CI/Build] Fix machete generated kernel files ordering (#8976) Kevin H. LuuandCody Yu 2024-09-30 20:33:12 -07:00
• 062c89e7c9 [Frontend][Core] Move guided decoding params into sampling params (#8252) Joe RundeandNick Hill 2024-09-30 19:34:25 -06:00
• bce324487a [CI][SpecDecode] Fix spec decode tests, use flash attention backend for spec decode CI tests. (#8975) Lily Liu 2024-09-30 17:51:40 -07:00
• 1425a1bcf9 [ci] Add CODEOWNERS for test directories (#8795) Kevin H. Luu 2024-09-30 17:47:08 -07:00
• 1cabfcefb6 [Misc] Adjust max_position_embeddings for LoRA compatibility (#8957) Jee Jee Li 2024-09-30 20:57:39 +08:00
• be76e5aabf [Core] Make scheduling policy settable via EngineArgs (#8956) Sebastian Schoennenbeck 2024-09-30 14:28:44 +02:00
• 2ae25f79cf [Model] Expose InternVL2 max_dynamic_patch as a mm_processor_kwarg (#8946) Isotr0py 2024-09-30 13:01:20 +08:00
• 8e60afa15e [Model][LoRA]LoRA support added for MiniCPMV2.6 (#8943) Jee Jee LiandDarkLight1337 2024-09-30 12:31:55 +08:00
• b6d7392579 [Misc][CI/Build] Include cv2 via mistral_common[opencv] (#8951) Roger Wang 2024-09-29 21:28:26 -07:00
• e01ab595d8 [Model] support input embeddings for qwen2vl (#8856) whyiug 2024-09-30 11:16:10 +08:00
• f13a07b1f8 [Kernel][Model] Varlen prefill + Prefill chunking support for mamba kernels and Jamba model (#8533) Mor Zusman 2024-09-30 00:35:58 +03:00
• 6c9ba48fde [Frontend] Added support for HF's new continue_final_message parameter (#8942) danieljannai21 2024-09-29 20:59:47 +03:00
• 1fb9c1b0bf [Misc] Fix typo in BlockSpaceManagerV1 (#8944) juncheoll 2024-09-30 00:05:54 +09:00
• 31f46a0d35 [BugFix] Fix seeded random sampling with encoder-decoder models (#8870) Nick HillandRoger Wang 2024-09-29 10:43:14 +01:00
• 3d49776bbb [Model][LoRA]LoRA support added for MiniCPMV2.5 (#7199) Jee Jee Li 2024-09-29 14:59:45 +08:00
• bc2ef1f77c [Model] Support Qwen2.5-Math-RM-72B (#8896) Zilin Zhu 2024-09-29 12:19:39 +08:00
• 2e7fe7e79f [Build/CI] Set FETCHCONTENT_BASE_DIR to one location for better caching (#8930) Tyler Michael Smith 2024-09-28 23:13:01 -04:00
• 26a68d5d7e [CI/Build] Add test decorator for minimum GPU memory (#8925) Cyrus Leung 2024-09-29 10:50:51 +08:00
• d081da0064 [Bugfix] Fix Marlin MoE act order when is_k_full == False (#8741) ElizaWszolaandTyler Michael Smith 2024-09-29 03:19:40 +02:00
• 5bf8789b2a [Bugfix] Block manager v2 with preemption and lookahead slots (#8824) sroy745 2024-09-28 18:17:45 -07:00
• d1537039ce [Core] Improve choice of Python multiprocessing method (#8823) Russell Bryantandyoukaichao 2024-09-28 21:17:07 -04:00
• cc276443b5 [doc] organize installation doc and expose per-commit docker (#8931) youkaichao 2024-09-28 17:48:41 -07:00
• e585b583a9 [Bugfix] Support testing prefill throughput with benchmark_serving.py --hf-output-len 1 (#8891) Chen Zhang 2024-09-28 11:51:22 -07:00
• 090e945e36 [Frontend] Make beam search emulator temperature modifiable (#8928) Edouard B.andEduard Balzin 2024-09-28 20:30:21 +02:00
• e1a3f5e831 [CI/Build] Update models tests & examples (#8874) Cyrus LeungandRoger Wang 2024-09-29 00:54:35 +08:00
• 19d02ff938 [Bugfix] Fix PP for Multi-Step (#8887) Varun Sundar Rabindranath 2024-09-28 11:52:46 -04:00
• 39d3f8d94f [Bugfix] Fix code for downloading models from modelscope (#8443) tastelikefeet 2024-09-28 23:24:12 +08:00
• b0298aa8cc [Misc] Remove vLLM patch of BaichuanTokenizer (#8921) Cyrus Leung 2024-09-28 16:11:25 +08:00
• 260024a374 [Bugfix][Intel] Fix XPU Dockerfile Build (#7824) Tyler Titsworthandyoukaichao 2024-09-27 23:45:50 -07:00
• d86f6b2afb [misc] fix wheel name (#8919) youkaichao 2024-09-27 22:10:44 -07:00
• bd429f2b75 [Core] Priority-based scheduling in async engine (#8850) Sebastian Schoennenbeck 2024-09-28 00:07:10 +02:00
• 18e60d7d13 [misc][distributed] add VLLM_SKIP_P2P_CHECK flag (#8911) youkaichao 2024-09-27 14:27:56 -07:00
• c2ec430ab5 [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-09-27 16:32:07 -04:00
• c5d55356f9 [Bugfix] fix for deepseek w4a16 (#8906) Lucas Wilkinsonandmgoin 2024-09-27 15:12:34 -04:00
• 172d1cd276 [Kernel] AQ AZP 4/4: Integrate asymmetric quantization to linear method (#7271) Luka Govedič 2024-09-27 14:25:10 -04:00
• a9b15c606f [torch.compile] use empty tensor instead of None for profiling (#8875) youkaichao 2024-09-27 08:11:32 -07:00
• 8df2dc3c88 [TPU] Update pallas.py to support trillium (#8871) Brittany 2024-09-27 01:16:55 -07:00
• 6d792d2f31 [Bugfix][VLM] Fix Fuyu batching inference with max_num_seqs>1 (#8892) Isotr0py 2024-09-27 16:15:58 +08:00
• 0e088750af [MISC] Fix invalid escape sequence '\' (#8830) Peter Pan 2024-09-27 16:13:25 +08:00
• dc4e3df5c2 [misc] fix collect env (#8894) youkaichao 2024-09-27 00:26:38 -07:00
• 3b00b9c26c [Core] renamePromptInputs and inputs (#8876) Cyrus Leung 2024-09-27 11:35:15 +08:00
• 344cd2b6f4 [Feature] Add support for Llama 3.1 and 3.2 tool use (#8343) Maximilien de Bayser 2024-09-26 21:01:42 -03:00
• 1b49148e47 [Installation] Allow lower versions of FastAPI to maintain Ray 2.9 compatibility (#8764) Cyrus Leung 2024-09-27 07:54:09 +08:00
• 4b377d6feb [BugFix] Fix test breakages from transformers 4.45 upgrade (#8829) Nick Hill 2024-09-27 00:46:43 +01:00
• 71d21c73ab [Bugfix] Fixup advance_step.cu warning (#8815) Tyler Michael Smith 2024-09-26 19:23:45 -04:00
• ee2da3e9ef fix validation: Only set tool_choice auto if at least one tool is provided (#8568) Chirag Jain 2024-09-27 04:53:17 +05:30
• e2f6f26e86 [Bugfix] Fix print_warning_once's line info (#8867) Tyler Michael Smith 2024-09-26 19:18:26 -04:00
• b28d2104de [Misc] Change dummy profiling and BOS fallback warns to log once (#8820) Michael Goin 2024-09-26 19:18:14 -04:00
• 93d364da34 [Bugfix] Include encoder prompts len to non-stream api usage response (#8861) Pernekhan Utemuratov 2024-09-26 15:47:00 -07:00
• d9cfbc891e [ci] Soft fail Entrypoints, Samplers, LoRA, Decoder-only VLM (#8872) Kevin H. Luu 2024-09-26 15:02:16 -07:00
• 70de39f6b4 [misc][installation] build from source without compilation (#8818) youkaichao 2024-09-26 13:19:04 -07:00
• 68988d4e0d [CI/Build] Fix missing ci dependencies (#8834) fyuan1316 2024-09-27 02:04:39 +08:00
• 520db4dbc1 [Docs] Add README to the build docker image (#8825) Michael Goin 2024-09-26 14:02:52 -04:00
• f70bccac75 [Build/CI] Upgrade to gcc 10 in the base build Docker image (#8814) Tyler Michael Smith 2024-09-26 13:07:18 -04:00
• 4bb98f2190 [Misc] Update config loading for Qwen2-VL and remove Granite (#8837) Roger Wang 2024-09-26 07:45:30 -07:00
• 7193774b1f [Misc] Support quantization of MllamaForCausalLM (#8822) Michael Goin 2024-09-25 17:46:22 -04:00
• e2c6e0a829 [Doc] Update doc for Transformers 4.45 (#8817) Roger Wang 2024-09-25 13:29:48 -07:00
• 770ec6024f [Model] Add support for the multi-modal Llama 3.2 model (#8811) 6 people 2024-09-25 13:29:32 -07:00
• 4f1ba0844b Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) Simon Mo 2024-09-25 10:36:26 -07:00
• 873edda6cf [Misc] Support FP8 MoE for compressed-tensors (#8588) Michael Goin 2024-09-25 12:43:36 -04:00
• 64840dfae4 [Frontend] MQLLMEngine supports profiling. (#8761) 科英 2024-09-26 00:37:41 +08:00
• 28e1299e60 rename PromptInputs and inputs with backward compatibility (#8760) Cyrus Leung 2024-09-26 00:36:47 +08:00
• 0c4d2ad5e6 [VLM][Bugfix] internvl with num_scheduler_steps > 1 (#8614) DefTruth 2024-09-26 00:35:53 +08:00
• c6f2485c82 [[Misc]] Add extra deps for openai server image (#8792) Jee Jee Li 2024-09-26 00:35:23 +08:00
• 300da09177 [Kernel] Fullgraph and opcheck tests (#8479) bnellnm 2024-09-25 10:35:52 -04:00
• 1c046447a6 [CI/Build][Bugfix][Doc][ROCm] CI fix and doc update after ROCm 6.2 upgrade (#8777) Hongxia Yang 2024-09-25 10:26:37 -04:00
• 8fae5ed7f6 [Misc] Fix minor typo in scheduler (#8765) Woo-Yeon Lee 2024-09-25 16:53:03 +09:00
• 3368c3ab36 [Bugfix] Ray 2.9.x doesn't expose available_resources_per_node (#8767) David Newman 2024-09-25 17:52:26 +10:00
• 1ac3de09cd [Frontend] OpenAI server: propagate usage accounting to FastAPI middleware layer (#8672) Adam Tilghman 2024-09-25 00:49:26 -07:00
• 3e073e66f1 [Bugfix] load fc bias from config for eagle (#8790) sohamparikh 2024-09-25 02:16:30 -04:00
• c23953675f [Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (#8770) Isotr0py 2024-09-25 14:16:11 +08:00
• e3dd0692fa [BugFix] Propagate 'trust_remote_code' setting in internvl and minicpmv (#8250) zifeitong 2024-09-24 22:53:43 -07:00
• fc3afc20df Fix tests in test_chunked_prefill_scheduler which fail with BlockManager V2 (#8752) sroy745 2024-09-24 21:26:36 -07:00
• b4522474a3 [Bugfix][Kernel] Implement acquire/release polyfill for Pascal (#8776) sasha0552 2024-09-25 04:26:33 +00:00
• ee777d9c30 Fix test_schedule_swapped_simple in test_scheduler.py (#8780) sroy745 2024-09-24 21:26:18 -07:00
• 6e0c9d6bd0 [Bugfix] Use heartbeats instead of health checks (#8583) Joe Runde 2024-09-24 21:37:38 -06:00
• 6da1ab6b41 [Core] Adding Priority Scheduling (#5958) Archit Patke 2024-09-24 21:50:50 -05:00
• 01b6f9e1f0 [Core][Bugfix] Support prompt_logprobs returned with speculative decoding (#8047) Travis Johnson 2024-09-24 18:29:56 -06:00
• 13f9f7a3d0 [[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (#8768) Jee Jee Li 2024-09-25 08:08:55 +08:00
• 1e7d5c01f5 [misc] soft drop beam search (#8763) youkaichao 2024-09-24 15:48:39 -07:00
• 2467b642dd [CI/Build] fix setuptools-scm usage (#8771) Daniele 2024-09-24 21:38:12 +02:00
• 72fc97a0f1 [Bugfix] Fix torch dynamo fixes caused by replace_parameters (#8748) Lucas Wilkinson 2024-09-24 14:33:21 -04:00
• 2529d09b5a [Frontend] Batch inference for llm.chat() API (#8648) 5 people 2024-09-24 12:44:11 -04:00
• a928ded995 [Kernel] Split Marlin MoE kernels into multiple files (#8661) ElizaWszolaandmgoin 2024-09-24 18:31:42 +02:00
• cc4325b66a [Bugfix] Fix potentially unsafe custom allreduce synchronization (#8558) Hanzhi Zhou 2024-09-24 01:08:14 -07:00
• 8ff7ced996 [Model] Expose Phi3v num_crops as a mm_processor_kwarg (#8658) 3 people 2024-09-24 01:36:46 -06:00
• 3f06bae907 [Core][Model] Support loading weights by ID within models (#7931) Peter Salas 2024-09-24 00:14:15 -07:00
• b8747e8a7c [MISC] Skip dumping inputs when unpicklable (#8744) Cody Yu 2024-09-23 23:10:03 -07:00
• 3185fb0cca Revert "[Core] Rename PromptInputs to PromptType, and inputs to prompt" (#8750) Simon Mo 2024-09-23 22:45:20 -07:00
• 0250dd68c5 re-implement beam search on top of vllm core (#8726) youkaichaoandBrendan Wong 2024-09-23 22:08:12 -07:00
• 88577ac928 Fix tests in test_scheduler.py that fail with BlockManager V2 (#8728) sroy745 2024-09-23 21:43:13 -07:00
• 530821d00c [Hardware][AMD] ROCm6.2 upgrade (#8674) Hongxia Yang 2024-09-23 21:52:39 -04:00
• 1a2aef3e59 Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335) Alexander Matveev 2024-09-23 18:38:04 -04:00
• 5f7bb58427 Fix typical acceptance sampler with correct recovered token ids (#8562) jiqing-feng 2024-09-24 03:32:27 +08:00