Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• ad9f1aa679 [doc] update wheels url (#11830) youkaichao 2025-01-08 14:36:49 +08:00
• 889e662eae [misc] improve memory profiling (#11809) youkaichaoandCyrus Leung 2025-01-08 14:36:03 +08:00
• ef68eb28d8 [Bug] Fix pickling of ModelConfig when RunAI Model Streamer is used (#11825) Cyrus Leung 2025-01-08 13:40:09 +08:00
• 259abd8953 [Docs] reorganize sponsorship page (#11639) Simon Mo 2025-01-07 21:16:08 -08:00
• f645eb6954 [Bugfix] Add checks for LoRA and CPU offload (#11810) Jee Jee Li 2025-01-08 13:08:48 +08:00
• f4923cb8bc [OpenVINO] Fixed Docker.openvino build (#11732) Ilya Lavrenov 2025-01-08 09:08:30 +04:00
• b640b19cc0 Fixed docker build for ppc64le (#11518) Nishidha 2025-01-08 10:35:37 +05:30
• dc71af0a71 Remove the duplicate imports of MultiModalKwargs and PlaceholderRange… (#11824) WangErXiao 2025-01-08 12:09:25 +08:00
• 4d29e91be8 [Misc] sort torch profiler table by kernel timing (#11813) Divakar Verma 2025-01-07 20:57:04 -06:00
• 91445c7bc8 [Bugfix] Fix image input for Pixtral-HF (#11741) Cyrus Leung 2025-01-08 10:17:16 +08:00
• 5950f555a1 [Doc] Group examples into categories (#11782) Harry Mellor 2025-01-08 01:20:12 +00:00
• a4e2b26856 [Bugfix] Significant performance drop on CPUs with --num-scheduler-steps > 1 (#11794) Jie Fu (傅杰) 2025-01-08 08:15:50 +08:00
• 973f5dc581 [Doc]Add documentation for using EAGLE in vLLM (#11417) sroy745 2025-01-07 11:19:12 -08:00
• c994223d56 [Bugfix] update the prefix for qwen2 (#11795) jiangjiadiandjiadi.jjd 2025-01-08 02:36:34 +08:00
• 869579a702 [optimization] remove python function call for custom op (#11750) youkaichao 2025-01-08 01:04:28 +08:00
• c0efe92d8b [Doc] Add note to gte-Qwen2 models (#11808) Cyrus Leung 2025-01-07 21:50:58 +08:00
• d9fa1c05ad [doc] update how pip can install nightly wheels (#11806) youkaichao 2025-01-07 21:42:58 +08:00
• 2de197bdd4 [V1] Support audio language models on V1 (#11733) Roger Wang 2025-01-07 03:47:36 -08:00
• 869e829b85 [doc] add doc to explain how to use uv (#11773) youkaichaoandCyrus Leung 2025-01-07 18:41:17 +08:00
• 8f37be38eb [Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calculation (#11800) Cyrus Leung 2025-01-07 18:25:02 +08:00
• 8082ad7950 [V1][Doc] Update V1 support for LLaVa-NeXT-Video (#11798) Roger Wang 2025-01-07 01:55:39 -08:00
• 1e4ce295ae [CI][CPU] adding build number to docker image name (#11788) Yuan 2025-01-07 15:28:01 +08:00
• ce1917fcf2 [Doc] Create a vulnerability management team (#9925) Russell Bryant 2025-01-07 01:57:32 -05:00
• e512f76a89 fix init error for MessageQueue when n_local_reader is zero (#11768) XiaobingZhang 2025-01-07 14:12:48 +08:00
• 898cdf033e [CI] Fix neuron CI and run offline tests (#11779) Liangfu Chen 2025-01-06 21:36:10 -08:00
• 0f3f3c86ec [Bugfix] Update attention interface in Whisper (#11784) Roger Wang 2025-01-06 20:36:24 -08:00
• b278557935 [Kernel][LoRA]Punica prefill kernels fusion (#11234) Jee Jee LiandZhonghua Deng 2025-01-07 12:01:39 +08:00
• 8ceffbf315 [Doc][3/N] Reorganize Serving section (#11766) Cyrus Leung 2025-01-07 11:20:01 +08:00
• d93d2d74fd [XPU] Make pp group initilized for pipeline-parallelism (#11648) YiSheng5 2025-01-07 11:09:58 +08:00
• d0169e1b0f [Model] Future-proof Qwen2-Audio multi-modal processor (#11776) Cyrus Leung 2025-01-07 11:05:17 +08:00
• 08fb75c72e [Bugfix] Fix LLaVA-NeXT feature size precision error (for real) (#11772) Cyrus Leung 2025-01-07 09:10:54 +08:00
• 91b361ae89 [V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (#11685) Roger WangandDarkLight1337 2025-01-06 11:58:16 -08:00
• e20c92bb61 [Kernel] Move attn_type to Attention.__init__() (#11690) Chen Zhang 2025-01-07 00:11:28 +08:00
• 32c9eff2ff [Bugfix][V1] Fix molmo text-only inputs (#11676) Jee Jee Li 2025-01-06 23:22:25 +08:00
• 4ca5d40adc [doc] explain how to add interleaving sliding window support (#11771) youkaichao 2025-01-06 21:57:44 +08:00
• 9279b9f83d [Bugfix] Fix max image size for LLaVA-Onevision (#11769) Roger Wang 2025-01-06 05:48:53 -08:00
• ee77fdb5de [Doc][2/N] Reorganize Models and Usage sections (#11755) Cyrus Leung 2025-01-06 21:40:31 +08:00
• 996357e480 [VLM] Separate out profiling-related logic (#11746) Cyrus Leung 2025-01-06 16:02:21 +08:00
• 2a622d704a k8s-config: Update the secret to use stringData (#11679) Suraj Deshmukh 2025-01-06 00:01:22 -08:00
• 9c749713f6 [mypy] Forward pass function type hints in lora (#11740) Lucas Tuckerandlucast2021 2025-01-06 01:59:36 -06:00
• 022c5c6944 [V1] Refactor get_executor_cls (#11754) Rui Qiao 2025-01-05 23:59:16 -08:00
• f8fcca100b [Misc] Fix typo for valid_tool_parses (#11753) Rui Qiao 2025-01-05 23:12:38 -08:00
• 06bfb51963 [V1] Add BlockTable class (#11693) Woosuk Kwon 2025-01-06 14:24:42 +09:00
• 408e560015 [Bugfix] Remove block size constraint (#11723) Cody Yu 2025-01-05 20:49:55 -08:00
• 402d378360 [Doc] [1/N] Reorganize Getting Started section (#11645) Cyrus Leung 2025-01-06 10:18:33 +08:00
• 9e764e7b10 [distributed] remove pynccl's redundant change_state (#11749) cennn 2025-01-06 09:05:48 +08:00
• 33fc1e2e86 [Frontend] Improve StreamingResponse Exception Handling (#11752) Robert Shaw 2025-01-05 16:35:01 -05:00
• eba17173d3 fix: [doc] fix typo (#11751) LancerandLancer 2025-01-06 00:48:16 +08:00
• 635b897246 [distributed] remove pynccl's redundant stream (#11744) cennn 2025-01-05 23:09:11 +08:00
• 4068f4b5b5 [MISC] Replace c10::optional with std::optional (#11730) Lu Fang 2025-01-04 17:20:34 -08:00
• 47831430cc [Bugfix][V1] Fix test_kv_cache_utils.py (#11738) Jee Jee Li 2025-01-05 00:07:59 +08:00
• 65c08928c2 [Model] Remove unnecessary weight initialization logic (#11736) Cyrus LeungandIsotr0py 2025-01-04 23:46:21 +08:00
• ba214dffbe [Bugfix] Fix precision error in LLaVA-NeXT (#11735) Cyrus Leung 2025-01-04 23:45:57 +08:00
• eed11ebee9 [VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-OneVision (#11717) Cyrus Leung 2025-01-04 19:40:53 +08:00
• 300acb8347 [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233) Yan Burman 2025-01-04 08:50:16 +02:00
• d91457d529 [V1] Add kv cache utils tests. (#11513) xcnick 2025-01-04 14:49:46 +08:00
• fbf2564554 [V1] Add RayExecutor support for AsyncLLM (api server) (#11712) Kunshang Ji 2025-01-04 14:41:31 +08:00
• d1d49397e7 Update bnb.md with example for OpenAI (#11718) Alberto Ferrer 2025-01-04 00:29:02 -06:00
• 9c93636d84 Update tool_calling.md (#11701) Hust_YangXian 2025-01-04 14:16:30 +08:00
• e5d7ed0c53 [V1] log GPU blocks num for MultiprocExecutor (#11656) WangErXiao 2025-01-04 08:13:12 +08:00
• ad0d567e1c [V1] Chore: cruft removal (#11724) Robert Shaw 2025-01-03 18:25:02 -05:00
• bf0d97d786 Update requirements-tpu.txt to support python 3.9 and 3.11 (#11695) Michael Goin 2025-01-03 17:36:46 -05:00
• a655eb3025 [Misc]Add BNB quantization for Qwen2VL (#11719) Jee Jee LiandIsotr0py 2025-01-04 06:19:02 +08:00
• 1543914c04 [V1] Improve TP>1 Error Handling + Stack Trace (#11721) Robert ShawandTyler Michael Smith 2025-01-03 16:29:11 -05:00
• 61fed92c7e [Bugfix] Fix ColumnParallelLinearWithLoRA slice (#11708) ZincCat 2025-01-03 13:02:34 -08:00
• 80c751e7f6 [V1] Simplify Shutdown (#11659) Robert Shaw 2025-01-03 12:25:38 -05:00
• e1a5c2f0a1 [Model] Whisper model implementation (#11280) Aurick QiaoandAurick Qiao 2025-01-03 03:39:19 -05:00
• fd3a62a122 [perf-benchmark] Fix dependency for steps in benchmark pipeline (#11710) Kevin H. Luu 2025-01-03 13:38:37 +07:00
• 07064cb1d4 [Bugfix] Check chain_speculative_sampling before calling it (#11673) Lu Fang 2025-01-02 16:58:56 -08:00
• 2f1e8e8f54 Update default max_num_batch_tokens for chunked prefill (#11694) Sachin Varghese 2025-01-02 19:25:53 -05:00
• 68d37809b9 [Misc] Minimum requirements for SageMaker compatibility (#11576) Nathan Azrak 2025-01-03 10:59:25 +11:00
• 5dba257506 Resolve race conditions in Marlin kernel (#11493) wchen61 2025-01-03 06:58:56 +08:00
• 187e32997c [Bugfix] Change kv scaling factor by param json on nvidia gpu (#11688) bjmsongandbjmsong 2025-01-03 05:11:39 +08:00
• b55ed6ef8a [V1][Minor] Optimize token_ids_cpu copy (#11692) Woosuk Kwon 2025-01-03 04:04:58 +09:00
• 2f385183f3 [Bugfix] Free cross attention block table for preempted-for-recompute sequence group. (#10013) Kathy Yu 2025-01-02 10:28:09 -08:00
• 84c35c374a According to vllm.EngineArgs, the name should be distributed_executor_backend (#11689) Chunyang Wen 2025-01-03 02:14:16 +08:00
• 8c38ee7007 [VLM] Merged multi-modal processor for LLaVA-NeXT (#11682) Cyrus Leung 2025-01-03 00:39:27 +08:00
• b6087a6bee [mypy] Pass type checking in vllm/inputs (#11680) Tobias Pitters 2025-01-02 17:18:15 +01:00
• 23c1b10a4c [VLM][Bugfix] Multi-modal processor compatible with V1 multi-input (#11674) Cyrus Leung 2025-01-02 17:00:00 +08:00
• a115ac46b5 [VLM] Move supported limits and max tokens to merged multi-modal processor (#11669) Cyrus LeungandIsotr0py 2025-01-01 23:44:42 +08:00
• 73001445fb [V1] Implement Cascade Attention (#11635) Woosuk Kwon 2025-01-01 21:56:46 +09:00
• 6d70198b17 [Doc] Fix typo (#11666) Kazuhiro Serizawa 2025-01-01 17:10:10 +09:00
• f962f426bc [Misc] Replace space with - in the file names (#11667) Lu Fang 2024-12-31 23:39:30 -08:00
• 11d8a091c6 [Misc] Optimize Qwen2-VL LoRA test (#11663) Jee Jee Li 2025-01-01 14:42:23 +08:00
• 365801fedd [VLM] Add max-count checking in data parser for single image models (#11661) Cyrus LeungandRoger Wang 2025-01-01 14:15:21 +08:00
• 4db72e57f6 [Bugfix][Refactor] Unify model management in frontend (#11660) Joe Runde 2024-12-31 18:21:51 -08:00
• 0c6f998554 [Benchmark] Add benchmark script for CPU offloading (#11533) Yihua ChengandKuntaiDu 2024-12-31 18:10:55 -06:00
• e7c7c5e822 [V1][VLM] V1 support for selected single-image models. (#11632) 3 people 2024-12-31 13:17:22 -08:00
• 8c3230d8c1 [V1] Simpify vision block hash for prefix caching by removing offset from hash (#11646) Chen Zhang 2024-12-31 16:56:01 +08:00
• 2c5718809b [Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. (#11565) sakunkun 2024-12-31 14:29:04 +08:00
• 82c49d3260 [Misc][LoRA] Support Rank Stabilized LoRA (RSLoRA) (#6909) John GiorgiandJee Jee Li 2024-12-31 01:15:58 -05:00
• 74fa1d123c [Bugfix] Fix OpenAI parallel sampling when using xgrammar (#11637) Michael Goin 2024-12-30 22:43:54 -05:00
• a2a40bcd0d [Model][LoRA]LoRA support added for MolmoForCausalLM (#11439) 3 people 2024-12-31 02:33:06 +01:00
• ccb1aabcca [benchmark] Remove dependency for H100 benchmark step (#11572) Kevin H. Luu 2024-12-30 12:27:07 -08:00
• 36e7670045 [Bugfix] Validate and concatenate image embeddings in MiniCPMVBaseModel (#11631) whyiug 2024-12-31 02:51:04 +08:00
• 5886aa496e [V1] [6/N] API Server: Better Shutdown (#11586) Robert Shaw 2024-12-30 10:51:02 -05:00
• 8d9b6721e7 [VLM] Abstract out multi-modal data parsing in merged processor (#11620) Cyrus Leung 2024-12-30 23:01:35 +08:00
• b12e87f942 [platforms] enable platform plugins (#11602) youkaichao 2024-12-30 20:24:45 +08:00
• 5dbf854553 [CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels (#11618) Li, Jiang 2024-12-30 18:17:04 +08:00
• 970d6d0776 [Build][Kernel] Update CUTLASS to v3.6.0 (#11607) Tyler Michael Smith 2024-12-30 04:22:13 -05:00