Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• dc5ce861bf [torch.compile] remove compilation_context and simplify code (#10838) youkaichao 2024-12-02 22:19:02 -08:00
• 21fe7b481a [core][distributed] add pynccl broadcast (#10843) youkaichao 2024-12-02 20:53:23 -08:00
• a4cf256159 [Bugfix] Fix QKVParallelLinearWithShardedLora bias bug (#10844) Jee Jee Li 2024-12-03 12:10:29 +08:00
• d746268e92 [Model] support bitsandbytes quantization with minicpm model (#10842) zixuanzhang226 2024-12-02 19:06:41 -08:00
• 4433195ab7 [Bugfix] Prevent benchmark_throughput.py from using duplicated random prompts (#10753) Michael Goin 2024-12-02 21:26:15 -05:00
• 4c05edb33a [Model] Add TP and BNB quantization support to LlavaMultiModalProjector (#10834) Isotr0pyandCyrus Leung 2024-12-03 07:06:09 +08:00
• 9b14d978aa Fix openvino on GPU (#10793) Jani Monoses 2024-12-02 20:52:19 +02:00
• 519cc6ca12 [Misc][XPU] Avoid torch compile for XPU platform (#10747) Yan Maandyoukaichao 2024-12-03 01:53:55 +08:00
• b45f0d7946 [Misc][LoRA] Move the implementation of lora bias to punica.py (#10829) Jee Jee Li 2024-12-03 01:53:36 +08:00
• a4c4daf364 [misc] use out argument for flash attention (#10822) youkaichao 2024-12-02 02:50:10 -08:00
• e95f275f57 [CI/Build] Update mistral_common version for tests and docs (#10825) Cyrus Leung 2024-12-02 18:26:10 +08:00
• ef31eabc68 [Model]: add some tests for aria model (#10770) zhou fanandIsotr0py 2024-12-02 13:36:36 +08:00
• 995a148575 [doc]Update config docstring (#10732) wangxiyuan 2024-12-02 12:14:45 +08:00
• 63a164172d [misc] remove xverse modeling file (#10814) youkaichao 2024-12-01 19:27:13 -08:00
• e25810ae29 Fill TorchSDPAAttentionMetadata seq_lens_field for prefill (#10799) Maximilien de Bayser 2024-12-01 23:05:32 -03:00
• 073a4bd1c0 [Kernel] Use out arg in flash_attn_varlen_func (#10811) Woosuk Kwon 2024-12-01 17:55:39 -08:00
• b7954776fd [core] Avoid metrics log noise when idle - include speculative decodi… (#10809) cduk 2024-12-02 02:49:48 +01:00
• b18c9bbaba [Model] Add BNB support to Llava and Pixtral-HF (#10795) Isotr0py 2024-12-02 09:31:09 +08:00
• 0590ec3fd9 [Core] Implement disagg prefill by StatelessProcessGroup (#10502) Kuntai Du 2024-12-01 19:01:00 -06:00
• c11f172187 [Misc] Adding MMMU-Pro vision dataset to serving benchmark (#10804) 3 people 2024-12-01 00:47:05 -08:00
• 169a0ff911 [doc] add warning about comparing hf and vllm outputs (#10805) youkaichao 2024-12-01 00:41:38 -08:00
• d2f058e76c [Misc] Rename embedding classes to pooling (#10801) Cyrus Leung 2024-12-01 14:36:51 +08:00
• f877a7d12a [Misc] Improve type annotations for support_torch_compile (#10763) Cyrus Leung 2024-12-01 09:48:35 +08:00
• 133707123e [Model] Replace embedding models with pooling adapter (#10769) Cyrus Leung 2024-12-01 08:02:54 +08:00
• 7e4bbda573 [doc] format fix (#10789) wangxiyuan 2024-11-30 19:38:40 +08:00
• e7cfc4ef4c [Interleaved ATTN] Support for Mistral-8B (#10591) Patrick von Platenandyoukaichao 2024-11-30 08:45:50 +01:00
• 16ee07f22a [Model] Refactor Molmo weights loading to use AutoWeightsLoader (#10771) Isotr0py 2024-11-30 12:19:14 +08:00
• 40bc242579 [Bugfix] Fix OpenVino/Neuron driver_worker init (#10779) Nicolò LucchesiandCyrus Leung 2024-11-30 05:07:13 +01:00
• 661175bc82 [platform] Add verify_quantization in platform. (#10757) wangxiyuan 2024-11-29 23:22:21 +08:00
• 3132aac043 [Bugfix] Fix Idefics3 bug (#10778) Jee Jee Li 2024-11-29 21:56:46 +08:00
• c82b432d4a [Misc] typo find in sampling_metadata.py (#10740) wang.yuqi 2024-11-29 13:17:57 +08:00
• fa6ecb9aa7 [Model] Clean up MiniCPMV (#10751) Cyrus Leung 2024-11-29 12:47:06 +08:00
• c83919c7a6 [Model] Add Internlm2 LoRA support (#5064) Isotr0py 2024-11-29 01:29:04 +08:00
• 98f47f2a40 [V1] Optimize the CPU overheads in FlashAttention custom op (#10733) Woosuk Kwon 2024-11-28 09:01:02 -08:00
• 8c1e77fb58 [Kernel] Update vllm-flash-attn version to reduce CPU overheads (#10742) Woosuk Kwon 2024-11-28 08:31:28 -08:00
• 5fc5ce0fe4 [Model] Added GLM-4 series hf format model support vllm==0.6.4 (#10561) 3 people 2024-11-28 22:53:31 +08:00
• 3ed5e73146 [TPU] Update requirements-tpu (#10726) Richard Liu 2024-11-28 02:30:48 -08:00
• 9a8bff0285 [Kernel] Update vllm-flash-attn version (#10736) Woosuk Kwon 2024-11-28 02:25:59 -08:00
• a79b122400 [V1] Do not allocate beyond the max_model_len (#10730) Woosuk Kwon 2024-11-28 00:13:15 -08:00
• d9b4b3f069 [Bug][CLI] Allow users to disable prefix caching explicitly (#10724) Ricky Xu 2024-11-27 23:59:28 -08:00
• 278be671a3 [Doc] Update model in arch_overview.rst to match comment (#10701) 罗泽轩 2024-11-28 15:58:39 +08:00
• 70dc14fbd0 [Model] support bitsandbytes quantization with minicpm3 model (#10682) zixuanzhang226 2024-11-27 23:58:02 -08:00
• cb4e1c3f3a [misc] upgrade filelock version (#10731) youkaichao 2024-11-27 19:54:58 -08:00
• 395b1c7454 [Frontend] don't block event loop in tokenization (preprocess) in OpenAI compatible server (#10635) tomeras91 2024-11-27 23:21:10 +02:00
• 9b4b150395 [Bugfix] Ignore lm_head when loading embedding models (#10719) Cyrus Leung 2024-11-28 03:05:29 +08:00
• 197b4484a3 [Bugfix][Mamba] Fix Multistep on Mamba-like models (#10705) Mor Zusman 2024-11-27 21:02:27 +02:00
• b98c62ba49 [Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (#10675) Isotr0py 2024-11-28 02:43:17 +08:00
• c411def234 [torch.compile] fix shape specialization (#10722) youkaichao 2024-11-27 10:16:10 -08:00
• 308cc5e21e [ci] fix slow tests (#10698) youkaichao 2024-11-27 09:26:14 -08:00
• 9e0a147d50 [V1] Update interface for mistral-format Pixtral (#10703) Roger Wang 2024-11-27 04:26:27 -08:00
• 418cb3b93f [Bugfix][Hardware][CPU] Fix intel-omp version to avoid segfault (#10700) Li, Jiang 2024-11-27 19:55:38 +08:00
• 1209261e93 [Model] Support telechat2 (#10311) 3 people 2024-11-27 19:32:35 +08:00
• e2251109c7 [Kernel] Remove if-else with identical branches in marlin 2:4 (#10687) Tyler Michael Smith 2024-11-27 01:55:32 -05:00
• 15cc2a9f1a [Misc]Further reduce BNB static variable (#10597) Jee Jee Li 2024-11-27 14:54:12 +08:00
• e85250b1d1 [Hardware][Gaudi]add get_name method for HPUAttentionBackend (#10667) Kunshang Ji 2024-11-27 14:49:40 +08:00
• cfb3bf25fb [bugfix] fix the default value of llm_int8_threshold in BitsAndBytesConfig (#10657) yansh97 2024-11-27 13:55:23 +08:00
• 1bf905ddaa [Bugfix][SpecDecode] apply sampling parameters to target probabilities for consistency in rejection sampling. (#10198) jeongin601 2024-11-27 14:07:30 +09:00
• 0a4d968500 [V1] Update interface for idefics3 (#10680) Roger Wang 2024-11-26 18:04:01 -08:00
• 0a71900bc9 Remove hard-dependencies of Speculative decode to CUDA workers (#10587) Chendi.Xue 2024-11-26 19:57:11 -06:00
• 2f0a0a17a4 [V1] Refactor model executable interface for multimodal models (#10570) Roger Wang 2024-11-26 12:46:11 -08:00
• 7576cd38df [Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (#10642) Michael Goin 2024-11-26 15:29:00 -05:00
• 9a99273b48 [Bugfix] Fix using -O[0,3] with LLM entrypoint (#10677) Michael Goin 2024-11-26 13:44:01 -05:00
• f5792c7c4a [Hardware][NVIDIA] Add non-NVML CUDA mode for Jetson (#9735) Conroy Cheers 2024-11-27 05:26:28 +11:00
• db66e018ea [Bugfix] Fix for Spec model TP + Chunked Prefill (#10232) Murali AndoorveeduandSourashis Roy 2024-11-26 09:11:16 -08:00
• 1f6584ee85 [V1] Enable profile for LLMEngine (#10665) Kunshang Ji 2024-11-26 18:36:45 +08:00
• 334d64d1e8 [ci] add vllm_test_utils (#10659) youkaichao 2024-11-26 00:20:04 -08:00
• 940635343a [Misc] Remove outdated init protocols (#10655) Cyrus Leung 2024-11-26 14:55:00 +08:00
• 9a88f89799 custom allreduce + torch.compile (#10121) Sage Mooreandyoukaichao 2024-11-26 00:00:16 -06:00
• 519e8e4182 [v1] EngineArgs for better config handling for v1 (#10382) Ricky Xu 2024-11-25 21:09:43 -08:00
• a6760f6456 [Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228) 3 people 2024-11-26 08:02:39 +05:30
• 45ac4ff270 [bugfix] fix aria model and add torch.compile (#10645) youkaichao 2024-11-25 18:32:09 -08:00
• 6e9ff050c8 [misc] do not read HOST_IP (#10644) youkaichao 2024-11-25 17:04:50 -08:00
• 9db713a1dc [Model] Add OLMo November 2024 model (#10503) Shane A 2024-11-25 14:26:40 -08:00
• 1b583cfefa [Doc] Fix typos in docs (#10636) Cyrus Leung 2024-11-26 02:15:45 +08:00
• cf73f0c95e [Model] Enable optional prefix when loading embedding models (#10639) Cyrus Leung 2024-11-26 02:14:33 +08:00
• b1d920531f [Model]: Add support for Aria model (#10514) zhou fanandIsotr0py 2024-11-26 02:10:55 +08:00
• 452a4e80c3 [Docs] Add Snowflake Slides (#10641) Simon Mo 2024-11-25 09:34:46 -08:00
• c27df94e1f [Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850) Wallas HenriqueandMichael Goin 2024-11-25 14:23:32 -03:00
• d04b13a380 [Bug]: Authorization ignored when root_path is set (#10606) Chauncey 2024-11-26 00:21:41 +08:00
• 2b0879bfc2 Super tiny little typo fix (#10633) fzyzcjy 2024-11-25 21:08:30 +08:00
• ed46f14321 [Model] Support is_causal HF config field for Qwen2 model (#10621) Cyrus Leung 2024-11-25 17:51:20 +08:00
• 05d1f8c9c6 [misc] move functions to config.py (#10624) youkaichao 2024-11-25 01:27:30 -08:00
• 25d806e953 [misc] add torch.compile compatibility check (#10618) youkaichao 2024-11-24 23:40:08 -08:00
• 65813781a2 [torch.compile] add warning for unsupported models (#10622) youkaichao 2024-11-24 23:27:51 -08:00
• 7c2134beda [torch.compile] force inductor threads (#10620) Jee Jee Li 2024-11-25 15:04:21 +08:00
• a30a605d21 [Doc] Add encoder-based models to Supported Models page (#10616) Cyrus Leung 2024-11-25 14:34:07 +08:00
• 571841b7fc [torch.compile] support encoder based models (#10613) youkaichao 2024-11-24 21:24:33 -08:00
• 7ea3cd7c3e [Refactor][MISC] del redundant code in ParallelConfig.postinit (#10614) Mengqing Cao 2024-11-25 13:14:56 +08:00
• 214efc2c3c Support Cross encoder models (#10400) Maximilien de BayserandFlavia Beo 2024-11-24 23:56:20 -03:00
• 49628fe13e [Doc] Update README.md with Ray Summit talk links (#10610) Zhuohan Li 2024-11-24 16:45:09 -08:00
• e4fbb14414 [doc] update the code to add models (#10603) youkaichaoandCyrus Leung 2024-11-24 11:21:40 -08:00
• c055747867 [model][utils] add extract_layer_index utility function (#10599) youkaichao 2024-11-23 22:22:54 -08:00
• eda2b3589c Revert "Print running script to enhance CI log readability" (#10601) youkaichao 2024-11-23 21:31:47 -08:00
• 1c445dca51 [CI/Build] Print running script to enhance CI log readability (#10594) Jee Jee Li 2024-11-24 11:57:13 +08:00
• 1700c543a5 [Bugfix] Fix LoRA weight sharding (#10450) Jee Jee LiandCyrus Leung 2024-11-24 09:23:17 +08:00
• 17d8fc1806 [bugfix] Fix example/tensorize_vllm_model tests (#10595) Jee Jee Li 2024-11-24 09:22:33 +08:00
• 04668ebe7a [Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) Isotr0py 2024-11-24 02:12:20 +08:00
• 651f6c31ac For ppc64le, disabled tests for now and addressed space issues (#10538) Nishidha 2024-11-23 15:03:53 +05:30
• 86a44fb896 [Platforms] Refactor openvino code (#10573) JiHuazhong 2024-11-23 14:23:12 +08:00
• 4cfe5d2bca [Bugfix] multi_modal_kwargs broadcast for CPU tensor parallel (#10541) Isotr0py 2024-11-23 13:25:46 +08:00