Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 09972e716c [Bugfix] Allow fallback to AWQ from AWQMarlin at per-layer granularity (#13119) Michael Goin 2025-02-12 12:19:53 -05:00
• 36a08630e8 [CORE] [QUANT] Support for GPTQModel's dynamic quantization per module override/control (#7086) Qubitium-ModelCloud 2025-02-13 01:19:43 +08:00
• 2c2b560f48 [CI/Build] Use mypy matcher for pre-commit CI job (#13162) Russell Bryant 2025-02-12 12:12:22 -05:00
• 042c3419fa Introduce VLLM_CUDART_SO_PATH to allow users specify the .so path (#12998) Lu Fang 2025-02-12 09:06:13 -08:00
• 82cabf53a3 [Misc] Delete unused LoRA modules (#13151) Jee Jee Li 2025-02-13 00:58:24 +08:00
• 314cfade02 [Frontend] Generate valid tool call IDs when using tokenizer-mode=mistral (#12332) Rafael Vasquez 2025-02-12 11:29:56 -05:00
• 985b4a2b19 [Bugfix] Fix num video tokens calculation for Qwen2-VL (#13148) Cyrus Leung 2025-02-12 19:55:23 +08:00
• f4d97e4fc2 [Bug] [V1] Try fetching stop_reason from EngineOutput before checking the request (#13108) bnellnm 2025-02-12 05:39:16 -05:00
• f1042e86f0 [Misc] AMD Build Improvements (#12923) Shiyan Deng 2025-02-12 02:36:10 -08:00
• 7c4033acd4 Further reduce the HTTP calls to huggingface.co (#13107) Maximilien de Bayser 2025-02-12 07:34:09 -03:00
• d59def4730 Bump actions/setup-python from 5.3.0 to 5.4.0 (#12672) dependabot[bot] 2025-02-12 16:41:22 +08:00
• 0c7d9effce Bump helm/chart-testing-action from 2.6.1 to 2.7.0 (#12463) dependabot[bot] 2025-02-12 16:41:06 +08:00
• dd3b4a01f8 Bump actions/stale from 9.0.0 to 9.1.0 (#12462) dependabot[bot] 2025-02-12 00:40:25 -08:00
• a0597c6b75 Bump helm/kind-action from 1.10.0 to 1.12.0 (#11612) dependabot[bot] 2025-02-12 00:40:19 -08:00
• e92694b6fe [Neuron][Kernel] Support Longer Sequences in NKI-based Flash PagedAttention and Improve Efficiency (#12921) Lingfan Yu 2025-02-11 21:12:37 -08:00
• 842b0fd402 [ci] Add more source file dependencies for some tests (#13123) Kevin H. LuuandEC2 Default User 2025-02-11 20:38:10 -08:00
• 974dfd4971 [Model] IBM/NASA Prithvi Geospatial model (#12830) Christian Pinto 2025-02-12 04:34:30 +00:00
• 3ee696a63d [RFC][vllm-API] Support tokenizer registry for customized tokenizer in vLLM (#12518) Keyun Tong 2025-02-11 20:25:58 -08:00
• 72c2b68dc9 [Misc] Move pre-commit suggestion back to the end (#13114) Russell Bryant 2025-02-11 17:34:16 -05:00
• 14ecab5be2 [Bugfix] Guided decoding falls back to outlines when fails to import xgrammar (#12976) Yuan Tang 2025-02-11 13:17:44 -05:00
• deb6c1c6b4 [Doc] Improve OpenVINO installation doc (#13102) Harry Mellor 2025-02-11 18:02:46 +00:00
• 565c1efa65 [CI/Build][Bugfix] Fix CPU backend default threads num (#13077) Li, Jiang 2025-02-12 00:55:56 +08:00
• 2b25b7d2e1 Fix initializing GGUF weights for ColumnParallelLinear when using tensor parallel > 1 (#13023) Szymon Ożóg 2025-02-11 17:38:48 +01:00
• 6c4dbe23eb [BugFix] Pop instead of del CUDA_VISIBLE_DEVICES (#12962) ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 2025-02-11 18:21:50 +02:00
• 21f5d50fa5 [Bugfix] Do not use resource module on Windows (#12858) (#13029) MoonRide303 2025-02-11 17:21:18 +01:00
• bf3e05215c [Misc] Fix typo at comments at metrics.py (#13024) Jewon Lee 2025-02-12 01:20:37 +09:00
• ad9776353e Set torch_dtype in TransformersModel (#13088) Harry Mellor 2025-02-11 15:51:19 +00:00
• 75e6e14516 [V1][Metrics] Add several request timing histograms (#12644) Mark McLoughlin 2025-02-11 15:14:00 +00:00
• 110f59a33e [Bugfix] fix flaky test (#13089) மனோஜ்குமார் பழனிச்சாமி 2025-02-11 20:11:20 +05:30
• 2e3b969ec0 [Platform] add pre_register_and_update function (#12432) wangxiyuan 2025-02-11 22:06:46 +08:00
• da317197dd [Build] Fix cuda link target of cumem_allocator in CPU env (#12863) Yuhong GuoandTyler Michael Smith 2025-02-11 21:55:57 +08:00
• 7539bbc6a6 [ROCm] Using a more precise memory profiling (#12624) Gregory Shtrasberg 2025-02-11 08:47:10 -05:00
• 9cf4759493 [executor] init local_rank as device index (#13027) Mengqing Cao 2025-02-11 21:20:53 +08:00
• 41c5dd45b9 [V1][Metrics] Add GPU prefix cache hit rate % gauge (#12592) Cody Yu 2025-02-11 00:27:25 -08:00
• fc6485d277 [Bugfix]: Reasoning output bug according to the chat template change (#13025) Ce Gao 2025-02-11 15:49:03 +08:00
• 78a141d768 [Misc] LoRA - Refactor Punica ops tests (#12970) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-02-11 12:56:03 +05:30
• c320ca8edd [Core] Don't do platform detection at import time (#12933) Russell Bryant 2025-02-11 02:25:25 -05:00
• 58047c6f04 [Benchmark] Add BurstGPT to benchmark_serving (#13063) Woosuk KwonandRoger Wang 2025-02-10 21:25:30 -08:00
• cb080f32e3 [Bugfix] Support missing tool parameters in mistral tokenizer (#12884) Florian Greinacher 2025-02-11 04:33:33 +01:00
• 2c0f58203c [Docs] Annouce Meta Meetup (#13065) Simon Mo 2025-02-10 18:24:29 -08:00
• 2ff4857678 [V1][Minor] Move scheduler outputs to a separate file (#13062) Woosuk Kwon 2025-02-10 18:10:06 -08:00
• 91e876750e [misc] Fix setup.py condition to avoid AMD from being mistaken with CPU (#13022) Kevin H. Luu 2025-02-10 18:06:16 -08:00
• 08b2d845d6 [Model] Ultravox Model: Support v0.5 Release (#12912) Farzad Abdolhosseini 2025-02-10 14:02:48 -08:00
• 2ae889052c Fix seed parameter behavior in vLLM (#13007) மனோஜ்குமார் பழனிச்சாமி 2025-02-10 20:56:50 +05:30
• 51f0b5f7f6 [Bugfix] Clean up and fix multi-modal processors (#13012) Cyrus Leung 2025-02-10 18:45:21 +08:00
• fde71262e0 [misc] Add retries with exponential backoff for HF file existence check (#13008) Kevin H. Luu 2025-02-10 01:15:02 -08:00
• 243137143c [Doc] Add link to tool_choice tracking issue in tool_calling.md (#13003) Yuan Tang 2025-02-10 01:09:33 -05:00
• b2496bb07f [core] fix sleep mode and pytorch checkpoint compatibility (#13001) youkaichao 2025-02-10 13:03:43 +08:00
• 44607e07d3 Check if selected backend is None in get_attn_backend_cls() (#12975) Yuan Tang 2025-02-09 22:45:07 -05:00
• 67c4637ccf [V1] Use msgpack for core request serialization (#12918) Nick Hill 2025-02-09 19:35:56 -08:00
• aa0ca5ebb7 [core][rlhf] add colocate example for RLHF (#12984) youkaichao 2025-02-10 10:28:59 +08:00
• 59fff4a01a [core] improve error handling when wake up from sleep mode (#12981) youkaichao 2025-02-10 09:38:57 +08:00
• 29f1d47e73 [MISC] Always import version library first in the vllm package (#12979) Lu Fang 2025-02-09 02:56:40 -08:00
• cf797aa856 [core] port pynvml into vllm codebase (#12963) youkaichao 2025-02-09 15:00:00 +08:00
• 24700c346b [V1] Cache uses_mrope in GPUModelRunner (#12969) Woosuk Kwon 2025-02-08 15:32:32 -08:00
• d366ccc4e3 [RFC] [Mistral] FP8 format (#10130) Patrick von Platenandmgoin 2025-02-08 22:12:53 +01:00
• 870c37481e [V1][Minor] Remove outdated comment (#12968) Woosuk Kwon 2025-02-08 12:48:30 -08:00
• 86222a3dab [VLM] Merged multi-modal processor for GLM4V (#12449) Jee Jee Li 2025-02-09 04:32:16 +08:00
• fe743b798d [bugfix] fix early import of flash attention (#12959) youkaichao 2025-02-09 00:06:56 +08:00
• 913df14da3 [Bugfix] Remove unused seq_group_metadata_list from ModelInputForGPU (#12935) shangmingc 2025-02-08 22:46:19 +08:00
• 8a69e0e20e [CI/Build] Auto-fix Markdown files (#12941) Cyrus Leung 2025-02-08 20:25:15 +08:00
• 4c8dd12ef3 [Misc] Add qwen2.5-vl BNB support (#12944) Isotr0py 2025-02-08 20:24:47 +08:00
• 256a2d29dc [Doc] Correct HF repository for TeleChat2 models (#12949) Jun Duan 2025-02-08 04:42:15 -05:00
• c45d398e6f [CI] Resolve transformers-neuronx version conflict (#12925) Liangfu Chen 2025-02-08 01:41:35 -08:00
• 011e612d92 [Misc] Log time consumption on weight downloading (#12926) Jun Duan 2025-02-08 04:16:42 -05:00
• 7e1837676a [misc] Add LoRA to benchmark_serving (#12898) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-02-08 14:45:44 +05:30
• 2880e21e3d [Hardware][Intel-Gaudi] Enable long-contexts + LoRA support for Intel Gaudi (#12812) Sanju C Sudhakaran 2025-02-08 14:45:30 +05:30
• 407b5537db [Build] Make pypi install work on CPU platform (#12874) wangxiyuan 2025-02-08 17:15:15 +08:00
• 4ea48fb35c [V1][Minor] Move cascade attn logic outside _prepare_inputs (#12943) Woosuk Kwon 2025-02-08 00:39:09 -08:00
• e31498bdcb [Misc] Add offline test for disaggregated prefill (#12418) Shaoting 2025-02-08 02:38:20 -06:00
• 91dd8f7aa6 [bugfix] respect distributed_executor_backend in world_size=1 (#12934) youkaichao 2025-02-08 16:17:08 +08:00
• d01f66b039 [Bugfix] Fix multi-round chat error when mistral tokenizer is used (#12859) zifeitongandCyrus Leung 2025-02-07 23:04:34 -08:00
• cc01223f3b [Misc] Fix typo in the example file (#12896) Ke Zhao 2025-02-08 14:56:43 +08:00
• 306923da82 [Bugfix] Fix Qwen2_5_VLForConditionalGeneration packed_modules_mapping (#12905) Jee Jee Li 2025-02-08 13:02:53 +08:00
• 3243158336 [V1] Move KV block hashes from Request to KVCacheManager (#12922) Woosuk Kwon 2025-02-07 19:14:10 -08:00
• b21f0f9d17 [V1][Minor] Remove outdated comment (#12928) Woosuk Kwon 2025-02-07 19:07:37 -08:00
• 45cbc4991d [Bugfix] Fix disagg hang caused by the prefill and decode communication issues (#12723) Lu Fang 2025-02-07 16:39:50 -08:00
• 932c6b7461 [V1] LM Eval With Streaming Integration Tests (#11590) Robert Shaw 2025-02-07 18:07:03 -05:00
• eaa92d4437 [ROCm] [Feature] [Doc] [Dockerfile] [BugFix] Support Per-Token-Activation Per-Channel-Weight FP8 Quantization Inferencing (#12501) TJian 2025-02-08 00:13:43 +08:00
• 0630d4537a [V1] Logprobs and prompt logprobs support (#9880) 3 people 2025-02-07 10:26:20 -05:00
• 538fab93cd PR #12718 (#12718) Amit Garg 2025-02-07 06:22:37 -08:00
• ce26b16268 [Misc] Remove unnecessary detokenization in multimodal processing (#12868) Cyrus Leung 2025-02-07 22:21:17 +08:00
• 1918aa1b80 [MISC][EASY] Break check file names into entry and args in the pre-commit hooks (#12880) Lu Fang 2025-02-07 05:04:39 -08:00
• 6e1fc61f0f Prevent unecessary requests to huggingface hub (#12837) Maximilien de Bayser 2025-02-07 02:37:41 -03:00
• aa375dca9f [Bugfix] Missing quant_config in deepseek embedding layer (#12836) Szymon Ożóg 2025-02-07 06:35:09 +01:00
• 433c4a4923 Make vllm compatible with verl (#12824) ZSL98andzhangshulai 2025-02-07 11:54:20 +08:00
• ef533d25fb [Bugfix] FA2 illegal memory access (#12848) Lucas Wilkinson 2025-02-06 22:54:07 -05:00
• b260782357 [misc] Revert # 12833 (#12857) Kevin H. LuuandEC2 Default User 2025-02-06 16:29:12 -08:00
• 741429a4cd [MISC] Check space in the file names in the pre commit checks (#12804) Lu Fang 2025-02-06 15:36:21 -08:00
• aff404571b Add Bamba Model (#10909) Yu Chin Fabian LimandTyler Michael Smith 2025-02-07 07:22:42 +08:00
• 467a96a541 [V1] LoRA Support (#10957) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-02-06 23:02:51 +05:30
• 8108ac841d [Bugfix] Fix unsupported FA version check for Turing GPU (#12828) Isotr0py 2025-02-07 01:18:22 +08:00
• afe74f7a96 [Doc] double quote cmake package in build.inc.md (#12840) Jitse Klomp 2025-02-06 18:17:55 +01:00
• 09b95e36ab [torch.compile] PyTorch 2.6 and nightly compatibility (#12393) youkaichao 2025-02-07 01:09:07 +08:00
• 85ac82d228 [Kernel] Make rotary_embedding ops more flexible with input shape (#12777) Isotr0py 2025-02-07 00:46:13 +08:00
• 1e57b1ee63 [Misc] Remove unnecessary decode call (#12833) Cyrus Leung 2025-02-07 00:45:44 +08:00
• e152f29502 [misc] Reduce number of config file requests to HuggingFace (#12797) Kevin H. LuuandEC2 Default User 2025-02-06 06:59:18 -08:00
• c786e757fa [Attention] Use FA3 for MLA on Hopper (#12807) Lucas Wilkinson 2025-02-06 06:43:12 -05:00
• cefd56ee35 [Docs] Add Google Cloud Slides (#12814) Simon Mo 2025-02-06 01:02:38 -08:00
• 7ca9934fe7 [Misc] Update w2 scale loading for GPTQMarlinMoE (#12757) Dipika Sikka 2025-02-06 04:02:14 -05:00