Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 7ffcccfa5c Revert "[CI/Build] Use uv python for docker rather than ppa:deadsnakess/ppa (#13569)" (#15377) Simon Mo 2025-03-24 05:53:10 -07:00
• cc8accfd53 [Misc] Update guided decoding logs to debug (#15310) sfbemerkandBenjamin Merkel 2025-03-24 12:25:20 +01:00
• 948ab03e7e [Bugfix][V1] Avoid importing PreTrainedModel (#15366) ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 2025-03-24 12:33:12 +02:00
• 5797fb97e9 [Misc] Remove ignore_reinit_error for ray.init() (#15373) Rui Qiao 2025-03-24 00:41:53 -07:00
• 3892e58ad7 [Misc] Upgrade BNB version (#15183) Jee Jee Li 2025-03-24 13:51:42 +08:00
• d20e261199 Fix non-contiguous input passed to Marlin kernel (#15319) Qubitium-ModelCloud 2025-03-24 11:09:44 +08:00
• f622dbcf39 [Fix] [torch.compile] Improve UUID system for custom passes (#15249) Luka Govedič 2025-03-23 21:54:07 -04:00
• dccf535f8e [V1] Enable V1 Fp8 cache for FA3 in the oracle (#15191) Lucas Wilkinson 2025-03-23 18:07:04 -04:00
• 9c5c81b0da [Misc][Doc] Add note regarding loading generation_config by default (#15281) Roger Wang 2025-03-23 14:00:55 -07:00
• d6cd59f122 [Frontend] Support tool calling and reasoning parser (#14511) Robin 2025-03-24 05:00:07 +08:00
• bc8ed3c4ba [V1][Spec Decode] Use better defaults for N-gram (#15358) Woosuk Kwon 2025-03-23 10:52:30 -07:00
• b9bd76ca14 [V1][Spec Decode] Respect prompt_lookup_max (#15348) Woosuk Kwon 2025-03-23 10:41:44 -07:00
• 6ebaf9ac71 [Bugfix] consider related env vars for torch.compiled cache hash (#14953) DefTruth 2025-03-23 23:53:09 +08:00
• f90d34b498 [Misc] Add tuned R1 w8a8 and MoE configs for NVIDIA L20 (#15322) DefTruth 2025-03-23 16:10:10 +08:00
• f68cce8e64 [ci/build] fix broken tests in LLM.collective_rpc (#15350) youkaichao 2025-03-23 14:49:48 +08:00
• 09b6a95551 [ci/build] update torch nightly version for GH200 (#15135) youkaichao 2025-03-23 14:04:13 +08:00
• 50c9636d87 [V1][Usage] Refactor speculative decoding configuration and tests (#14434) shangmingc 2025-03-23 13:28:10 +08:00
• 0661cfef7a Fix v1 supported oracle for worker-cls and worker-extension-cls (#15324) hijkzzzandyoukaichao 2025-03-23 10:23:35 +08:00
• a827aa815d [doc] Add back previous news (#15331) Chen Zhang 2025-03-23 08:38:33 +08:00
• b877031d80 Remove openvino support in favor of external plugin (#15339) Russell Bryant 2025-03-22 17:06:39 -04:00
• dd861b992f [BugFix][Typing] Fix Imprecise Type Annotations (#15208) Wang Ran (汪然) 2025-03-23 00:05:03 +08:00
• eb63ea1e18 [V1] Add disable-any-whitespace option support for xgrammar (#15316) Russell Bryant 2025-03-22 11:56:17 -04:00
• 2f4bd358f1 [Model] Support Tele-FLM Model (#15023) 3 people 2025-03-22 17:04:44 +08:00
• 8a8b30eac1 [Bugfix] LoRA V0 - Fix case where max_num_seqs is between cudagraph capture sizes (#15308) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-03-22 05:03:32 -04:00
• 2fa0e1396b [Bugfix] Fix torch.compile raise FileNotFoundError (#15278) Jee Jee Li 2025-03-22 13:49:34 +08:00
• 1c2bec0f82 [Doc] add load_format items in docs (#14804) wwl2755 2025-03-22 00:36:43 -05:00
• ec870fba9a [FEAT] [ROCm]: Add AITER RMS Norm (Layer Norm) Feature (#14959) TJian 2025-03-22 13:36:14 +08:00
• df1430265c [Bugfix][V0] Multi-sequence logprobs streaming edge case (#15259) Andy Lo 2025-03-22 05:35:37 +00:00
• 4c69e228b3 [Misc] Increase RayDistributedExecutor RAY_CGRAPH_get_timeout (#15301) Rui Qiao 2025-03-21 22:25:43 -07:00
• 790b79750b [Build/CI] Fix env var typo (#15305) Russell Bryant 2025-03-21 18:28:46 -04:00
• cfbb8c930f [TPU][V1] MHA Pallas backend (#15288) Nicolò Lucchesi 2025-03-21 16:50:39 +01:00
• baec0d4de9 Revert "[Feature] specify model in config.yaml (#14855)" (#15293) Cyrus Leung 2025-03-21 23:30:23 +08:00
• c21b99b912 [Bugfix][VLM] fix llava processor (#15285) Mengqing Cao 2025-03-21 20:14:36 +08:00
• 93a00d7dde [v1] Refactor KVCacheConfig (#14079) Chen Zhang 2025-03-21 19:56:27 +08:00
• 61e8c18350 [Misc] Add cProfile helpers (#15074) Russell Bryant 2025-03-21 07:56:09 -04:00
• 8afcd0f633 [Bugfix] Fix broken kernel test due to missing rename for v1 Triton backend (#15282) Isotr0py 2025-03-21 19:42:06 +08:00
• 91ca929dc7 [V1] Fix wrong import path of get_flash_attn_version (#15280) Lehua Ding 2025-03-21 18:54:11 +08:00
• 84e00adc8a [Bugfix] Fix incorrect resolving order for transformers fallback (#15279) Isotr0py 2025-03-21 18:54:08 +08:00
• 47c7126213 [Misc] Add attention mask pre-computation optimization back to Qwen2.5-VL (#15273) Isotr0py 2025-03-21 18:32:33 +08:00
• a989ca2bf6 [Bugfix] Add int8 torch dtype for KVCache (#15260) Shanshan Shen 2025-03-21 16:58:28 +08:00
• 0fa3970deb [Feature] specify model in config.yaml (#14855) Wei Zeng 2025-03-21 00:26:03 -07:00
• da6ea29f7a [V1] Avoid redundant input processing in n>1 case (#14985) Nick Hill 2025-03-20 22:24:10 -07:00
• 7297941b38 [Doc] Update LWS docs (#15163) Edwin Hernandez 2025-03-20 21:18:47 -07:00
• f8a08cb90d [V1] Enable Triton(ROCm) Attention backend for Nvidia GPUs (#14071) Isotr0pyandWoosuk Kwon 2025-03-21 11:14:19 +08:00
• b15fd2be2a [Hardware][TPU] Add check for no additional graph compilation during runtime (#14710) Siyuan Liu 2025-03-20 20:05:28 -07:00
• e588ac237c Add an example for reproducibility (#15262) Woosuk Kwon 2025-03-20 19:55:47 -07:00
• 5df2da5b97 [Misc] Better RayExecutor and multiprocessing compatibility (#14705) Cody Yu 2025-03-20 19:27:46 -07:00
• 11b986b3fb [Docs] Trim the latest news in README (#15261) Woosuk Kwon 2025-03-20 19:24:21 -07:00
• 296f927f24 [Model] RE: Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14857) Chih-Chieh Yang 2025-03-20 19:21:08 -07:00
• 0032903a5b [Bugfix] detect alibi and revert to FA2 (#15231) Travis Johnson 2025-03-20 20:20:16 -06:00
• 47195057e9 [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) Hyesoo Yang 2025-03-20 19:19:40 -07:00
• 6edbfa924d Mention extra_body as a way top pass vLLM only parameters using the OpenAI client (#15240) Harry Mellor 2025-03-21 02:18:36 +00:00
• 1e508343e1 [Bugfix] Fix incorrect qwen2.5-vl attention mask pre-computation (#15200) Isotr0py 2025-03-21 10:18:04 +08:00
• 2e0b4cfde0 [ROCM] Upgrade torch to 2.6 (#15244) Sage Moore 2025-03-20 19:17:33 -07:00
• 10f55fe6c5 [Misc] Clean up the BitsAndBytes arguments (#15140) Jee Jee Li 2025-03-21 10:17:12 +08:00
• d3ccbd6350 Fix CUDA kernel index data type in vllm/csrc/quantization/fused_kernels/layernorm_utils.cuh +10 (#15159) Lu FangandRichard Barnes 2025-03-20 19:01:11 -07:00
• 0cfe7d386d [CI/Build] LoRA : make add_lora_test safer (#15181) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-03-20 21:28:53 -04:00
• 0c6f5023c3 [V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250) 3 people 2025-03-20 17:50:43 -07:00
• 06dd08256f Enforce that TP > 1 is not supported for Mamba2 if Quantization is Enabled. (#14617) Yu Chin Fabian Lim 2025-03-21 08:44:37 +08:00
• 2b22290ce0 [V1] Add flag to disable cascade attention (#15243) Woosuk Kwon 2025-03-20 15:24:16 -07:00
• d8e82bc06d [Bugfix] fix V1 Engine crash while handling requests with duplicate request id (#15043) Jason 2025-03-21 01:01:02 +08:00
• 086b56824c [ci] feat: make the test_torchrun_example run with tp=2, external_dp=2 (#15172) Chi Zhangandyoukaichao 2025-03-21 00:30:04 +08:00
• 5a0905ba2a Replace misc issues with link to forum (#15226) Harry Mellor 2025-03-20 15:18:20 +00:00
• a8f12a63fd Fix env vars for running Ray distributed backend on GKE (#15166) Richard Liu 2025-03-20 07:59:33 -07:00
• 69ae2380c6 Add user forum to README (#15220) Harry Mellor 2025-03-20 14:39:51 +00:00
• 27261e40a6 [Bugfix] Multi-video inference on LLaVA-Onevision (#15082) Cyrus LeungandIsotr0py 2025-03-20 22:10:45 +08:00
• e3f813c33b [macOS] Ugrade pytorch to 2.6.0 (#15129) Quang-Linh LE 2025-03-20 09:22:40 +01:00
• c607a2652b Fixing Imprecise Type Annotations (#15192) Wang Ran (汪然) 2025-03-20 16:19:55 +08:00
• 3d45e3d749 [release] Tag vllm-cpu with latest upon new version released (#15193) Kevin H. Luu 2025-03-20 01:19:10 -07:00
• 742369d35a [Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824) billishyahaoandZhai Feiyue 2025-03-20 15:00:33 +08:00
• bfe2fe0af4 typo: Update config.py (#15189) Wang Ran (汪然) 2025-03-20 14:31:21 +08:00
• a8652f4f0f Enable CUDA graph support for llama 3.2 vision (#14917) Matt Ritter 2025-03-19 23:29:16 -07:00
• 2f726b241e [Doc] Update README.md (#15187) Cyrus Leung 2025-03-20 13:25:58 +08:00
• a597a57595 [Attention] Flash Attention 3 - fp8 (#14570) Mickaël Seznec 2025-03-20 06:14:20 +01:00
• ae65f3e237 [Misc]fixed disable these http request logs (#14754) Chauncey 2025-03-20 12:53:40 +08:00
• 34868b106a [Doc] Update Mistral Small 3.1/Pixtral example (#15184) Roger Wang 2025-03-19 21:46:06 -07:00
• 1f16b7fe74 [Core][V0] Add guidance backend for structured output (#14589) 4 people 2025-03-20 00:33:51 -04:00
• b88be22165 [Benchmark] Allow oversample request in benchmark dataset (#15170) Jennifer Zhao 2025-03-19 21:32:58 -07:00
• d8c6d7d6b5 [V1][TPU] Support V1 Sampler for ragged attention (#14227) Nicolò Lucchesi 2025-03-20 05:00:39 +01:00
• 40828ce5fe fix "Total generated tokens:" is 0 if using --backend tgi and --endpo… (#14673) Wang, Yi 2025-03-20 11:56:16 +08:00
• ffa443afed [Bugfix] Fix embedding assignment for InternVL-based models (#15086) Cyrus Leung 2025-03-20 11:40:13 +08:00
• 70e500cad9 Fix broken tests (#14713) Jovan SardinhaandLuka Govedič 2025-03-19 19:06:49 -07:00
• 4cb1c05c9e [Doc] Clarify run vllm only on one node in distributed inference (#15148) Rui Qiao 2025-03-19 18:55:59 -07:00
• c47aafa37c [BugFix] Lazily import XgrammarBackend to avoid early cuda init (#15171) Nick Hill 2025-03-19 18:30:43 -07:00
• cfbca8a2f2 [V1] TPU - Tensor parallel MP support (#15059) Alexander Matveev 2025-03-19 20:55:18 -04:00
• 0fe5609874 [Docs] Annouce Ollama and Singapore Meetups (#15161) Simon Mo 2025-03-19 16:18:04 -07:00
• 22d33baca2 [FrontEnd][Perf] merge_async_iterators fast-path for single-prompt requests (#15150) Nick Hill 2025-03-19 14:04:41 -07:00
• b0e96aaebb [V1][TPU] Change kv cache shape. (#15145) iefgnoix 2025-03-19 12:16:42 -07:00
• 8310e0b59b simple bugfix: Update stats.py (#15139) Wang Ran (汪然) 2025-03-20 02:26:27 +08:00
• 26dd972adb [FEAT]Support reset prefix cache by specified device (#15003) maobaolong 2025-03-20 01:54:41 +08:00
• 61c7a1b856 [V1] Minor V1 async engine test refactor (#15075) Murali Andoorveeduandandoorve 2025-03-19 10:37:17 -07:00
• 374ee287d8 [Frontend] Remove custom_cache_manager (#13791) Alessandro Sangiorgi 2025-03-19 11:13:50 -05:00
• a4d83661d7 [Misc] Update the "the first vLLM China Meetup" slides link to point to the first page (#15134) Kero Liang 2025-03-19 23:07:39 +08:00
• 8363cd093d [Bugfix] Adjust mllama to regional compilation (#15112) Jan Kaniecki 2025-03-19 15:57:25 +01:00
• 6c5a3195db [Misc][Benchmark] Add support for different tokenizer_mode (#15040) Aaron PhamandRussell Bryant 2025-03-19 10:56:50 -04:00
• 073d1ed354 [Doc] Update tip info on using latest transformers when creating a custom Dockerfile (#15070) Marc-Alexandre Côté 2025-03-19 09:33:40 -04:00
• 3d446433ec [Bugfix] Fix size calculation of processing cache (#15114) Cyrus Leung 2025-03-19 20:53:19 +08:00
• 1fe0fd12d3 [Misc] Avoid unnecessary HF do_rescale warning when passing dummy data (#15107) Cyrus Leung 2025-03-19 18:42:31 +08:00
• dafb4e504a [V1][Bugfix] Fix oracle for device checking (#15104) Roger Wang 2025-03-19 03:35:32 -07:00
• 68cf1601d3 [CI][Intel GPU] update XPU dockerfile and CI script (#15109) Kunshang Ji 2025-03-19 01:29:25 -07:00