Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• fabe89bbc4 [Spec Decode] Don't fall back to V0 when spec decoding is enabled (#18265) Woosuk Kwon 2025-05-16 16:10:27 -07:00
• e73b7dfd69 [Bugfix] fix an illegal memory access was encountered of marlin kernel + act_order (#18245) Jinzhen Lin 2025-05-17 07:02:44 +08:00
• 7fdfa01530 [Sampler] Adapt to FlashInfer 0.2.3 sampler API (#15777) Bowen Wangandmgoin 2025-05-16 15:14:03 -07:00
• aef94c6d07 [CI] Assign reviewer to mergify with changes to Tensorizer files (#18278) Sanger Steel 2025-05-16 15:04:14 -04:00
• 0ceaebf87b [BugFix] Fix ordering of KVConnector finished send/rcv sets (#18211) Nick Hill 2025-05-16 09:20:54 -07:00
• 1db4f47f81 [BugFix] Fix multi async save in MultiConnector (#18246) Nick Hill 2025-05-16 08:13:47 -07:00
• d3d91b6f71 [Misc][MacOS] fix bfloat16 error (#18249) Reidandreidliu41 2025-05-16 23:05:59 +08:00
• 87d871470d [Model] Use autoweightloader for dbrx (#18251) learner0810 2025-05-16 22:54:13 +08:00
• a5f8c111c2 [Fix] Fix typo in resolve_hf_chat_template (#18259) fxmarty-amd 2025-05-16 16:52:41 +02:00
• e23564cb70 use ceil_div in cutlass block scaling shape check (#17918) Lain 2025-05-16 03:02:58 -07:00
• 390ec88905 [Misc] Consolidate Audio tests into multimodal common generation tests (#18214) Isotr0py 2025-05-16 17:18:08 +08:00
• 541817670c [Misc] Add Ray Prometheus logger to V1 (#17925) Seiji Eicher 2025-05-16 03:02:42 -05:00
• 67da5720d4 [PERF] Speed up Qwen2.5-VL model by speed up rotary position embedding (#17973) Vadim Gimpelson 2025-05-16 10:31:02 +04:00
• 5c04bb8b86 [doc] fix multimodal example script (#18089) David Xia 2025-05-16 02:05:34 -04:00
• 3d2779c29a [Feature] Support Pipeline Parallism in torchrun SPMD offline inference for V1 (#17827) Lucia Fang 2025-05-15 22:28:27 -07:00
• 6b31c84aff Throw better error for when running into k8s service discovery issue (#18209) Will Eaton 2025-05-16 00:07:28 -04:00
• b18201fe06 Allow users to pass arbitrary JSON keys from CLI (#18208) Harry Mellor 2025-05-16 05:05:34 +01:00
• f4937a51c1 [Model] vLLM v1 supports Medusa (#17956) Sky Leeandlisiqi23 2025-05-16 12:05:31 +08:00
• ee659e3b60 [Bugfix][ROCm] Use chunked_prefill_paged_decode as fallback for V1 attention on ROCm (#18093) kliuae 2025-05-16 10:30:17 +08:00
• 4e1c6a0264 [Bugfix] fix rotary embedding test for _get_padded_tensor_shape (#18229) Lucas Wilkinson 2025-05-15 21:32:45 -04:00
• c7852a6d9b [Build] Allow shipping PTX on a per-file basis (#18155) Lucas Wilkinson 2025-05-15 19:41:55 -04:00
• 8795eb9975 [Bugfix] Fix test_eagle test (#18223) Lucia Fang 2025-05-15 15:59:42 -07:00
• 0b34593017 Adding "AMD: Tensorizer Test" to amdproduction. (#18216) Alexei-V-Ivanov-AMD 2025-05-15 13:01:25 -05:00
• e3f3aee6f4 [Misc] Avoid cuda graph log when sizes still match (#18202) Nicolò Lucchesi 2025-05-15 18:59:38 +02:00
• 92540529c0 [Bugfix] [ROCm]: Remove assertion logic when using AITER fused moe in unquantizedMethod to reenable LLama4 BF16 (#18205) TJian 2025-05-16 00:53:18 +08:00
• fadb8d5c2d [Bugfix]Change the exception thrown by call_hf_processor from RuntimeError to ValueError (#18181) Zhonghua Deng 2025-05-16 00:01:47 +08:00
• 2aa5470ac5 [Frontend] Fix chat template content format detection (#18190) Sebastian Schoennenbeck 2025-05-15 18:00:21 +02:00
• 51ff154639 Improve examples rendering in docs and GitHub (#18203) Harry Mellor 2025-05-15 16:57:49 +01:00
• 566ec04c3d Adding "Basic Models Test" and "Multi-Modal Models Test (Extended) 3" in AMD Pipeline (#18106) Alexei-V-Ivanov-AMDandCyrus Leung 2025-05-15 10:49:23 -05:00
• 01c22335ba [Kernel] [V1] Fix performance regression for triton unified attention (#18161) Thomas ParnellandLucas Wilkinson 2025-05-15 15:39:00 +02:00
• 451da4bcbd add tools into TokenizeChatRequest (#18187) hustxiayang 2025-05-15 07:01:49 -04:00
• 07ad27121f Update deprecated type hinting in model_loader (#18130) Harry Mellor 2025-05-15 12:00:21 +01:00
• a9944aabfa fix: typos (#18151) omahs 2025-05-15 11:16:15 +02:00
• a8f5aec20a [V1] Update zmq socket creation in nixl connector (#18148) Russell Bryant 2025-05-15 02:17:57 -04:00
• de71fec81b [CI] don't skip fixed test_kv_cache_events() (#18183) David Xia 2025-05-15 02:17:16 -04:00
• 70f8b96724 [Bugfix] Fix FusedMoEPrepareAndFinalize for cuda-disalike backends (#18178) Mengqing Cao 2025-05-15 14:16:31 +08:00
• dd2a94596a [Model] Allow the use of sliding window in Qwen2 (#17772) inkcherry 2025-05-15 13:29:38 +08:00
• 420caf7557 [UT] Add ut for none hash (#17892) Ning Xie 2025-05-15 13:28:11 +08:00
• 4f07a64075 Support custom implementations of VideoLoader backends. (#18091) Chenheli Hua 2025-05-14 22:26:49 -07:00
• e6b8e65d2d [Bugfix] Fix fp8 tests for triton_unified_attention for Triton 3.3 (#18013) Thomas ParnellandLucas Wilkinson 2025-05-15 07:26:34 +02:00
• 26d0419309 Update deprecated type hinting in models (#18132) Harry Mellor 2025-05-15 06:06:50 +01:00
• 83f74c698f [Fix][ROCm] Enforce eager for all encoder-decoder models on ROCm (#18154) Luka Govedič 2025-05-15 01:04:43 -04:00
• 2dff093574 [Misc] add lobe-chat support (#18177) Reidandreidliu41 2025-05-15 13:02:23 +08:00
• afe3236e90 [Chore] astral's ty (#18116) Aaron Pham 2025-05-15 01:00:43 -04:00
• 65334ef3b9 [V1][Metrics] Remove unused code (#18158) Mark McLoughlin 2025-05-15 04:13:17 +01:00
• e60f550b38 [v1] Support multiple KV cache groups in GPU model runner (#17945) Chen Zhang 2025-05-15 09:54:54 +08:00
• f25e0d1125 [Bugfix]: make most of test_openai_schema.py pass (#17664) David Xia 2025-05-14 20:04:35 -04:00
• 09f106a91e Upload vllm index for the rc builds (#18173) Andrey Talman 2025-05-14 16:35:56 -07:00
• 2142035b51 [V1] Support multiple kv connectors (#17564) Michael GoinandNick Hill 2025-05-14 19:28:02 -04:00
• 78aa341d12 [CI] Fix race condition in test_kv_cache_events test (#18169) Russell Bryant 2025-05-14 19:27:48 -04:00
• 7974736740 Add support for loading torchao models with AOPerModuleConfig (#17826) Jerry Zhang 2025-05-14 16:24:59 -07:00
• 2fc9075b82 [V1] Structured Outputs + Thinking compatibility (#16577) Aaron PhamandRussell Bryant 2025-05-14 18:45:24 -04:00
• d93c976a0d [Kernel] Have rotary embeddings support tensors (#18046) Lucas Wilkinson 2025-05-14 18:43:55 -04:00
• 749f792553 [Frontend] decrease import time of vllm.multimodal (#18031) David XiaandAaron Pham 2025-05-14 18:43:32 -04:00
• 856865008e [CI] Disable Failing Tests (#18165) Robert Shaw 2025-05-14 16:49:56 -04:00
• f9c069c85e Modularize fused experts and integrate PPLX kernels (#15956) bnellnm 2025-05-14 16:11:54 -04:00
• 418d2f8bfb [V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326) 3 people 2025-05-14 15:31:46 -04:00
• 964472b966 [Doc] Update prefix cache metrics to counting tokens (#18138) Chen Zhang 2025-05-14 23:23:30 +08:00
• 59dd311cf5 [KVConnector] Keep KVTransferParams as a dict (#18033) Nick Hill 2025-05-14 08:05:57 -07:00
• d066e52013 [Bugfix] Fix chat utils tests (#18139) Cyrus Leung 2025-05-14 20:38:21 +08:00
• c8ea982d9b Update deprecated type hinting in platform, plugins, triton_utils, vllm_flash_attn (#18129) Harry Mellor 2025-05-14 13:28:16 +01:00
• dc372b9c8a Update deprecated type hinting in vllm/device_allocator and vllm/distributed (#18126) Harry Mellor 2025-05-14 12:07:57 +01:00
• 9b5b39b650 Update deprecated type hinting in vllm/lora (#18128) Harry Mellor 2025-05-14 11:57:59 +01:00
• 9ccc6ded42 [doc] add missing import (#18133) Reidandreidliu41 2025-05-14 18:57:34 +08:00
• d62a076e84 [Model] GritLM supports other attention backends (#18109) Cyrus Leung 2025-05-14 18:33:19 +08:00
• 259127f8b8 [Bugfix] Fix LoRA test (#18123) Jee Jee Li 2025-05-14 18:25:47 +08:00
• 612c2edb4f [FEAT] [ROCm]: Add AITER CK 2 Stages MoE support (#17110) TJianandGregory Shtrasberg 2025-05-14 18:03:11 +08:00
• 38fe728d60 [Bugfix] Fix QKVCrossParallelLinear::sync_weight_attrs for PyTorch compile (#17844) Andrzej Kotłowski 2025-05-14 11:39:51 +02:00
• 82e7f9bb03 [Misc] replace does not exist model (#18119) rongfu.leng 2025-05-14 17:13:47 +08:00
• 63dc3426e0 [Model] Add packed_modules_mapping for Qwen3-MOE (#18118) Jee Jee Li 2025-05-14 17:13:19 +08:00
• 8f5dc41481 [Bugfix] Fix entrypoints audio test failure (#18111) Cyrus Leung 2025-05-14 17:08:07 +08:00
• 63ad622233 [New Model]: support GTE NewModel (#17986) wang.yuqi 2025-05-14 16:31:31 +08:00
• e7ef61c1f0 [Bugfix][Example] make lmcache v0 work. (#18051) majianpeng 2025-05-14 14:43:44 +08:00
• d4154c35a2 [Bugfix] fix moe marlin topk_weight loading (#18080) Jinzhen Linandmgoin 2025-05-14 14:31:57 +08:00
• 6685890d11 [Fix] Move "model_config" as keyword args in chat_utils.py (#18098) lkchen 2025-05-13 23:27:26 -07:00
• 33011318c2 Fix broken example: examples/offline_inference/profiling at scheduler_config (#18117) Ecthlion_zyy 2025-05-14 14:19:14 +08:00
• 4f8b373225 [BugFix][AMD] Compatible patch for AITER lib after 04/20 (#17912) qli88 2025-05-14 01:05:20 -05:00
• 7b2f28deba [AMD][torch.compile] Enable silu+fp8_quant fusion for rocm (#18082) Charlie Fu 2025-05-14 00:13:56 -05:00
• 2d912fb66f [FEAT] [ROCm] [V1]: Add AITER biased group topk for DeepSeekV3 (#17955) vllmellmandtjtanaa 2025-05-14 13:03:47 +08:00
• 12e6c0b41c [Bugfix][V1] Fix FlashInfer V1 backend using the wrong VllmConfig (#18086) Michael Goin 2025-05-13 23:36:17 -04:00
• 9a2a6357de [Bugfix] Fix FP8 Marlin MoE and enable for compressed-tensors models (#18026) Michael Goin 2025-05-13 22:48:33 -04:00
• 6266c57bae [core][distributed] add ep group and all2all interface (#18077) youkaichao 2025-05-14 10:46:49 +08:00
• 754b699cbe [Bug]: Fix S3 model/tokenizer path resolution (#18083) Jon Gill 2025-05-13 19:34:17 -07:00
• 6e27c6d86b [Misc] Remove unused numpy tensor (#18084) Roger Wang 2025-05-13 19:33:40 -07:00
• d5af47a149 [P/D] Add some more debug logs to NixlConnector (#18102) Nick Hill 2025-05-13 19:33:03 -07:00
• 65f0f74b66 [Hardware/NVIDIA/Modelopt] Fix modelopt forward method for v1 torch.compile (#18101) Pavani Majety 2025-05-13 19:33:00 -07:00
• 176a95c670 [Fix] Support CUDAGraph capture for encoder-decoder on ROCm (#18104) Luka Govedič 2025-05-13 22:31:42 -04:00
• f2ae883b67 [v1][KVCacheManager] pass num_new_computed_tokens to kv cache manager (#18001) Chen Zhang 2025-05-14 10:09:39 +08:00
• 40de1ef455 [FEAT] [ROCm]: Add AITER Block-Scaled GEMM Feature (#14968) vllmellmandtjtanaa 2025-05-14 10:08:20 +08:00
• 0189a65a2e [Docs] Expand security doc with firewall info (#18081) Russell Bryant 2025-05-13 15:36:00 -04:00
• 55aa7af994 [V1] DP scale-out (2/N): Decouple engine process management and comms (#15977) Nick Hill 2025-05-13 10:48:21 -07:00
• 0b217da646 Update deprecated type hinting in vllm/adapter_commons (#18073) Harry Mellor 2025-05-13 16:32:51 +01:00
• 19324d660c Update deprecated type hinting in vllm/compilation (#18072) Harry Mellor 2025-05-13 16:32:48 +01:00
• fc407a1425 Give auto-merge label workflow permission to add labels to issues (#18078) Harry Mellor 2025-05-13 15:53:13 +01:00
• 009d9e7590 Convert benchmarks to ruff format (#18068) Harry Mellor 2025-05-13 14:43:29 +01:00
• b922c2ebd2 [Bugfix] Fix entrypoints metrics tests (#18063) Cyrus Leung 2025-05-13 21:42:43 +08:00
• 00b14e0f16 [CI] set token permissions for pre-commit CI job (#17729) Russell BryantandCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> 2025-05-13 09:38:30 -04:00
• 54e467e6f8 [CI] Add token permissions for add-ready-label CI job (#17730) Russell BryantandCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> 2025-05-13 09:38:13 -04:00
• 79a1d25bbd [CI] Add workflow permissions for helm CI job (#17727) Russell BryantandCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> 2025-05-13 08:49:07 -04:00
• 9944011b30 [CI] Set token permissions for reminder comment CI job (#17728) Russell BryantandCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> 2025-05-13 08:46:58 -04:00