Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• ca100c90fe Add benchmark for DeepGEMM and vLLM Block FP8 Dense GEMM (#13917) Michael Goin 2025-03-05 20:08:51 -05:00
• ffad94397d [CI/Build] Use spawn multiprocessing mode for V1 test pipeline (#14243) Russell Bryant 2025-03-05 20:08:02 -05:00
• 4dacaa4a83 [BugFix] Fix prefix caching V0 MLA (#14255) Lucas WilkinsonandYing Zhong 2025-03-05 20:07:42 -05:00
• a7ea35aa67 [Bugfix] Remove num_tokens_across_dp (#14302) Tyler Michael Smith 2025-03-05 18:55:55 -05:00
• 1e3e76b6cc [Bugfix] Fix DeepSeek MTP crash when using TP1ModelRunner with CUDA graph due to shape mismatch (#14237) pyc96 2025-03-05 14:22:40 -08:00
• 53ea6ad830 [V1][Easy] Add empty allowed_token_ids in the v1 sampler test (#14308) Lu Fang 2025-03-05 13:41:18 -08:00
• 1b7624bf5c [misc] Add FlashMLA as a new option of VLLM_ATTENTION_BACKEND env (#14267) Serena 2025-03-06 05:28:50 +08:00
• ac60dc7fe1 [V1][BugFix] Fix for mixed top_k batch (#14301) Nick HillandYe Cao 2025-03-05 12:43:04 -08:00
• a4f1ee35d6 Deprecate best_of Sampling Parameter in anticipation for vLLM V1 (#13997) 3 people 2025-03-05 15:22:43 -05:00
• a32c8669ca [V1][Minor] Remove obsolete FIXME comment (#14304) Nick Hill 2025-03-05 11:59:23 -08:00
• ca2ca8de57 [Docs] Add Meta Slides (#14297) Simon Mo 2025-03-05 08:30:23 -08:00
• f71b00a19e [Bugfix] Fix broken vision language example (#14292) Isotr0py 2025-03-05 23:57:10 +08:00
• 8f808cf86e prefix_caching.md: Fixed typo (#14293) DaividFrank 2025-03-05 16:43:13 +01:00
• 7bab4bb048 [Misc] Add Qwen2MoeForCausalLM moe tuning support (#14276) Jee Jee Li 2025-03-05 23:11:29 +08:00
• e17e4488bd [LoRA] Remove linear hack outside transformers backend (#14177) Isotr0py 2025-03-05 23:06:28 +08:00
• 257e200a25 [V1][Frontend] Add Testing For V1 Runtime Parameters (#14159) Robert Shaw 2025-03-05 14:18:55 +00:00
• 47d4a7e004 Small update for external_launcher backend docs (#14288) Zhe Zhang 2025-03-05 05:30:00 -08:00
• 7f89a594dd [Doc] [3/N] Refer code examples for common cases in dev multimodal processor (#14278) Cyrus Leung 2025-03-05 20:29:50 +08:00
• 961644e6a8 [Doc] Update nginx guide: remove privileged from vllm container run and add target GPU ID (#14217) Iacopo Poli 2025-03-05 12:44:10 +01:00
• 8d6cd32b7b [Bugfix][V1] Fix allowed_token_ids for v1 Sampler (#14169) Lu Fang 2025-03-05 00:49:44 -08:00
• ec79b67c77 [Misc][V1] Avoid using envs.VLLM_USE_V1 in mm processing (#14256) Roger Wang 2025-03-04 23:37:16 -08:00
• 32985bed7c [Frontend] Allow return_tokens_as_token_ids to be passed as a request param (#14066) Benjamin Chislett 2025-03-05 01:30:40 -05:00
• dae9ec464c Temporarily disable test_awq_gemm_opcheck (#14251) Michael Goin 2025-03-05 01:10:35 -05:00
• 6eaf93020d [platforms] improve rocm debugging info (#14257) youkaichao 2025-03-05 13:32:18 +08:00
• 72c62eae5f [V1] EP/TP MoE + DP Attention (#13931) Tyler Michael Smith 2025-03-05 00:27:26 -05:00
• 0a995d5434 [Model] New model support for Phi-4-multimodal-instruct (#14119) Congcong Chen 2025-03-04 20:57:01 -08:00
• ade3f7d988 [V1][Bugfix] Do not reset prefix caching metrics (#14235) Cody Yu 2025-03-04 20:39:13 -08:00
• 0df25101d6 [Bugfix] Fix gptq_marlin for deepseek-v3 (#13750) rainkertanddangshunya 2025-03-05 12:25:53 +08:00
• e123aafdf0 Disable GPTQ AllSpark kernels for CUDA Compiler < 12.0 (#14157) Michael Goin 2025-03-04 23:25:24 -05:00
• 5b143d33be Moved numba from common requirements to cuda/rocm specific requirements (#14199) Nishidha 2025-03-05 09:55:00 +05:30
• eb59b5a6cb [misc] announce china meetup (#14248) youkaichao 2025-03-05 10:33:50 +08:00
• fbfc3ee37e [V1][TPU] TPU multimodal model support for ragged attention (#14158) Michael Goin 2025-03-04 19:58:48 -05:00
• 3e1d223626 [ROCm] Disable a few more kernel tests that are broken on ROCm (#14145) Sage Moore 2025-03-04 15:37:55 -08:00
• 4f5b059f14 Clean up unused padding_idx variables across many model definitions (#13240) Tyler Michael Smith 2025-03-04 16:27:00 -05:00
• 288ca110f6 [Security] Serialize using safetensors instead of pickle in Mooncake Pipe (#14228) Kuntai Du 2025-03-04 15:10:32 -06:00
• c2bd2196fc [v1][Metrics] Add design doc (#12745) 3 people 2025-03-04 20:36:55 +00:00
• 550c7ba3dc [Docs] Update Dockerfile dependency image (#14215) Michael Goin 2025-03-04 15:22:11 -05:00
• e5b2f1601a [Frontend] Do prompt_logprobs clamping for chat as well as completions (#14225) Harry Mellor 2025-03-04 21:13:06 +01:00
• 9badee53de Fix performance when --generation-config is not None (#14223) Harry Mellor 2025-03-04 20:59:22 +01:00
• beebf4742a [TPU][Profiler] Support start_profile/stop_profile in TPU worker (#13988) Siyuan Liuandmgoin 2025-03-04 11:40:06 -08:00
• f89978ad7c add cutlass support for blackwell fp8 gemm (#13798) kushanam 2025-03-04 07:55:07 -08:00
• b3cf368d79 [V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (#14161) lkchen 2025-03-04 07:43:59 -08:00
• c8525f06fc [V0][Metrics] Deprecate some questionable request time metrics (#14135) Mark McLoughlin 2025-03-04 15:11:33 +00:00
• 5db6b2c961 [V1][BugFix] Fix remaining sync engine client shutdown errors/hangs (#13869) Nick Hill 2025-03-04 07:06:47 -08:00
• 6247bae6c6 [Bugfix] Restrict MacOS CPU detection (#14210) Michael Goin 2025-03-04 09:25:27 -05:00
• 3610fb4930 [doc] add "Failed to infer device type" to faq (#14200) youkaichao 2025-03-04 20:47:06 +08:00
• 71c4b40562 [sleep mode] error out with expandable_segments (#14189) youkaichao 2025-03-04 18:54:19 +08:00
• ac65bc92df [platform] add debug logging during inferring the device type (#14195) youkaichao 2025-03-04 18:39:16 +08:00
• f78c0be80a Fix benchmark_moe.py tuning for CUDA devices (#14164) Michael Goin 2025-03-04 00:11:03 -05:00
• 66233af7b6 Use math.prod instead of np.prod for trivial ops (#14142) Zhanwen Chen 2025-03-04 00:09:22 -05:00
• bf13d40972 [core] Pass all driver env vars to ray workers unless excluded (#14099) Rui Qiao 2025-03-03 19:44:17 -08:00
• 989f4f430c [Misc] Remove lru_cache in NvmlCudaPlatform (#14156) Cody Yu 2025-03-03 19:09:34 -08:00
• bb5b640359 [core] moe fp8 block quant tuning support (#14068) Divakar Verma 2025-03-03 19:30:23 -06:00
• c060b71408 [Model] Add support for GraniteMoeShared models (#13313) Travis JohnsonandCyrus Leung 2025-03-03 17:04:52 -07:00
• 79e4937c65 [v1] Add comments to the new ragged paged attention Pallas kernel (#14155) iefgnoixandMichael Goin 2025-03-03 15:00:55 -08:00
• cd1d3c3df8 [Docs] Add GPTQModel (#14056) Qubitium-ModelCloudandmgoin 2025-03-04 05:59:09 +08:00
• 19d98e0c7d [Kernel] Optimize moe intermediate_cache usage (#13625) Michael Goin 2025-03-03 16:29:53 -05:00
• 2b04c209ee [Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (#14100) Michael Goin 2025-03-03 16:20:24 -05:00
• ae122b1cbd [WIP][[V1][Metrics] Implement max_num_generation_tokens, request_params_n, and request_params_max_tokens metrics (#14055) Mark McLoughlin 2025-03-03 19:04:45 +00:00
• 872db2be0e [V1] Simplify stats logging (#14082) Nick Hill 2025-03-03 10:34:14 -08:00
• 2dfdfed8a0 [V0][Metrics] Deprecate some KV/prefix cache metrics (#14136) Mark McLoughlin 2025-03-03 18:25:46 +00:00
• c41d27156b [V0][Metrics] Remove unimplemented vllm:tokens_total (#14134) Mark McLoughlin 2025-03-03 17:50:22 +00:00
• 91373a0d15 Fix head_dim not existing in all model configs (Transformers backend) (#14141) Harry Mellor 2025-03-03 17:48:11 +00:00
• 848a6438ae [ROCm] Faster Custom Paged Attention kernels (#12348) TJian 2025-03-04 01:24:45 +08:00
• 98175b2816 Improve the docs for TransformersModel (#14147) Harry Mellor 2025-03-03 17:03:05 +00:00
• 4167252eaf [V1] Refactor parallel sampling support (#13774) Mark McLoughlin 2025-03-03 16:15:27 +00:00
• f35f8e2242 [Build] Make sure local main branch is synced when VLLM_USE_PRECOMPILED=1 (#13921) Cody Yu 2025-03-03 00:43:14 -08:00
• b87c21fc89 [Misc][Platform] Move use allgather to platform (#14010) Mengqing Cao 2025-03-03 15:40:04 +08:00
• e584b85afd [Misc] duplicate code in deepseek_v2 (#14106) wang.yuqi 2025-03-03 14:10:11 +08:00
• 09e56f9262 [Bugfix] Explicitly include "omp.h" for MacOS to avoid installation failure (#14051) Sheng Yao 2025-03-03 09:35:01 +08:00
• cf069aa8aa Update deprecated Python 3.8 typing (#13971) Harry Mellor 2025-03-03 01:34:51 +00:00
• bf33700ecd [v0][structured output] Support reasoning output (#12955) Ce Gao 2025-03-03 03:49:42 +08:00
• bc6ccb9878 [Doc] Source building add clone step (#14086) qux-bbb 2025-03-02 18:59:50 +08:00
• 82fbeae92b [Misc] Accurately capture the time of loading weights (#14063) Jun Duan 2025-03-01 20:20:30 -05:00
• cc5e8f6db8 [Model] Add LoRA support for TransformersModel (#13770) Jee Jee Li 2025-03-02 09:17:34 +08:00
• d54990da47 [v1] Add __repr__ to KVCacheBlock to avoid recursive print (#14081) Chen Zhang 2025-03-02 04:46:02 +08:00
• b9f1d4294e [v1][Bugfix] Only cache blocks that are not in the prefix cache (#14073) Chen Zhang 2025-03-01 16:25:54 +08:00
• b28246f6ff [ROCm][V1][Bugfix] Add get_builder_cls method to the ROCmAttentionBackend class (#14065) Sage Moore 2025-02-28 23:18:32 -08:00
• 3b5567a209 [V1][Minor] Do not print attn backend twice (#13985) Woosuk Kwon 2025-02-28 23:09:14 -08:00
• fdcc405346 [Doc] Consolidate whisper and florence2 examples (#14050) Isotr0py 2025-03-01 14:49:15 +08:00
• 8994dabc22 [Documentation] Add more deployment guide for Kubernetes deployment (#13841) Kuntai Du 2025-02-28 22:44:24 -08:00
• 02296f420d [Bugfix][V1][Minor] Fix shutting_down flag checking in V1 MultiprocExecutor (#14053) Li, Jiang 2025-03-01 14:31:01 +08:00
• 6a92ff93e1 [Misc][Kernel]: Add GPTQAllSpark Quantization (#12931) YajieWang 2025-03-01 14:30:59 +08:00
• 6a84164add [Bugfix] Add file lock for ModelScope download (#14060) Jee Jee Li 2025-03-01 14:10:28 +08:00
• f64ffa8c25 [Docs] Add pipeline_parallel_size to optimization docs (#14059) Brayden Zhong 2025-03-01 00:43:54 -05:00
• bd56c983d6 [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902) Luka Govedič 2025-02-28 18:20:11 -05:00
• 084bbac8cc [core] Bump ray to 2.43 (#13994) Rui Qiao 2025-02-28 13:47:44 -08:00
• 28943d36ce [v1] Move block pool operations to a separate class (#13973) Chen ZhangandCody Yu 2025-03-01 04:53:31 +08:00
• b526ca6726 Add RELEASE.md (#13926) Andrey Talman 2025-02-28 20:25:50 +00:00
• e7bd944e08 [v1] Cleanup the BlockTable in InputBatch (#13977) Chen Zhang 2025-03-01 03:03:16 +08:00
• c3b6559a10 [V1][TPU] Integrate the new ragged paged attention kernel with vLLM v1 on TPU (#13379) iefgnoixandmgoin 2025-02-28 10:01:36 -08:00
• 4be4b26cb7 Fix entrypoint tests for embedding models (#14052) Harry Mellor 2025-02-28 16:56:44 +00:00
• 2aed2c9fa7 [Doc] Fix ROCm documentation (#14041) Brayden Zhong 2025-02-28 11:42:07 -05:00
• 9b61dd41e7 [Bugfix] Initialize attention bias on the same device as Query/Key/Value for QwenVL Series (#14031) Yang Liu 2025-02-28 23:36:08 +08:00
• f7bee5c815 [VLM][Bugfix] Enable specifying prompt target via index (#14038) Cyrus Leung 2025-02-28 23:35:55 +08:00
• e0734387fb [Bugfix] Fix MoeWNA16Method activation (#14024) Jee Jee Li 2025-02-28 23:22:42 +08:00
• f58f8b5c96 Update AutoAWQ docs (#14042) Harry Mellor 2025-02-28 15:20:29 +00:00
• b3f7aaccd0 [V1][Minor] Restore V1 compatibility with LLMEngine class (#13090) Thibault Schueller 2025-02-28 09:52:25 +01:00
• b91660ddb8 [Hardware][Intel-Gaudi] Regional compilation support (#13213) Kacper Pietkun 2025-02-28 09:51:49 +01:00
• 76c89fcadd Use smaller embedding model when not testing model specifically (#13891) Harry Mellor 2025-02-28 08:50:43 +00:00