Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 015815fe01 [Bugfix] use_fast failing to be propagated to Qwen2-VL image processor (#17838) Cyrus Leung 2025-05-08 20:39:21 +08:00
• e4ca6e3a99 Fix transient dependency error in docs build (#17848) Harry Mellor 2025-05-08 11:42:03 +01:00
• 53d0cb7423 [Misc] add chatbox integration (#17828) Reidandreidliu41 2025-05-08 18:05:26 +08:00
• f50dcb7c21 [Easy] Eliminate c10::optional usage in vllm/csrc (#17819) Lu Fang 2025-05-08 03:05:10 -07:00
• a1e19b635d [Doc] Fix a typo in the file name (#17836) Cyrus Leung 2025-05-08 18:04:18 +08:00
• bb239a730f [Bugfix] Fix quark fp8 format loading on AMD GPUs (#12612) fxmarty-amdandkewang2 2025-05-08 11:53:53 +02:00
• a463555dee [TPU] Fix the test_sampler (#17820) Jevin Jiang 2025-05-08 02:51:33 -07:00
• ca04b97c93 [Bugfix] Fix tool call template validation for Mistral models (#17644) Rick YuanandAaron Pham 2025-05-08 17:47:19 +08:00
• 0a9bbaa104 [Misc] support model prefix & add deepseek vl2 tiny fused moe config (#17763) xsankand唯勤 2025-05-08 15:50:22 +08:00
• 39956efb3f [Bugfix] Fix bad words for Mistral models (#17753) Qiong Zhou Huang 2025-05-07 23:32:10 -07:00
• 597051e56f [Qwen3]add qwen3-235b-bf16 fused moe config on A100 (#17715) Ximingwang-09 2025-05-08 14:09:32 +08:00
• 96722aa81d [Frontend] Chat template fallbacks for multimodal models (#17805) Cyrus Leung 2025-05-08 14:05:54 +08:00
• 843b222723 [Hardware][Intel-Gaudi] Support Automatic Prefix Caching on HPU (#17648) Agata Dobrzyniewicz 2025-05-08 07:37:03 +02:00
• e515668edf [Hardware][Power] Enable compressed tensor W8A8 INT8 quantization for POWER (#17153) 3 people 2025-05-08 11:05:03 +05:30
• 5a499e70d5 [Kernel][Hardware][AMD] Bf16 mfma opt for ROCm skinny GEMMs (#17071) Hashem Hashemiandcharlifu 2025-05-07 22:34:49 -07:00
• 6930a41116 [V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (#17490) Russell BryantandNick Hill 2025-05-08 01:34:02 -04:00
• 998eea4a0e Only log non-default CLI args for online serving (#17803) Harry Mellor 2025-05-08 06:33:29 +01:00
• c747d84576 [Installation] OpenTelemetry version update (#17771) Mikhail Podvitskii 2025-05-08 07:32:49 +02:00
• b2da14a05a Improve exception reporting in MP engine (#17800) Vadim Markovtsev 2025-05-08 07:32:39 +02:00
• 7ea2adb802 [Core] Support full cuda graph in v1 (#16072) Chanh NguyenandChanh Nguyen 2025-05-07 22:30:15 -07:00
• 3d13ca0e24 [BugFix] Fix --disable-log-stats in V1 server mode (#17600) Nick Hill 2025-05-07 21:08:15 -07:00
• 66ab3b13c9 Don't call the venv vllm (#17810) Harry Mellor 2025-05-08 05:06:39 +01:00
• a8238bbdb0 [Chore][Doc] uses model id determined from OpenAI client (#17815) Aaron Pham 2025-05-07 21:48:57 -04:00
• d43f914d42 [Core][Feature] Input metadata dump on crash (#13407) Wallas Henrique 2025-05-07 19:15:09 -03:00
• ed5272cf21 [BugFix] Avoid secondary missing MultiprocExecutor.workers error (#17811) Nick Hill 2025-05-07 14:55:04 -07:00
• c20ef40fd0 [Hardware][TPU][V1] Multi-LoRA implementation for the V1 TPU backend (#14238) Akshat TripathiandChengji Yao 2025-05-07 21:28:47 +01:00
• db593aa67f [Quantization] Quark MXFP4 format loading (#16943) Bowen Bao 2025-05-07 12:05:05 -07:00
• f98e307588 [Bugfix] Fix missing lora name mapping for lora without prefix (#17793) Isotr0py 2025-05-08 00:17:12 +08:00
• 646a31e51e Fix and simplify deprecated=True CLI kwarg (#17781) Harry Mellor 2025-05-07 16:51:06 +01:00
• be8ff88e66 [Bugfix] Fix Video IO error for short video (#17791) Isotr0py 2025-05-07 23:36:06 +08:00
• 1a6af1453d Only depend on importlib-metadata for Python < 3.10 (#17776) Christian Heimes 2025-05-07 16:51:06 +02:00
• 32aa74c09c [ROCm][FP8][Kernel] FP8 quantization fused into Custom Paged Attention (#17139) Gregory Shtrasberg 2025-05-07 10:12:35 -04:00
• 7377dd0307 [doc] update the issue link (#17782) Reidandreidliu41 2025-05-07 20:29:05 +08:00
• 98c89e16ff Make key optional for rotary embedding (#17566) Yong Hoon Shin 2025-05-07 00:11:46 -07:00
• 324a3119b0 Fix test_memory_usage_no_spec (#17754) Yong Hoon Shin 2025-05-07 00:10:33 -07:00
• 8a15c2603a [Frontend] Add missing chat templates for various MLLMs (#17758) Cyrus Leung 2025-05-07 15:10:01 +08:00
• 043e4c4955 Add NeuronxDistributedInference support, Speculative Decoding, Dynamic on-device sampling (#16357) +111 people 2025-05-07 00:07:30 -07:00
• ba7703e659 [Misc] Remove qlora_adapter_name_or_path (#17699) Jee Jee Li 2025-05-07 14:10:37 +08:00
• f80ae5bdcf [Kernel] Use fused rmsnorm for some models like qwen3 series (#17735) Wanrui Daiandevian 2025-05-07 14:10:02 +08:00
• 1a45a61387 [Kernel] GGUF MoeVec kernel (#16780) Szymon OżógandIsotr0py 2025-05-07 14:07:23 +08:00
• c3e9d5060e [Misc] Use apply_rotary_emb from vllm_flash_attn for Qwen2-VL vision RoPE (#17726) Isotr0py 2025-05-07 12:51:33 +08:00
• 822de7fb94 [Misc] Split model loader (#17712) Jee Jee Li 2025-05-07 12:42:26 +08:00
• 8d84d836d1 [BugFix][Spec Decode] Fix hidden size mismatch between target and eagle head (#17740) Woosuk Kwon 2025-05-06 19:51:26 -07:00
• 950b71186f Replace lm-eval bash script with pytest and use enforce_eager for faster CI (#17717) Michael Goin 2025-05-06 21:00:10 -04:00
• e50a1f1a9c [TPU] Add kernel test for moe_pallas (#17496) Michael Goin 2025-05-06 20:59:57 -04:00
• a17cef70ea Removed unused marlin cuda code (#17684) Michael Goin 2025-05-06 20:59:47 -04:00
• 18dd5e01f2 [Model] Mamba2 causal conv1d Refactor to Split Prefill and Decode Requests for Corresponding Kernels (#17146) Chih-Chieh Yang 2025-05-06 20:59:30 -04:00
• 6de3e13413 Add logging for torch nightly version (#17669) Yang Wang 2025-05-06 17:45:51 -07:00
• ed3a1d2106 [ROCm] fix num_stages for default moe config to avoid triton OutOfResource error (#17744) Hongxia Yang 2025-05-06 20:39:48 -04:00
• 022afbeb4e Fix doc build performance (#17748) Harry Mellor 2025-05-07 01:36:41 +01:00
• 2f925e5777 [Kernel] Unified Triton kernel that doesn't distinguish between prefill + decode (#16828) Thomas ParnellandLucas Wilkinson 2025-05-06 18:21:48 -04:00
• de906b95f9 [Bugfix] Fix for the condition to accept empty encoder inputs for mllama (#17732) Gregory Shtrasberg 2025-05-06 15:59:06 -04:00
• d456aea71f [Misc] Add Next Edit Prediction (NEP) datasets support in benchmark_serving.py (#16839) d.transposedanddtransposed 2025-05-06 21:38:45 +02:00
• 621ca2c0ab [TPU] Increase block size and reset block shapes (#16458) Jevin Jiang 2025-05-06 10:55:04 -07:00
• 6115b11582 Make right sidebar more readable in "Supported Models" (#17723) Harry Mellor 2025-05-06 17:48:26 +01:00
• 5b8c390747 [Bugfix] Fix modality limits in vision language example (#17721) Cyrus Leung 2025-05-07 00:12:28 +08:00
• 7525d5f3d5 [doc] Add RAG Integration example (#17692) Reidandreidliu41 2025-05-07 00:10:23 +08:00
• aabcd2cae3 [v1] Introduce KVCacheBlocks as interface between Scheduler and KVCacheManager (#17479) Chen Zhang 2025-05-06 23:50:34 +08:00
• 0d115460a7 [Docs] Use gh-file to add links to tool_calling.md (#17709) Michael Yao 2025-05-06 23:27:19 +08:00
• 175bda67a1 [Feat] Add deprecated=True to CLI args (#17426) Aaron Pham 2025-05-06 11:11:27 -04:00
• cba31c47c4 [v1] AttentionMetadata for each layer (#17394) Chen Zhang 2025-05-06 22:58:37 +08:00
• a6fed02068 [V1][PP] Support PP for MultiprocExecutor (#14219) Li, Jiang 2025-05-06 22:58:05 +08:00
• d419aa5dc4 [V1] Enable TPU V1 backend by default (#17673) Michael Goin 2025-05-06 09:49:49 -04:00
• f9bc5a0693 [Bugfix] Fix triton import with local TritonPlaceholder (#17446) Mengqing Cao 2025-05-06 17:53:09 +08:00
• 05e1f96419 Fix dockerfilegraph pre-commit hook (#17698) Harry Mellor 2025-05-06 09:56:48 +01:00
• 6eae34533a [Misc] Fix ScalarType float4 naming (#17690) Lucas Wilkinson 2025-05-06 04:07:15 -04:00
• 63ced7b43f [Doc] Update notes for H2O-VL and Gemma3 (#17219) Cyrus Leung 2025-05-06 15:51:02 +08:00
• dc47ba32f8 [Bugfix] Fixed prompt length for random dataset (#17408) Mikhail Podvitskii 2025-05-06 09:00:08 +02:00
• edbf2d609e [easy] Fix logspam on PiecewiseBackend errors (#17138) Richard Zou 2025-05-06 02:46:11 -04:00
• 999328be0d [Model] Add GraniteMoeHybrid 4.0 model (#17497) 4 people 2025-05-06 06:00:31 +02:00
• 98834fefaa Update nm to rht in doc links + refine fp8 doc (#17678) Michael Goin 2025-05-05 20:41:14 -04:00
• 90bd2ae172 [Bugfix] LoRA - Retire unused maxnreg LoRA kernel argument (#17677) Varun Sundar Rabindranath 2025-05-06 06:04:29 +05:30
• 5941e0b7ea [TPU][V1] Add support for top-logprobs (#17072) Nicolò Lucchesi 2025-05-05 23:20:15 +02:00
• 9765940824 [TPU] Enable gemma3-27b with TP>1 on multi-chips. (#17335) XiongfeiWei 2025-05-05 14:19:58 -07:00
• 5ea5c514da [BugFix] Increase timeout for startup failure test (#17642) Nick Hill 2025-05-05 13:53:19 -07:00
• d3efde8176 [Benchmarks] Remove invalid option under V1 engine (#17651) Russell Bryant 2025-05-05 16:30:22 -04:00
• aea302be6c Use git-path commit in hook (#17616) Thomas J. Fan 2025-05-05 13:55:32 -04:00
• cc05b90d86 [Doc] Fix broken cuda installation doc rendering (#17654) Isotr0py 2025-05-06 01:52:40 +08:00
• 1d0c9d6b2d [Kernel] some optimizations for dense marlin and moe marlin (#16850) Jinzhen Lin 2025-05-06 00:39:30 +08:00
• f62cad6431 [Build/CI] Upgrade CUTLASS to 3.9.2 (#17641) Tyler Michael Smith 2025-05-04 22:23:17 -04:00
• 5394ad7387 [Bugfix] fix KeyError on top logprobs are special tokens (#17637) Chauncey 2025-05-05 10:22:35 +08:00
• 68e1ee0072 [Bugfix][Easy] Fix whitespace in shm_broadcast.py logging (#17635) Tyler Michael Smith 2025-05-04 22:20:19 -04:00
• 2858830c39 [Bugfix] Prioritize dtype in root config before checking text config (#17629) Cyrus Leung 2025-05-04 20:43:05 +08:00
• d6484ef3c3 Add full API docs and improve the UX of navigating them (#17485) Harry Mellor 2025-05-04 03:42:43 +01:00
• 46fae69cf0 [Misc] V0 fallback for --enable-prompt-embeds (#17615) Cyrus Leung 2025-05-04 06:59:24 +08:00
• f66f1e0fa3 [Bugfix] Fix broken Qwen2.5-omni tests (#17613) Isotr0py 2025-05-04 01:08:14 +08:00
• 887d7af882 [Core] Gate prompt_embeds behind a feature flag (#17607) Cyrus Leung 2025-05-04 00:19:20 +08:00
• a92842454c [Bugfix][ROCm] Using device_type because on ROCm the API is still torch.cuda (#17601) Gregory Shtrasberg 2025-05-03 01:25:47 -04:00
• c8386fa61d [Build/CI] Upgrade CUTLASS to 3.9.1 (#17602) Tyler Michael Smith 2025-05-03 01:25:14 -04:00
• 87baebebd8 [Frontend][TPU] Add TPU default max-num-batched-tokens based on device name (#17508) Chenyaaang 2025-05-02 21:42:44 -07:00
• e3d0a1d190 [Quantizaton] [AMD] Add support for running DeepSeek int8 w8a8 MoE on ROCm (#17558) rasmith 2025-05-02 23:41:10 -05:00
• d47b605eca Update test requirements to CUDA 12.8 (#17576) 22quinn 2025-05-02 21:40:15 -07:00
• 22c6f6397f [Neuron][Build] Require setuptools >= 77.0.3 for PEP 639 (#17603) Liangfu Chen 2025-05-02 19:41:59 -07:00
• 3ec97e2cc5 [release] Add command to clean up Docker containers/images in TPU release machine (#17606) Kevin H. Luu 2025-05-02 18:54:34 -07:00
• 9b103a1d76 fix typo in logging (#17605) Eric Hartford 2025-05-02 21:04:40 -04:00
• b90b0852e9 [easy] Print number of needed GPUs in skip message (#17594) Richard Zou 2025-05-02 18:27:43 -04:00
• 9352cdb56d [Hardware][AMD] Improve OAM device ID + llama4 Maverick MOE tuning (#16263) Xiaodong WangandLu Fang 2025-05-02 12:44:19 -07:00
• 182f40ea8b Add NVIDIA TensorRT Model Optimizer in vLLM documentation (#17561) Zhiyu 2025-05-02 11:36:46 -07:00
• 3e887d2e0c permute/unpermute kernel for moe optimization (#14568) Caleb_Du 2025-05-03 02:31:55 +08:00
• 0f87d8f7b2 [BugFix][Attention] Fix sliding window attention in V1 giving incorrect results (#17574) Lucas Wilkinson 2025-05-02 14:01:38 -04:00