This website requires JavaScript.
06ffc7e1d3
[Misc][ROCm] Exclude cutlass_mla_decode for ROCm build (#17289 )
TY-AMD
2025-04-30 01:26:42 +08:00
d3cf61b89b
fix gemma3 results all zero (#17364 )
Qiming Zhang
2025-04-29 09:40:25 -07:00
a39203f99e
[Bugfix] add qwen3 reasoning-parser fix content is None when disable … (#17369 )
mofanke
2025-04-30 00:32:40 +08:00
24e6ad3f16
[V1] Remove num_input_tokens from attn_metadata (#17193 )
Chen Zhang
2025-04-30 00:28:41 +08:00
2ef5d106bb
Improve literal dataclass field conversion to argparse argument (#17391 )
Harry Mellor
2025-04-29 17:25:08 +01:00
0ed27ef66c
Fix: Spelling of inference (#17387 )
a2q1p
2025-04-30 00:23:39 +08:00
900edfa8d4
Transformers backend tweaks (#17365 )
Harry Mellor
2025-04-29 17:08:03 +01:00
88ad9ec6b2
[Frontend] Support chat_template_kwargs in LLM.chat (#17356 )
Cyrus Leung
2025-04-29 22:03:35 +08:00
40896bdf3f
pre-commit autoupdate (#17380 )
Harry Mellor
2025-04-29 14:46:55 +01:00
00ee37efa2
[Bugfix] Clean up MiniMax-VL and fix processing (#17354 )
Cyrus Leung
2025-04-29 20:42:16 +08:00
890f104cdf
[Doc] Fix QWen3MOE info (#17381 )
Jee Jee Li
2025-04-29 20:38:32 +08:00
4a5e13149a
Update docs requirements (#17379 )
Harry Mellor
2025-04-29 12:35:47 +01:00
97cc8729f0
[Model] Ignore rotary embed load for Cohere model (#17319 )
Ekagra Ranjan
2025-04-29 03:30:40 -04:00
4464109219
[Build][Bugfix] Restrict setuptools version to <80 (#17320 )
Gregory Shtrasberg
2025-04-29 03:17:23 -04:00
193e78e35d
[Fix] Documentation spacing in compilation config help text (#17342 )
Hyogeun Oh (오효근)
2025-04-29 16:16:17 +09:00
bdb2cddafc
[Misc]Use a platform independent interface to obtain the device attributes (#17100 )
ponix-j
2025-04-29 14:59:13 +08:00
ebb3930d28
[Misc] Move config fields to MultiModalConfig (#17343 )
Cyrus Leung
2025-04-29 14:37:21 +08:00
cde384cd92
[Model] support MiniMax-VL-01 model (#16328 )
qscqesze
2025-04-29 12:05:50 +08:00
96e06e3cb7
[Misc] Add a Jinja template to support Mistral3 function calling (#17195 )
Chauncey
2025-04-29 10:53:44 +08:00
17eb306fcc
[Bugfix] Add contiguous call inside rope kernel wrapper (#17091 )
Zhengyuan Su (苏政渊) and 苏政渊
2025-04-29 10:24:07 +08:00
165cb56329
Ignore '<string>' filepath (#17330 )
Richard Zou
2025-04-28 22:23:29 -04:00
d6da8a8ff2
[Bugfix] Fix numel() downcast in fused_layernorm_dynamic_per_token_quant.cu (#17316 )
Richard Barnes
2025-04-28 19:23:18 -07:00
b4ac4fa04d
[model] make llama4 compatible with pure dense layers (#17315 )
Lucia Fang
2025-04-28 19:22:22 -07:00
e136000595
[V1][Spec Decode] Make Eagle model arch config driven (#17323 )
Ekagra Ranjan
2025-04-28 22:22:02 -04:00
86d9fc29cb
implement Structural Tag with Guidance backend (#17333 )
Michał Moskal
2025-04-28 19:21:32 -07:00
506475de5f
[Optim] Compute multimodal hash only once per item (#17314 )
Cyrus Leung
2025-04-29 09:40:35 +08:00
cfe4532093
[Benchmark] Add single turn MTBench to Serving Bench (#17202 )
Ekagra Ranjan
2025-04-28 19:46:15 -04:00
8fc88d63f1
[Model] Add tuned triton fused_moe configs for Qwen3Moe (#17328 )
Michael Goin
2025-04-28 16:20:24 -06:00
6e74fd4945
Support loading transformers models with named parameters (#16868 )
Alex Wu
2025-04-28 15:15:58 -07:00
dcbac4cb4b
[Model] Qwen3 Dense FP8 Compat Fixes (#17318 )
Simon Mo
2025-04-28 14:12:01 -07:00
ed2462030f
[Bugfix] Fix moe weight losing all extra attrs after process_weights_after_loading. (#16854 )
Charlie Fu
2025-04-28 16:05:07 -05:00
cc5befbced
[BugFix] Fix cascade attention - RuntimeError: scheduler_metadata must have shape (metadata_size) (#17283 )
Lucas Wilkinson
2025-04-28 16:55:50 -04:00
2c89cd96a8
[Chore] cleanup license indicators in light of SPDX (#17259 )
Aaron Pham and Russell Bryant
2025-04-28 15:43:52 -04:00
a0304dc504
[Security] Don't bind tcp zmq socket to all interfaces (#17197 )
Russell Bryant
2025-04-28 13:08:20 -04:00
c7941cca18
Explicitly explain quant method override ordering and ensure all overrides are ordered (#17256 )
Harry Mellor
2025-04-28 17:55:31 +01:00
b6dd32aa07
Make name of compressed-tensors quant method consistent across vLLM (#17255 )
Harry Mellor
2025-04-28 17:28:13 +01:00
f94886946e
Improve conversion from dataclass configs to argparse arguments (#17303 )
Harry Mellor
2025-04-28 17:22:12 +01:00
72dfe4c74f
[Docs] Add a security guide (#17230 )
Russell Bryant and Cyrus Leung
2025-04-28 11:12:17 -04:00
8b464d9660
[Misc] Clean up Qwen2.5-Omni code (#17301 )
Cyrus Leung
2025-04-28 21:20:45 +08:00
889ebb2638
[Misc] Minor typo/grammar in platforms/interface.py (#17307 )
Nicolò Lucchesi
2025-04-28 14:45:42 +02:00
3ad986c28b
[doc] update wrong model id (#17287 )
Reid and reidliu41
2025-04-28 19:20:51 +08:00
344e193b7d
[Bugfix] Add missing get_language_model to new MLLMs (#17300 )
Cyrus Leung
2025-04-28 19:09:57 +08:00
fb1c933ade
Add missing class docstring for PromptAdapterConfig (#17302 )
Harry Mellor
2025-04-28 12:06:59 +01:00
72c5b97231
Update tpu_worker.py 's typo (#17288 )
idouba
2025-04-28 19:01:15 +08:00
fa93cd9f60
[Model] Add Granite Speech Support (#16246 )
Alex Brooks
2025-04-28 04:05:00 -06:00
aec9674dbe
[Core] Remove legacy input mapper/processor from V0 (#15686 )
Cyrus Leung
2025-04-28 15:38:48 +08:00
7fcc4223dc
[Minor][Models] Pass partial_rotary_factor parameter to rope (#17266 )
Wanrui Dai and evian
2025-04-28 12:28:59 +08:00
8262a3e23b
[Misc] Validate stop_token_ids contents (#17268 )
Nick Hill
2025-04-27 20:54:05 -07:00
f211331c48
[Doc] small fix (#17277 )
Reid and reidliu41
2025-04-28 11:53:35 +08:00
9053d0b134
[Doc] Fix wrong github link in LMCache examples (#17274 )
Kuntai Du
2025-04-27 20:09:11 -07:00
cb3f2d8d10
[Bugfix] Fix Mistral3 spatial merge error (#17270 )
Michael Goin
2025-04-27 20:40:05 -06:00
c12df53b60
[Bugfix] Fix cutlass dispatch for fp8/int8 to properly invoke M<=16 c… (#16751 )
TherLF
2025-04-28 10:38:42 +08:00
d1aeea7553
[Bugfix] Fix missing ARG in Dockerfile for arm64 platforms (#17261 )
Lennart K. M. Schulz
2025-04-28 04:38:14 +02:00
d8bccde686
[BugFix] Fix vllm_flash_attn install issues (#17267 )
2025-04-27 20:27:56 -04:00
20e489eaa1
[V1][Spec Decode] Make eagle compatible with prefix caching. (#17137 )
Lily Liu
2025-04-27 09:29:43 -07:00
4213475ec7
[Metrics] Fix minor inconsistencies in bucket progression (#17262 )
Cyrus Leung
2025-04-28 00:19:39 +08:00
d92879baf6
[doc] Add feature status legend (#17257 )
Reid and reidliu41
2025-04-27 23:17:02 +08:00
690fe019f0
[Feature] support sequence parallelism using compilation pass (#16155 )
cascade and Tyler Michael Smith
2025-04-27 06:29:35 -07:00
ed7a29d9f8
[NVIDIA] Support Cutlass MLA for Blackwell GPUs (#16032 )
Kaixi Hou
2025-04-27 06:29:21 -07:00
756848e79e
[Bugfix] Fix Lora Name Parsing (#17196 )
Alex Brooks and Jee Jee Li
2025-04-27 06:33:09 -06:00
18445edd0f
[Misc] Change buckets of histogram_iteration_tokens to [1, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8096] to represent number of tokens (#17033 )
Flex Wang
2025-04-27 05:30:53 -07:00
30215ca61f
[MISC] Use string annotation types for class definitions (#17244 )
Jade Zheng
2025-04-27 16:39:57 +08:00
838cedade7
[Bugfix] Get a specific type of layer from forward context (#17222 )
Chen Zhang
2025-04-27 15:58:05 +08:00
4283a28c2f
[Bugfix] Fix QWen2 VL multimodal mapping (#17240 )
Jee Jee Li
2025-04-27 13:53:23 +08:00
93a126fbc7
[Misc] Make cached tokenizer pickle-compatible (#17048 )
Cyrus Leung
2025-04-27 13:05:00 +08:00
8e4b351a0c
[Kernel][Triton][FP8] Adding fp8 and variable length sequence support to Triton FAv2 kernel (#12591 )
rasmith
2025-04-26 19:35:08 -05:00
9869453c42
Update test_flash_attn.py (#17102 )
Happy
2025-04-27 06:17:35 +08:00
3642c59aa8
[CI/Build] remove -t for run-lm-eval-gsm-hf-baseline.sh (#16271 )
Reid and reidliu41
2025-04-27 02:25:05 +08:00
43eea2953b
[Minor] Fix lint error in main branch (#17233 )
Woosuk Kwon
2025-04-26 11:10:14 -07:00
de7eb10ce4
[Bugfix] Fix Qwen2.5-Omni M-RoPE position ids generation (#16878 )
Kero Liang
2025-04-27 01:41:35 +08:00
fd11a325b8
[MISC] rename interval to max_recent_requests (#14285 )
Ning Xie
2025-04-27 00:59:18 +08:00
4d17e20310
Disable the torch.compile cache checks when VLLM_DISABLE_COMPILE_CACHE=1 (#16573 )
Lu Fang
2025-04-26 09:17:58 -07:00
10fd1d7380
[Bugfix] fix error due to an uninitialized tokenizer when using skip_tokenizer_init with num_scheduler_steps (#9276 )
changjun.lee
2025-04-27 00:51:17 +09:00
52b4f4a8d7
[Docs] Update structured output doc for V1 (#17135 )
Russell Bryant
2025-04-26 11:12:18 -04:00
e782e0a170
[Chore] added stubs for vllm_flash_attn during development mode (#17228 )
Aaron Pham
2025-04-26 10:45:26 -04:00
dc2ceca5c5
[BUGFIX] use random for NONE_HASH only when PYTHONHASHSEED not set (#17088 )
Ning Xie
2025-04-26 22:34:24 +08:00
f8acd01ff7
[V1] Add structural_tag support using xgrammar (#17085 )
Russell Bryant
2025-04-26 10:06:37 -04:00
c48334d405
[Hardware][Intel-Gaudi] Update hpu-extension and update bucketing system for HPU device (#17186 )
Agata Dobrzyniewicz
2025-04-26 14:55:14 +02:00
909fdaf152
[Bugfix] Fix standard models tests (#17217 )
Cyrus Leung
2025-04-26 17:26:41 +08:00
8c1c926d00
[Bugfix] Fix missing int type for -n in multi-image example (#17223 )
Isotr0py
2025-04-26 16:49:52 +08:00
df6f3ce883
[Core] Remove prompt string from engine core data structures (#17214 )
Nick Hill
2025-04-25 23:41:05 -07:00
513f074766
[CI/test] Fix Eagle Correctness Test (#17209 )
Woosuk Kwon
2025-04-25 23:40:36 -07:00
b07bf83c7d
[BugFix] Avoid race conditions in zero-copy tensor transmission (#17203 )
Nick Hill
2025-04-25 23:00:07 -07:00
53e8cf53a4
[V1][Metrics] Allow V1 AsyncLLM to use custom logger (#14661 )
2025-04-25 22:05:40 -07:00
54271bb766
[ROCm][Misc] Follow-ups for Skinny Gemms on ROCm. (#17011 )
Charlie Fu
2025-04-26 00:05:10 -05:00
9e96f56efb
Allocate kv_cache with stride order (#16605 )
Shu Wang
2025-04-26 00:03:31 -05:00
b278911229
[Minor][Models] Fix Return Types of Llama & Eagle (#17220 )
Woosuk Kwon
2025-04-25 21:54:47 -07:00
7bd0c7745c
[Doc] Minor fix for the vLLM TPU setup page (#17206 )
yarongmu-google
2025-04-25 21:39:56 -07:00
1cf0719ebd
[Minor][Spec Decode] Add use_eagle to SpeculativeConfig (#17213 )
Woosuk Kwon
2025-04-25 21:08:15 -07:00
537d5ee025
[doc] add Anything LLM integration (#17216 )
Reid and reidliu41
2025-04-26 12:03:23 +08:00
c8e5be35f7
[MISC][AMD] Add unused annotation to rocm kernel file (#17097 )
Lu Fang
2025-04-25 20:33:35 -07:00
a6e72e1e4f
[Bugfix] [pytorch] Patch AOTAutogradCache._get_shape_env (#17142 )
James Wu
2025-04-25 23:28:20 -04:00
5e83a7277f
[v1] [P/D] Adding LMCache KV connector for v1 (#16625 )
Yihua Cheng
2025-04-25 22:03:38 -05:00
68af5f6c5c
[AMD][FP8][BugFix] Remove V1 check in arg_utils.py for FP8 since it is not necessary (#17215 )
rasmith
2025-04-25 21:55:05 -05:00
8de2901fea
[Bugfix] gemma[2,3] interleaved attention when sliding window is disabled (#17180 )
Chen Zhang
2025-04-26 10:53:51 +08:00
c53e0730cb
[Misc] Refine ray_serve_deepseek example (#17204 )
Rui Qiao
2025-04-25 16:06:59 -07:00
a0e619e62a
[V1][Spec Decode] EAGLE-3 Support (#16937 )
Benjamin Chislett and Bryan Lu
2025-04-25 18:43:07 -04:00
70116459c3
[BugFix][Frontend] Fix LLM.chat() tokenization (#16081 )
Nick Hill
2025-04-25 15:20:05 -07:00
65e262b93b
Fix Python packaging edge cases (#17159 )
Christian Heimes
2025-04-26 00:15:07 +02:00
43faa0461a
[Bugfix] Fix hybrid model tests (#17182 )
Cyrus Leung
2025-04-26 06:14:37 +08:00
TY-AMD