iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
2339 Commits
Author SHA1 Message Date
Woosuk Kwon 9ab4388cd3 [Model Runner V2] Support FLASHINFER_MLA backend (#32709) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-20 10:26:17 -08:00
杨朱 · KikiandClaude Haiku 4.5 bb9172030e [Metrics] Complete removal of deprecated vllm:time_per_output_token_seconds metric (#32661) 
This PR completes the removal of the deprecated vllm:time_per_output_token_seconds
metric that was deprecated in v0.11, hidden in v0.12, scheduled for removal in v0.13,
but delayed until v0.15.

Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-01-20 12:28:41 +00:00
Walter Beller-Morales 8be263c3fb [Core] Cleanup shm based object store on engine shutdown (#32429) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
 2026-01-20 08:53:37 +00:00
Woosuk Kwon e9c83cdc51 [Model Runner V2] Skip kernel launch for penalties & logit_bias (#32634) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-19 22:20:19 -08:00
Cyrus Leung 4753f3bf69 [Model] Use context managers for encoder- and LM-only mode (#32605) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-20 11:43:38 +08:00
Woosuk Kwon 6c01ffb897 [Model Runner V2] Decouple temperature from penalties (#32629) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-19 19:13:24 -08:00
Woosuk Kwon 7b7cdce968 [Model Runner V2] Refactor get_cudagraph_and_dp_padding (#32625) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-19 18:25:02 -08:00
Woosuk Kwon 05dc4bfab6 [Model Runner V2] Initialized communication buffer for DP (#32624) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-19 17:27:06 -08:00
Woosuk Kwon 43fada5360 [Model Runner V2] Refactor dummy_run (#32533) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-19 14:50:59 -08:00
Tomas Ruiz 4a5299c93f feat: spec decode with draft models (#24322) 
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
 2026-01-19 16:05:46 -05:00
Nicolò Lucchesi 74c583bc50 [Core] Whisper support torch.compile (#30385) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-01-19 10:02:31 +00:00
Woosuk Kwon 9a1f16da1e [Model Runner V2] Refactor update_states (#32562) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-18 17:32:42 -08:00
Woosuk Kwon bb1848cd62 [Model Runner V2] Support VLM (#32546) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-18 16:58:51 -08:00
Vadim Gimpelson 6101a26dc9 [BUGFIX] Fix degenerate strides in TRTLLM query tensors for FlashInfer backend. Fixes issue #32353 (#32417) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-01-18 16:57:32 -08:00
Wentao Ye 16de822c71 [Refactor] Remove unused file pallas_kv_cache_update.py (#32433) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-18 12:46:39 -08:00
Li Xie c826c72a96 [Model] Support Step1 Model (#32511) 
Signed-off-by: xieli <xieli@stepfun.com>
 2026-01-18 10:20:46 +00:00
Woosuk Kwon 963dc0b865 [Model Runner V2] Minor optimization for eagle input processing (#32535) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-17 21:55:17 -08:00
Isotr0py 8cc26acd8b [Performance] Improve Triton prefill attention kernel's performance (#32403) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-17 20:19:59 -08:00
Woosuk Kwon 4147910f1e [Model Runner V2] Move mrope_positions buffer to MRopeState (#32532) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-17 20:09:48 -08:00
Guofang.TangandGuofang Tang 2b99f210f5 [Misc] Fix typo: seperator -> separator in flashmla_sparse.py (#32411) 
Signed-off-by: Guofang Tang <tinggofun@gmail.com>
Co-authored-by: Guofang Tang <tinggofun@gmail.com>
 2026-01-17 12:18:30 +00:00
Chenyaaang 484e22bc18 [TPU][Core] Enable Pipeline Parallelism on TPU backend (#28506) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2026-01-16 15:29:20 -08:00
Hongxin Xuandarlenxu 03da3b52ef [Bugfix] Refactor to support DP parallel in R3 (#32306) 
Signed-off-by: xhx1022 <1737006628@qq.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
 2026-01-16 15:13:58 +08:00
Matthias Gehre 047413375c [Attention][AMD] Make flash-attn optional (#30361) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-01-15 17:18:24 +00:00
Wentao Ye b34474bf2c [Feature] Support async scheduling + PP (#32359) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-15 12:06:23 -05:00
Woosuk Kwon 6218034dd7 [Model Runner V2] Support FlashInfer backend & Fix CUDA Graph bug [1/2] (#32348) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-15 08:59:23 -08:00
Pleaplusone 77c16df31d [ROCm][Bugfix] Disable hip sampler to fix deepseek's accuracy issue on ROCm (#32413) 
Signed-off-by: ganyi <ygan@amd.com>
 2026-01-15 16:35:47 +00:00
Pleaplusone 130d6c9514 [ROCm][Perf] Enable shuffle kv cache layout and assembly paged attention kernel for AiterFlashAttentionBackend (#29887) 
Signed-off-by: ganyi <ygan@amd.com>
 2026-01-15 15:29:53 +00:00
Cyrus Leung cbbae38f93 [2/N] Move cache factories to MM registry (#32382) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-15 01:02:30 -08:00
dtc 1e584823f8 [Bugfix] Strengthen the check of X-data-parallel-rank in Hybrid LB mode (#32314) 
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
 2026-01-15 16:31:16 +08:00
Ofir Zafrir e9ec2a72d8 [Bugfix] Fix stale common_attn_metadata.max_seq_len in speculative decoding with Eagle (#32312) 
Signed-off-by: Ofir Zafrir <ofir.zafrir@intel.com>
 2026-01-15 06:39:37 +00:00
Ning Xie 9d7ae3fcdb [code clean] remove duplicate check (#32376) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-01-15 05:29:34 +00:00
Lumosis 66652e8082 [BugFix] Assign page_size_padded when unifying kv cache spec. (#32283) 
Signed-off-by: Lihao Ran <imlihao.ran@gmail.com>
 2026-01-14 20:10:01 +00:00
vllmellm e27078ea80 [Bugfix][ROCm][performance] Resolve the performance regression issue of the Qwen3-Next-80B-A3B-Thinking under rocm_atten (#32336) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-01-14 19:32:48 +00:00
Cyrus Leung 9ea07b41da [1/N] Reorganize multimodal processing code (#32327) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-14 15:25:31 +00:00
Shanshan Shen ce0946249d [Misc] Make mem utils can be reused by other platforms (#32322) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-01-14 03:46:01 -08:00
Cyrus Leung 90db5b31e4 [Refactor] Move top-level dummy data generation to registry (#32310) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-14 02:17:46 -08:00
Angela Yi 7933638051 [misc] Remove is_torch_equal_or_newer(2.4) cases (#32296) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-13 23:22:07 -08:00
Woosuk Kwon 90c0836902 [Model Runner V2] Refactor Sampler (#32245) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-13 17:58:12 -08:00
Matthew BonanniandLuka Govedič 2263d44b68 [4/N][Attention] Move MLA common to model_executor (#32060) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-13 09:08:45 -08:00
Matthew Bonanni 98f60e5acb [6/N][Attention] Move utils to more appropriate locations (#32215) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-13 05:38:52 -08:00
Mickaël SeznecRobert ShawMichael Goin
 		a5bbbd2f24 [Quantization] fix: overflow with static per-tensor scaling (#29867) 
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-13 12:56:01 +00:00
Cyrus Leung eb28e8068d [Refactor] Remove get_encoder_dummy_data (#32241) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-13 09:21:23 +00:00
Wentao Ye 2a719e0865 [Perf] Optimize requests abort (#32211) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-13 04:11:37 +00:00
cjackal 15b33ff064 [Misc] improve warning/assert messages (#32226) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-01-13 03:11:23 +00:00
Nick Hill c6bb5b5603 [BugFix] Fix engine crash caused by chat tools + response_format (#32127) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-13 10:33:14 +08:00
Woosuk Kwon 0a7dd23754 [Model Runner V2] Add support for M-RoPE (#32143) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-12 13:37:43 -08:00
Woosuk Kwon dec28688c5 [Model Runner V2] Minor refactor for logit_bias (#32209) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-12 13:08:30 -08:00
Woosuk Kwon ca81811bfe [Model Runner V2] Support logit_bias, allowed_token_ids, min_tokens (#32163) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-12 11:31:10 -08:00
Nicolò Lucchesi 08e8e99ce7 [Misc] Change log level for batch queue log (#32192) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-01-12 18:59:31 +00:00
Or Ozeri 2be765b68a [BugFix] scheduler: Fix ordering preserving of skipped requests (#32173) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-12 18:39:38 +00:00