iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
17,598 Commits 3 Branches 0 Tags
521b88c29ef29b37efefa64efda7b59ec99d210c
Commit Graph
5912 Commits
Author SHA1 Message Date
WEI CHENG CHIUandClaude 521b88c29e [Bugfix] Reject structured outputs for diffusion decoders with a clear error (#45468) 
Signed-off-by: Wayne Chiu <waynehacking8@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-13 12:04:01 -07:00
Juan Pérez de Algaba 470229c37e [Security] Fix DoS via prompt_embeds on M-RoPE models (#45252) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-13 10:17:38 +00:00
Juan Pérez de AlgabaandHarry Mellor 2b3006076c [Security] Add timeout guard for regex compilation in structured outp… (#45118) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-13 09:52:56 +00:00
WEI CHENG CHIU 5b2943f5a6 [Bugfix] Return the tokenizer from maybe_make_thread_pool so it survives pickling (#45460) 
Signed-off-by: Wayne Chiu <waynehacking8@gmail.com>
 2026-06-13 06:01:35 +00:00
Martin HickeyandIsotr0py 43f0e024bc [Render] Add /derender endpoints for disaggregated postprocessing (#43606) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-13 13:55:33 +08:00
Andreas Karatzas 1033ffac2e [CI] Wait for SSL cert refresher events in the test (#45489) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-13 04:57:18 +00:00
WEI CHENG CHIU 17ee5b1ac5 [Bugfix] Set type/role explicitly in streaming message_start event (#45376) 
Signed-off-by: Wayne Chiu <waynehacking8@gmail.com>
 2026-06-13 01:40:50 +00:00
Nick Hill 1a369783e9 [BugFix] Avoid prematurely freeing cached mm encoder outputs (#45347) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-12 15:39:40 -07:00
Fangzhou AiandClaude Fable 5 badddd254f [ROCm][DSV4][Perf] Fuse inverse-RoPE and cache bf16 wo_a in o-projection (#45103) 
Signed-off-by: Fangzhou Ai <fangzhouai@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-12 15:57:09 -05:00
Michael GoinandClaude c90650088d Add the QuantizedActivation linear-kernel contract (#44260) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-12 13:48:15 -07:00
Michael Goin 9eaacb23ec [Kernel] Consolidate Marlin thread-tile padding across all dense Marlin paths (#45295) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-06-12 13:46:21 -07:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 78739c1946 [Model Runner v2] Migration from v1 to v2, with Qwen and DSv2 MOE models [3/N] (#42667) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-12 20:44:52 +00:00
Flora Feng 6e4a547176 [Refactor] Deprecate ResponsesParser wrapper, inline parsing into ParsableContext (#45431) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-12 16:15:41 -04:00
Ryan Rockandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> aab639c705 [Core][AMD] Propagate shutdown timeout to MultiprocExecutor (#43154) 
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-06-12 15:13:31 -05:00
Isotr0py 6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-12 12:02:21 -07:00
Jonas I. Liechti d6fd7ce8da [Model][Dflash] Enable Dflash support for Qwen3NextForCausalLM targets (#45319) 
Signed-off-by: Jonas I. Liechti <j-i-l@t4d.ch>
 2026-06-12 10:30:09 -07:00
Xiaohong (Sean) ChenandYanan Cao 272c16953e [Kernel][Helion][1/N] Add Helion kernel for dynamic_per_token_scaled_fp8_quant (#33790) 
Signed-off-by: Sean Chen <seachen@redhat.com>
Co-authored-by: Yanan Cao <gmagogsfm@gmail.com>
 2026-06-12 12:50:06 -04:00
Chauncey 3b8fc3fe6d [Frontend] Support strict mode for tool calling with ResponsesAPI (#45396) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-06-12 10:59:59 -04:00
Srinivas Krovvidiandsrinivas_oo7 9ff278b1d2 [Core][KV Connector] fix scheduler KV connector stats aggregation (#43877) 
Fixes scheduler-side KV connector stats collection so that:

1. update_connector_output() runs before scheduler-side stats are collected.
2. worker-side and scheduler-side KV connector stats are aggregated when both are present.
3. scheduler-only KV connector stats are still emitted when no worker-side stats exist.

Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
 2026-06-12 14:51:55 +00:00
Guan-Ming (Wesley) Chiu c7aa3d2630 [Core] Support structured outputs for beam search (#35022) 
Signed-off-by: Guan-Ming (Wesley) Chiu <guanmingchiu@gmail.com>
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
 2026-06-12 06:56:25 -07:00
Thillai ChithambaramandMark McLoughlin 4171ae406c [V1][Metrics] Add MLA attention metrics for DeepSeek MFU estimation (#39457) 
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2026-06-12 14:28:40 +01:00
Ethan Feng b7f9b6ab27 [Metrics] Add group-aware KV cache capacity to vllm:cache_config_info (#42206) 
The startup log already reports the correct group-aware KV cache capacity for
hybrid models, but Prometheus did not expose matching info in 'vllm:cache_config_info`.

This PR adds kv_cache_size_tokens and kv_cache_max_concurrency.

Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
 2026-06-12 11:49:44 +00:00
Tiezhen WANGClaudeIsotr0py
 		f1e13f7df9 [Model] Remove Mono-InternVL (InternLM2VEForCausalLM) (#45129) 
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-12 10:41:09 +00:00
snadampalandNicolò Lucchesi 88ed636218 [KV Connector]: Support KV push from Prefill to Decode node using Nixl KV Connector (#35264) 
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-06-12 10:38:41 +00:00
Chaunceycjackalmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2043258dec [Frontend] Support strict mode for tool calling (#45003) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-12 07:51:48 +00:00
Yuwen Zhou 0cd9b7af25 [CPU] Support CPU W4A16 INT4 MoE (#43409) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
 2026-06-12 07:12:37 +00:00
allgatherandIsotr0py 39dee1114a [MM][Perf][CG] Support ViT full cudagraphs for mllama4 (#40660) 
Signed-off-by: allgather <all2allops@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-11 22:17:55 -07:00
Lucas WilkinsonMartin KuklaMatthew BonanniDipika SikkaNickLucchejiahancAlec KohlhoffPorras HuangClaudescoootscooob
 		eb28452b10 [Model] Add DiffusionGemma Support (#45163) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Martin Kukla <martin.kukla@cantab.net>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Dipika Sikka <dsikka@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com>
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com>
 2026-06-11 22:17:35 -07:00
Divakar Verma 1ce3cdc5c1 [ROCm][CI] fix fp8 support for test_deepep_moe (#45302) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-06-12 00:16:14 -05:00
Dao007forever 6fbfdd1831 [NIXL] Per-region KV transfer classification for mixed full-attn + MLA groups (#44583) 2026-06-11 21:42:41 -07:00
Ekagra Ranjan 226ba9fc9e [ASR] Add Long Audio benchmark and correctness test (#44587) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-06-12 04:11:16 +00:00
Varun Sundar Rabindranathandvarun sundar rabindranath b927004c44 [Bugfix] Mamba CPU Offloading (#44599) 
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
 2026-06-11 21:07:35 -07:00
sasindharanSasindharan SankarChauncey
 		42ae5e7ac6 [Bugfix] Fix --enable-prompt-tokens-details omitting zero cached tokens (#44383) 
Signed-off-by: Sasindharan Sankar <sasindharansankar@email.com>
Co-authored-by: Sasindharan Sankar <sasindharansankar@email.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-06-11 20:37:42 -07:00
Nick Hill 2263f8a3de [CI][BugFix] Fix broken test_mamba_prefix_cache.py due to stale mock (#45345) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-12 03:26:17 +00:00
Ting SUN c1076839c9 [Bugfix][Model] Pass revision by name in Run:ai and bitsandbytes index downloads (#45308) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-06-11 20:21:46 -07:00
Fangzhou AiandvLLM Contributor fcf5115c45 [ROCm][DSv4][Perf] Flash-decode split-K decode attention kernel (#44899) 
Co-authored-by: vLLM Contributor <contributor@vllm.ai>
 2026-06-12 03:17:52 +00:00
littlecircle0730Hsiao-Yuan ChenOr Ozeri
 		4bc83323f2 [Bugfix] OffloadingConnector: respect skip_reading_prefix_cache flag (#44592) 
Signed-off-by: Hsiao-Yuan Chen <hy.c@Hsiao-YuandeMacBook-Pro.local>
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com>
Co-authored-by: Hsiao-Yuan Chen <hy.c@Hsiao-YuandeMacBook-Pro.local>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-06-12 02:20:39 +00:00
yzong-rh e0871ad225 [Refactor] Chat Completions Streaming Harmony Refactor and Bugfixes (#45104) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-06-12 01:09:47 +00:00
jpwangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 6f573f486b [Bugfix] Initialize missing attributes in mistral eagle (#45217) 
Signed-off-by: jpwang <jpwang@smail.nju.edu.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-12 08:21:01 +08:00
Dao007foreverandClaude 8a91228dbe [Bugfix][KVConnector][Mooncake] Close MooncakeDistributedStore on connector teardown (#45206) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-11 14:33:48 -07:00
yzong-rh f712fd0d7d [Refactor] Chat Completions Harmony Refactor, non-streaming path. (#45171) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-06-11 21:18:30 +00:00
Wentao Ye 5a6c7b7ab5 [Bug] Fix test flashmla for DSv4 (#45052) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-11 16:22:26 -04:00
Tiezhen WANGandClaude c9340e6f35 [Model] Remove InternLMForCausalLM registry alias (#45128) 
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-11 20:02:51 +00:00
Ben Browning 235b63c004 [Bugfix] Fix Anthropic tool_use content handling dropping args (#45287) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2026-06-11 20:01:29 +00:00
Xiaohong (Sean) ChenYanan Caomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2ec6594db9 [Kernel][Helion][1/N] Add Helion kernel for per_token_group_fp8_quant (#36902) 
Signed-off-by: Sean Chen <seachen@redhat.com>
Co-authored-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-11 08:59:08 -07:00
Jiangyun Zhu f81daf8880 [Attention] add triton diff-kv backend for mimo (#41797) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-06-11 11:36:31 -04:00
Nick HillandClaude Opus 4.8 4085ff7cb4 [Core] Add kvcache watermark to reduce preemptions (#44594) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-11 08:27:31 -07:00
Nicolò Lucchesi 750aab5b8e [Bugfix] Fix CPU memory leak related to not cleaning up old remotes data (#44424) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-06-11 07:54:52 -07:00
Harry Mellor 03878d1c22 Deprecations for v0.23 and v0.24 (#44992) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-11 14:35:38 +00:00
zhanqiuhulHrHenry233underfituu
 		55911db580 [PD][Core] Fix Mamba prefix cache hit rate in PD disaggregation (#44243) 
Co-authored-by: lHrHenry233 <2381623149@qq.com>
Co-authored-by: underfituu <hzhucong@163.com>
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
 2026-06-11 14:10:25 +00:00