iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,991 Commits 3 Branches 0 Tags
c19c263ac8ea3fd6e1d2d689f0e4e725855d89cf
Commit Graph
763 Commits
Author SHA1 Message Date
Dakai AnYifan QiaoClaude Opus 5.5
 		df6c7ad031 [Core][Refactor] Move KV cache planning logic to kv_cache_planning.py (#61183) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 03:05:39 +00:00
simon-veitner-redhatClaudeLucas Wilkinson
 		a7b7bc1fbe [Metrics] Count finished requests by watermark status (#58715) 
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-12 17:21:22 +00:00
Tianyu Guo 258b97ff6b [Bugfix][EC Connector] Size transfers by measured encoder output widths (#57904) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-10-12 12:32:39 +00:00
Thuongvu HoandNicolò Lucchesi 99b44c7a42 [Metrics] Add admission rejection metrics (#55812) 
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-10-12 08:31:07 +00:00
aoshen02 bd5bbce6e4 [RL] Release weight-transfer communicators without stopping inference (#61060) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
 2026-10-11 21:15:14 +08:00
Shuolei Wangandaoshen02 b89852aad0 [RL][Frontend] Return token-level weight-version spans on final outputs (#59582) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-11 11:33:29 +08:00
Wentao Ye 276fbcff27 [Deprecation] Remove scheduled deprecated items (#60923) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-10 15:45:33 -04:00
Chauncey fa5cfd48d5 [Refactor] Deduplicate trace header handling in BaseServing (#60828) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-10-10 05:27:58 +00:00
Dao007foreverWei ZhaoClaude Opus 5.5zjy0516
 		f8e6e365ae [Bugfix][Mooncake] Save the EAGLE attention proof with hybrid Mamba partial tails (#60695) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Signed-off-by: Dao007forever <daole@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-10-10 00:08:08 +00:00
Ting SUNNick HillClaude Opus 5.5
 		51144dd83e [Bugfix][Spec Decode] Trim token_ids/logprobs left past a stop string under speculative decoding (#47616) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 14:22:31 -07:00
Dakai AnYifan QiaoOpenAI CodexOpenAI CodexLucas WilkinsonClaude Opus 5.5Lucas Wilkinson
 		02e55d2af3 [KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-09 15:48:37 -04:00
Taneem Ibrahim a6b39f0262 [Bugfix] Clean up async client sockets after event loop closure (#57757) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-10-09 10:32:47 -04:00
Ning XieandSimon Mo c4a5ff5c18 [Frontend] shutdown engine core and its sub processes asap when initializing (#52299) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2026-10-08 11:46:01 -07:00
Wei ZhaoWei ZhaoCodexzjy0516Claude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d8fa463e2b [Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries (#60533) 
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-08 18:33:48 +00:00
Bugen ZhaoOpenAI CodexClaude Opus 5.5
 		f226af0e11 [Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners (#54113) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 06:25:27 +00:00
aoshen02aoshen02ClaudeCodex
 		caff2b44cb [BugFix] Pick the DP world-group port at bind time via the coordination store (#51018) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-08 02:30:35 +00:00
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
Raphaël RiallandandSimon Veitner c32495d58f [watermark] compatibility validation (#56801) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
 2026-10-05 14:37:13 -04:00
aoshen02aoshen02Claude Opus 5.5
 		e319f86f15 [Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-03 14:17:09 +00:00
Benedikt Falk c4973d94fe [Bugfix][Frontend] Avoid generation for empty streaming input (#59015) 
Signed-off-by: Benedikt Falk <bfalk@nvidia.com>
 2026-10-02 18:48:43 +00:00
Taneem Ibrahim f0c44cc7d5 [Bugfix] Fix stalled local-only Elastic EP scale-up (#57820) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-10-02 10:47:55 -04:00
Chaoyinglinchaoying6OpenAI Codex
 		592c6f3fbc [Bugfix][Pooling] Handle multimodal cache misses without crashing (#58975) 
Signed-off-by: linchaoying6 <linchaoying6@jd.com>
Co-authored-by: linchaoying6 <linchaoying6@jd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-02 10:14:33 -04:00
Martin Hickey fcfacc3499 [Bugfix][Frontend] Seed derender detokenization from the prompt (#59046) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-02 03:17:56 +00:00
Shijie Lyu 9aaef4296f [Bugfix][Frontend] Check reused prompt token ids against the vocab before streaming (#59555) 
Signed-off-by: Shijie Lyu <lshjhf@gmail.com>
 2026-10-01 09:26:55 -07:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
vx120Cuiping Chengaoshen02Claude Opus 5.5
 		70be6aae19 [Feature][Spec Decode] Support sampling mask replay for MRV2 MTP (#59359) 
Signed-off-by: Cuiping Cheng <chengcuipingswu@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Cuiping Cheng <chengcuipingswu@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 13:51:39 +08:00
Ronaldaoshen02Claude Opus 5.5
 		e5f3d08d72 [Frontend][RL] Align sleep-mode API responses and operation metrics (#52864) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 23:16:38 -07:00
Andreas Karatzas 866fa130fa [Core] Bound UniProc EngineCore startup threads to available CPUs (#58946) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
 2026-09-29 23:02:55 -07:00
Mia Ojeda e006d761a5 [Bugfix][CPU][DiffusionGemma] Support narrower canvas w/ sync scheduling (#59107) 
Signed-off-by: ojeda-e <estefania.b.ojeda@gmail.com>
 2026-09-29 23:28:50 -04:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
aoshen02aoshen02Claude Opus 5.5
 		a7a9ac2745 [Core] Combine per-engine utility results like the Rust client (#59240) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 22:20:51 +00:00
Chanbin Lim 91de237333 [Bugfix][DP] add_dp_placement_groups does not require ray[default] (#57648) 
Signed-off-by: beenpow <57864041+beenpow@users.noreply.github.com>
 2026-09-29 18:15:13 +00:00
Almog Tavor 171d1b9ef6 [Elastic EP] Support Model Runner V2 (#53934) 
Signed-off-by: almogtavor <almogtavor@gmail.com>
Signed-off-by: Almog Tavor <70065337+almogtavor@users.noreply.github.com>
 2026-09-29 00:20:40 +00:00
Benedikt Falk c68eb98b69 [Bugfix][Frontend] Keep in-flight requests on the same DP engine (#59017) 
Signed-off-by: Benedikt Falk <bfalk@nvidia.com>
 2026-09-28 23:52:29 +00:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
Tianyu Guo 53d2e16a97 [Bugfix][EPD] Skip sampling for encoder-only async steps (#58490) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-09-27 20:35:09 -07:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Taneem Ibrahim bbd7c24d6e [MRV2] Validate MRV2 entrypoint logits processors (#57728) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-24 09:07:26 +00:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		c0c35ddd8e [Bugfix][CI] Fix the flaky sharded-sampling tests, and the engine teardown need (#58342) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 02:05:11 -07:00
Matt MastracciClaude Fable 5.1Lucas WilkinsonCodexRazorback16
 		1b3b88ec2b [Core] structured generation mode for DiffusionGemma model (Jev-like) (#57250) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Razorback16 <razorback16@protonmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Razorback16 <razorback16@protonmail.com>
 2026-09-22 20:51:17 +00:00
shaohuaxiandCodex 5abf85d27a [Bugfix][Frontend] Validate mixed prompt embedding mask lengths (#57006) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-22 05:50:31 +00:00
Juan Pérez de Algaba 4800024b98 [Security] Reject min_tokens that exceeds the filled max_tokens default (#57731) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
 2026-09-22 05:50:23 +00:00
Karen Chung 986e217158 [Feature] Add first-class KV hints request envelope for programmatic KV management (#53423) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-09-21 09:56:11 -07:00
Cyrus Leung c8824a1e42 [MM] Move supports_multimodal_inputs and cache out of registry (#57913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-21 07:43:07 -07:00
czhu-cohere 3918f3c5a3 [Profiler] Unify platform-aware torch profiling (#57460) 2026-09-20 19:17:23 -07:00
CrazyCoderaoshen02Codex
 		1185254851 [Bugfix][KV Connector] Propagate cache reset failure during sleep (#54581) 
Signed-off-by: CrazyCoder <18235787078@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-21 09:25:02 +08:00
David CheungClaudeNick Hill
 		5302d1fe40 [Model Runner V2] Support custom logits processors (#56497) 
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-19 17:46:49 +00:00
hangangqiangmukeshbhandarkaraoshen02Claude Fable 5.1aoshen02
 		90531daf6d [Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-18 18:11:05 -07:00
Nicolò Lucchesi a7156060c1 [Core] Make parallel sampling (n>1) reqs admission atomic (#53936) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-18 18:08:44 +02:00
Dakai AnAlanFokCoaoshen02OpenAI Codex
 		092bdd6d57 [Frontend][Core] Add release_kv_cache_memory() API (#44890) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-18 07:12:12 +08:00