iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,968 Commits 3 Branches 0 Tags
f83ea61cfe77f0b200f2ded1fda9be5ff591af2c
Commit Graph
727 Commits
Author SHA1 Message Date
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Taneem Ibrahim bbd7c24d6e [MRV2] Validate MRV2 entrypoint logits processors (#57728) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-24 09:07:26 +00:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		c0c35ddd8e [Bugfix][CI] Fix the flaky sharded-sampling tests, and the engine teardown need (#58342) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 02:05:11 -07:00
Matt MastracciClaude Fable 5.1Lucas WilkinsonCodexRazorback16
 		1b3b88ec2b [Core] structured generation mode for DiffusionGemma model (Jev-like) (#57250) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Razorback16 <razorback16@protonmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Razorback16 <razorback16@protonmail.com>
 2026-09-22 20:51:17 +00:00
shaohuaxiandCodex 5abf85d27a [Bugfix][Frontend] Validate mixed prompt embedding mask lengths (#57006) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-22 05:50:31 +00:00
Juan Pérez de Algaba 4800024b98 [Security] Reject min_tokens that exceeds the filled max_tokens default (#57731) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
 2026-09-22 05:50:23 +00:00
Karen Chung 986e217158 [Feature] Add first-class KV hints request envelope for programmatic KV management (#53423) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-09-21 09:56:11 -07:00
Cyrus Leung c8824a1e42 [MM] Move supports_multimodal_inputs and cache out of registry (#57913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-21 07:43:07 -07:00
czhu-cohere 3918f3c5a3 [Profiler] Unify platform-aware torch profiling (#57460) 2026-09-20 19:17:23 -07:00
CrazyCoderaoshen02Codex
 		1185254851 [Bugfix][KV Connector] Propagate cache reset failure during sleep (#54581) 
Signed-off-by: CrazyCoder <18235787078@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-21 09:25:02 +08:00
David CheungClaudeNick Hill
 		5302d1fe40 [Model Runner V2] Support custom logits processors (#56497) 
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-19 17:46:49 +00:00
hangangqiangmukeshbhandarkaraoshen02Claude Fable 5.1aoshen02
 		90531daf6d [Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-18 18:11:05 -07:00
Nicolò Lucchesi a7156060c1 [Core] Make parallel sampling (n>1) reqs admission atomic (#53936) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-18 18:08:44 +02:00
Dakai AnAlanFokCoaoshen02OpenAI Codex
 		092bdd6d57 [Frontend][Core] Add release_kv_cache_memory() API (#44890) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-18 07:12:12 +08:00
Wei Zhao ccde9c61e9 [Bugfix] Restore KV cache metadata GET method for external event consumer (#56925) 
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-09-17 23:01:35 +00:00
Oxana KorzhandAndreas Karatzas 2307a1d16c [Core] Don't issue a blocking collective RPC during engine handshake (#57226) 
Signed-off-by: okorzh <okorzh@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-16 21:30:59 -05:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Robert ShawandClaude 42919b49c5 [P/D] Report prefill worker cache hits in prompt_tokens_details (#54222) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-16 15:23:31 +00:00
JulienDarveBugen ZhaoOpenAI Codex
 		f37c550bf6 [Feature][Frontend] Expose effective attention block size for DCP (#56538) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-16 08:29:43 +00:00
Chaojun ZhangandKunshang Ji 622934e61f [XPU] Honor device_ids for worker device placement (#56015) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-16 11:53:23 +08:00
Chauncey f0fbc98c48 [Frontend] Share max_num_queued_reqs across API server processes (#54746) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-09-15 14:50:26 +00:00
aoshen02Bhushan AsatiMegha Agarwal
 		84e5f3edac [Bugfix] Validate routed-expert prompt offsets before engine submission (#56844) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bhushan Asati <bhushanasati25@gmail.com>
Co-authored-by: Megha Agarwal <19240983+meghaagr13@users.noreply.github.com>
 2026-09-15 09:17:41 +08:00
Itay Alroy 658c8131c7 [Elastic EP] Reuse CUDA graphs across reconfiguration (#54985) 2026-09-12 10:26:39 -04:00
djramicandAndreas Karatzas c6ba5e89d6 [Bugfix][Elastic EP] Route only to surviving engines during scale-down (#55772) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-08 21:28:28 -05:00
Wentao Ye 6b15bea080 [Refactor] Remove utils dead code (#53941) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-08 11:22:09 -04:00
Wei Zhao c3ec0d29f5 [Bugfix] Fix cuda profiler missing bug (#55237) 2026-09-07 03:15:57 +00:00
aoshen02 039ea82660 [Core] Sync DP state on the first step of a wave (#52957) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-06 02:16:58 +00:00
Shaurya Pratap Srivastava e862c2f45b [Docs] Add example for Renderer.render_cmpl() usage (#54265) 
Signed-off-by: null-Exception1 <rising1legend@gmail.com>
 2026-09-03 23:35:37 -07:00
aoshen02Claude Fable 5.1vx120
 		560ef78bfe [Perf][Model Runner V2] Compact sampling masks on GPU instead of unpacking the full-vocab bitmask on CPU (#54901) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: vx120 <893600387@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: vx120 <893600387@qq.com>
 2026-09-03 19:03:36 -07:00
Ning XieandCyrus Leung 2a336d8239 [warmup] overlap renderer warmup and engine core initialization (#54557) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-03 19:44:21 +00:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
Alberto MassiddaandAlberto Massidda 55178f2d09 [Bugfix][Profiler] Fix API server crash on double /stop_profile (#51678) 
Signed-off-by: Alberto Massidda <amassidda@nvidia.com>
Co-authored-by: Alberto Massidda <amassidda@nvidia.com>
 2026-09-01 13:05:04 -07:00
Nicolò Lucchesi dafbef15a1 [Core] Add max_num_queued_reqs and max_num_queued_tokens for queue size management (#49445) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-31 14:26:49 +00:00
Oxana KorzhandClaude Opus 5 fb68025138 [Bugfix][MRV2] Release model and KV cache on in-process engine shutdown (#54162) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-28 22:09:08 -07:00
Kevin H. Luu d04d3e0a07 [Bugfix] Revert renderer warmup overlap to avoid fork deadlock (#54023) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-27 16:36:33 -07:00
Ning Xie ca90b9e7d4 [warmup] overlap renderer warmup and engine core initialization (#52764) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-08-27 11:42:08 +08:00
aoshen02 17da48596c [Bugfix][DP] Synchronize the device on pause completion (#52914) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-08-26 09:50:33 -07:00
Tony 903a02192f [Bugfix] Handle empty FlatLogprobs slices and delta output (#53704) 
Signed-off-by: tony <864832769@qq.com>
 2026-08-26 21:00:54 +08:00
Naveenraj KamalakannanandBenjamin Chislett d5cadcee86 [Feature][DSpark]: Logprobs adaptive verification (#52242) 
Signed-off-by: Naveenraj Kamalakannan <therealnaveenkamal@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-08-25 10:22:33 -04:00
Ning Xie 23310bc3de [sleep functionality] code refactor about sleep/wake_up (#50431) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-08-25 07:45:20 +00:00
Jee Jee LiandCyrus Leung 4f686e182a [MISC] Cleanup deprecated parameters (#53559) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-24 10:19:04 -07:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Juan Pérez de Algaba e00a034ad1 (security) fix: enforce decoder prompt-length validation for skip-che… (#46588) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-08-21 09:07:19 -07:00
Matthew KotilaReed MeyersonClaude Opus 4.8
 		7cfb97e337 [Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915) 
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-20 15:42:49 -07:00
Cyrus Leung 4666a8ba9e [Refactor] Remove InputPreprocessor (#53064) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-20 08:39:48 +00:00
Zhixia LiuianzxliuClaudeaoshen02CodexNick Hill
 		a1c5b1fd9f [Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701) 
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 15:12:43 +08:00
lxyand“新颐” 0c8c3f41cf [Bugfix][V1] Sync mamba_block_size via EngineCoreReadyResponse (#50809) 
Signed-off-by: “新颐” <lxy537258@antgroup.com>
Co-authored-by: “新颐” <lxy537258@antgroup.com>
 2026-08-19 16:44:36 +00:00
Andreas KaratzasOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		3a386cfaf5 [ROCm] Give EngineCore cleanup grace after request abort (#52281) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 11:39:59 -05:00
Itay EtelisandItay Etelis d36cc4254e [Bugfix][Elastic EP] Reject scale below the minimum data parallel size (#52702) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
 2026-08-19 01:37:29 +00:00