iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,722 Commits 3 Branches 0 Tags
382970ee6ca490aeaaaf4e32c53695b581ff61ba
Commit Graph
41 Commits
Author SHA1 Message Date
Martin Hickey 8b98b7d0b4 [Frontend] Add streaming parity tests and docs for derender (#57922) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-21 14:13:52 +00:00
Hongxin Xuaoshen02Codex
 		10e6a7f210 [Frontend] Expose per-request spec decode metrics in generate API (#43310) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
 2026-09-20 12:21:33 +00:00
Cyrus Leung 56cac80b14 [Deprecation] Remove assistant_token_mask support (#57520) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-19 00:17:51 +08:00
Martin Hickey 70164bdadc [Frontend] Add stream reasoning and tool calls from the derender endpoint (#50550) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-18 16:36:27 +02:00
Shimi BandielClaudeCyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e8a2a0a8bb [Bugfix][Frontend] Reject stop strings on --tokens-only servers instead of silently ignoring them (#57058) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-17 22:02:00 +00:00
aoshen02 b978df3299 Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302) 2026-09-16 23:31:59 -07:00
aoshen02andClaude Fable 5.1 8538017f4b [Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-17 05:58:56 +00:00
aoshen02 0983aef8da [Frontend] Return prompt metadata from /inference/v1/generate (#53187) 2026-09-16 15:29:07 -07:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Martin Hickey 934b1fcbcb [Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-14 10:02:14 +00:00
shaohuaxiandCodex ae71862c51 [Bugfix][Frontend] Check EC requirements for each metadata item (#56070) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-10 19:23:34 +00:00
Francisco Javier ArceoandOpenAI Codex 6a2a2bb02b [Frontend] Add stateless /v1/responses/render endpoint (#50195) 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-07 12:50:42 +00:00
zhouyouandCursor Grok 4.6 52358e6e19 [Frontend] Expose multimodal metadata for disaggregated prefill (#54659) 
Signed-off-by: zhouyou9505 <zhouyou9505@gmail.com>
Co-authored-by: Cursor Grok 4.6 <noreply@cursor.com>
 2026-09-06 14:46:52 +00:00
Clinton ThomasandLucas Bourtoule 1970f3ed4b Validate scale-out multimodal data before engine handoff (#51898) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
 2026-09-06 04:58:04 +00:00
Clinton ThomasLucas BourtouleCyrus Leung
 		e962733e08 [Security] Validate cache salts before they reach LMCache (#51444) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-05 03:56:06 +00:00
Francisco Javier Arceo c00091e026 [Frontend] Gate scale-out endpoints behind opt-in flag (#54579) 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
 2026-09-02 14:31:14 +08:00
Nicolò Lucchesi dafbef15a1 [Core] Add max_num_queued_reqs and max_num_queued_tokens for queue size management (#49445) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-31 14:26:49 +00:00
wang.yuqiandCyrus Leung eeb549a74d [Frontend] Move engine/protocol.py out openai folder (#54492) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-08-31 10:13:50 +00:00
eligottsandOpenAI Codex 4ae172231c [Frontend] Forward cache salt for content parts (#54315) 
Signed-off-by: Eli Gotts <78387377+eligotts@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-30 23:29:53 -07:00
Taneem Ibrahim fe6db3ed5e [Bugfix] Validate scale-out transfer params (#54324) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-30 09:46:12 -04:00
Juan Pérez de Algaba 1dc464d426 [Bugfix] Bound cache_salt length to prevent DoS via scheduler CPU exhaustion (#54353) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-08-30 04:28:16 +00:00
Konstantin DunasandOpenAI Codex c71f6f8a81 [Bugfix] Thread kv_transfer_params into engine for /inference/v1/generate (disagg) (#42644) 
Signed-off-by: hallerite <git@hallerite.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-26 08:42:52 -07:00
Hyunkyun Moon f25c2692fe [Frontend] Add /v1/messages/render endpoint for the Anthropic Messages API (#45803) 
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com>
 2026-08-26 11:33:59 +08:00
Wei-Cheng (Wayne) Chiu b41dc4ec72 [Bugfix] Return HTTP 500 for non-streaming generate errors (#49195) 
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com>
 2026-08-21 09:07:17 -07:00
Quentin Gallouédec 92bdee05cb [Bugfix][Frontend] Return all choices from /inference/v1/generate when n > 1 (#52399) 
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
 2026-08-19 20:48:45 +08:00
aoshen02OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7553aac77b [Frontend] Add routed-experts prompt offset (#51906) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 08:09:00 -07:00
vx120aoshen02CodexClaude Opus 4.6OpenAI Codex
 		50ba4bc6b2 [Feature] Mask Replay (#49577) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-13 08:44:31 +00:00
Vinay R DamodaranandClaude Opus 5 3ee2df3033 [Frontend] Make model optional on all /derender request classes (#51463) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-12 04:48:58 +00:00
aoshen02Claude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		33584901ba [Frontend] Add content_parts to /inference/v1/generate for raw multim… (#51478) 
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 10:00:12 +08:00
Karen Chung f57123aa2d feat(frontend): session id plumbing into requests (#48048) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-08-03 22:18:58 +00:00
Cyrus Leung b3f97dae24 [Frontend] Require cache_salt to be non-empty via schema (#50816) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-03 06:51:27 +00:00
Julien Denize 1a20d23dab [PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947) 2026-07-30 15:38:00 +02:00
Martin Hickey 0a31372e5f [Frontend] Add detokenization streaming derender for disaggregated serving (#47301) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-29 18:01:20 -07:00
Martin Hickey 4d4e04f452 [Render] Add round trip parity test and docs for derender (#48617) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-17 05:02:46 +00:00
omerpaz95andOr Ozeri 8df14cfc8c [EC Connector] Add EC Transfer Params (#42433) 
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-07-12 14:35:33 +03:00
Juan Pérez de AlgabaandCursor b3e85be663 fix: use configured max_logprobs instead of hardcoded 20 in derender validation (#47834) 
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-07 09:42:47 +00:00
Juan Pérez de Algabaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8e61b646e2 fix(security): add resource bounds validation to derender endpoints (#47260) 
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-07 14:58:26 +08:00
wang.yuqi a14f57a3ac [Frontend] Refine the entrypoint class's inheritance hierarchy. (#47498) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-07-03 10:50:06 +00:00
Aleksei IvashovandNickLucche 024b06b0dc [Bugfix] Expose usage field in GenerateResponse for disaggregated serving (#42748) 
Signed-off-by: AIvashov <ivashov.aleksey@proton.me>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-01 10:00:19 +00:00
wang.yuqi 3483240b7e [Frontend] Consolidate scale out entrypoints (#44512) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-29 03:18:53 -07:00