iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,297 Commits 3 Branches 0 Tags
main
Commit Graph
132 Commits
Author SHA1 Message Date
Yang XuZihua WuKimi CodeCodexMisha Goinvllm-agent
 		71c20f73f2 [FlashInfer] Migrate autotune warmup to flashinfer.autotune_v2 managed cache (#55671) 
Signed-off-by: Yang Xu <yanxu@nvidia.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: YangXu <yanxu@nvidia.com>
Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 02:08:28 +00:00
Francisco Javier ArceoCodexMisha Goin
 		c923e7769b [Feat] Add text-only OpenAI Decisions API MVP (#60465) 
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-14 07:51:08 +08:00
Juan Pérez de AlgabaandOpenAI Codex 7173c64e2e [Security] Bound decoded and downloaded media per request (#61226) 
Signed-off-by: juanpas97 <juan.perezalgaba@gmail.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-13 10:17:14 +00:00
Jimmy WangandJuan Pérez de Algaba 2534a96fcf [Docs] List additional unauthenticated endpoints (#61081) 
Signed-off-by: Jimmy Wang <53066820+JimmyWang0417@users.noreply.github.com>
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com>
 2026-10-12 11:47:52 +02:00
Juan Pérez de AlgabaRobert ShawItay Etelis
 		93d2cab35e [Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions (#51504) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-10-11 11:39:11 -04:00
aoshen02andOpenAI Codex 1789f09588 [Frontend] Explain disabled scale-out endpoints with opt-in hints (#61042) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-11 08:58:07 +00:00
Matthew BonanniandClaude Opus 5.5 4c43254d52 [HiSparse] Report host-pool block residency as HiSparse metrics (#59485) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 16:45:34 -06:00
Taewoong KimandCyrus Leung 75e3b17c05 [Frontend] Add cache salt support to generative scoring (#60841) 
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-09 16:07:50 +00:00
Taewoong KimandChauncey 18c19eaf23 [Frontend] Add cache salt and tracing to structured decisions (#60786) 
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-10-09 08:53:50 +00:00
Matt MastracciClaude Opus 5.5Lucas WilkinsonLucas WilkinsonFrancisco Javier ArceoChauncey
 		d33872dfeb [Feat] Structured decisions endpoint (/v1/systemone) (#59299) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-10-08 14:48:12 +00:00
aoshen02wang.yuqiClaude Opus 5.5
 		cbee82324c [Docs][RL] Move pause/resume and batch invariance examples to examples/rl (#59949) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 06:22:10 +00:00
Sunny RangnaniClaudeJuan Pérez de Algaba
 		8bd7737a03 [Docs] Clarify that vLLM does not isolate tenants sharing a server (#59811) 
Signed-off-by: SunnyR <sunny.rangnani@chainguard.dev>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com>
 2026-10-06 22:39:59 +08:00
Robert ShawClaude Opus 5.5Matthew Bonanni
 		2e3154aa18 [HiSparse] Log steady-state max concurrency and expose host-tier utilization gauges (#58949) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-05 14:20:32 -07:00
Sohaib Ahmed 64cb683842 [Model] Add Nemotron 3.5 ASR transcription support (#59827) 
Signed-off-by: Sohaib-Ahmed21 <sohaibahmed1919@gmail.com>
 2026-10-05 16:28:09 +00:00
Martin Hickey bc21cba967 [Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-03 19:38:25 +08:00
Guy StoneandClaude Fable 5 37d6174096 [Metrics] Add --custom-histogram-buckets to override histogram bucket families (#48867) 
The core engine histograms ship with default bucket boundaries tuned for
typical serving workloads. The `--custom-histogram-buckets` option replaces
the boundaries of one or more bucket families. Use it, for example, to track
sub-300ms latency SLOs with the request-phase histograms, whose smallest
default boundary is 0.3s.

Signed-off-by: Guy Stone <guys@spotify.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-10-01 01:03:39 +00:00
JulienDarveBugen ZhaoClaude Opus 5.5
 		f28a508162 [Feature][Rust Frontend] Add Shutdown control RPC (#59316) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-30 12:49:25 +00:00
mevince cd1daf58a9 [Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector (#57251) 
Signed-off-by: Vincent <vincexxchan@gmail.com>
 2026-09-26 17:17:11 -07:00
Juan Pérez de AlgabaandCursor 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-26 11:10:35 +00:00
LucainandClaude Opus 5 6b858751f6 [docs] Fix legacy hf CLI references (vllm) (#57958) 
Signed-off-by: Wauplin <lucainp@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-21 15:32:27 +00:00
Kevin H. LuuOpenAI CodexTyler Michael Smith
 		caf56ce9da [CI] Pass the scale-out endpoint flag in EC E2E (#56819) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-09-15 12:25:30 +08:00
Martin Hickey 934b1fcbcb [Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-14 10:02:14 +00:00
Sören Dréano 9a35c081e8 [Docs] Correct API key authentication scope (/inference does NOT bypass the API key) (#56269) 
Signed-off-by: Sören Dréano <gailenstorm@gmail.com>
 2026-09-11 16:54:35 +00:00
Francisco Javier ArceoandOpenAI Codex 6a2a2bb02b [Frontend] Add stateless /v1/responses/render endpoint (#50195) 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-07 12:50:42 +00:00
Francisco Javier Arceo c00091e026 [Frontend] Gate scale-out endpoints behind opt-in flag (#54579) 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
 2026-09-02 14:31:14 +08:00
lzhan011andCyrus Leung ce7391712b [Bugfix][Security] Bound embedding densification before to_dense() (#54632) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-01 08:45:14 +00:00
Juan Pérez de Algaba 0d7d5ed0b2 fix(security): enforce VLLM_MAX_AUDIO_CLIP_FILESIZE_MB on all audio paths (#53561) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-08-24 21:37:02 +00:00
Juan Pérez de Algaba 9f295fe8ce [Docs][Security] Document multimodal media UUID security implications (#53582) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-08-24 07:19:18 -07:00
Clinton ThomasandLucas Bourtoule 2ec6f0d71e Reject oversized media before fully downloading it (#51896) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
 2026-08-24 01:15:09 +00:00
Russell Bryant 6d8cd88e8d [Docs] document cache salting for prefix cache timing side-channel mitigation (#39082) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-08-21 09:06:49 -07:00
Connor CarpenterandNick Hill f473870ecf [Rust Frontend][gRPC] Add RL lifecycle control (#51316) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-14 11:32:39 -07:00
Juan Pérez de AlgabaandCursor 640a09086d Fix DoS via sample-rate forgery bypassing audio decode duration guard (#49948) 
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-10 15:49:17 +00:00
Isotr0py 1180b604f8 [Multimodal] Expose mm hash algothrim selection to cli args (#49686) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-31 05:25:19 +00:00
Juan Pérez de Algaba 89d97d9a16 docs(security): document Ray cluster trust model and env var propagation (#50397) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-07-30 11:52:55 +00:00
Harry Mellor 3e74c60b9c [Docs] Use gen-files for generated docs content (#49587) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-25 14:43:21 +00:00
Harry Mellor 84d26b9ee3 [Model] Remove Plamo2 (#49729) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-24 13:54:24 -07:00
Martin Hickey 4d4e04f452 [Render] Add round trip parity test and docs for derender (#48617) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-17 05:02:46 +00:00
aoshen02Claude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		83762b77b0 [Frontend] Add /abort_requests to the RLHF dev API router (#47173) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-12 14:21:02 +08:00
Martin Hickey f7fc0ca993 [Frontend] Add endpoint plugins framework (#47454) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-08 10:00:41 +08:00
Juan Pérez de AlgabaRussell BryantRussell Bryant
 		5c4db60f01 docs(security): document gRPC interface as insecure for private use only (#45903) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
Co-authored-by: Russell Bryant <russell.bryant@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2026-07-01 12:39:57 +00:00
Juan Pérez de Algaba 364ee36af1 fix(security): prevent image decompression bomb OOM denial of service (#47010) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-30 09:39:22 +00:00
AgenticSparkandCopilot bf292b5f6b [Docs] Remove BambaForCausalLM from supported hybrid models list (#46071) 
Signed-off-by: liejiang <jianglie2023@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-26 08:02:50 +00:00
Tiezhen WANGClaudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d14e551a53 [Model] Remove MiniMaxText01, MiniMaxVL01, MiniMaxForCausalLM (#45993) 
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-22 15:20:46 +08:00
Mohammad Miadh Angkad da03e549b3 [UX] Add a persistent cache for FlashInfer autotuning (#42537) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-05-18 20:25:37 -07:00
DustHunterQwen-Coderwang.yuqi
 		39c67d714e fix: add API key authorization to /v2 endpoints (#42594) 
Signed-off-by: DustHunter <dusthunter@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-16 01:29:27 +00:00
Vinay R DamodaranandRussell Bryant 5497ffbf7c Add documentation about vLLM FIPS compliance (#42190) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Signed-off-by: Vinay R Damodaran <vrdn@hey.com>
Co-authored-by: Russell Bryant <russell.bryant@gmail.com>
 2026-05-11 18:17:02 +00:00
Russell Bryant 5a0a8fc1ea [Docs] add cache directory security guidance (#38920) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-05-06 16:54:29 -07:00
Russell Bryant f05f3664c3 [Doc] Add missing API endpoints to security documentation (#40532) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-04-28 21:53:19 +00:00
wang.yuqi a8208e6a81 [Examples] Resettle features examples. (#40995) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-28 00:33:41 -07:00
Juan Pérez de Algaba 57861ae48d (security) Fix SSRF in batch runner download_bytes_from_url (#38482) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-03-30 07:10:01 +00:00