8bd7737a03
[Docs] Clarify that vLLM does not isolate tenants sharing a server ( #59811 )
...
Signed-off-by: SunnyR <sunny.rangnani@chainguard.dev >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com >
2026-10-06 22:39:59 +08:00
2e3154aa18
[HiSparse] Log steady-state max concurrency and expose host-tier utilization gauges ( #58949 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-05 14:20:32 -07:00
Sohaib Ahmed
64cb683842
[Model] Add Nemotron 3.5 ASR transcription support ( #59827 )
...
Signed-off-by: Sohaib-Ahmed21 <sohaibahmed1919@gmail.com >
2026-10-05 16:28:09 +00:00
Martin Hickey
bc21cba967
[Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) ( #58588 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-03 19:38:25 +08:00
Guy Stone and Claude Fable 5
37d6174096
[Metrics] Add --custom-histogram-buckets to override histogram bucket families ( #48867 )
...
The core engine histograms ship with default bucket boundaries tuned for
typical serving workloads. The `--custom-histogram-buckets` option replaces
the boundaries of one or more bucket families. Use it, for example, to track
sub-300ms latency SLOs with the request-phase histograms, whose smallest
default boundary is 0.3s.
Signed-off-by: Guy Stone <guys@spotify.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-01 01:03:39 +00:00
f28a508162
[Feature][Rust Frontend] Add Shutdown control RPC ( #59316 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Julien Darve <jdarve@NVIDIA.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-30 12:49:25 +00:00
mevince
cd1daf58a9
[Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector ( #57251 )
...
Signed-off-by: Vincent <vincexxchan@gmail.com >
2026-09-26 17:17:11 -07:00
Juan Pérez de Algaba and Cursor
31f2e70cd3
[Security] Gate per-request multimodal processor kwargs ( #58830 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-26 11:10:35 +00:00
Lucain and Claude Opus 5
6b858751f6
[docs] Fix legacy hf CLI references (vllm) ( #57958 )
...
Signed-off-by: Wauplin <lucainp@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-21 15:32:27 +00:00
caf56ce9da
[CI] Pass the scale-out endpoint flag in EC E2E ( #56819 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-09-15 12:25:30 +08:00
Martin Hickey
934b1fcbcb
[Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out ( #55176 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-09-14 10:02:14 +00:00
Sören Dréano
9a35c081e8
[Docs] Correct API key authentication scope (/inference does NOT bypass the API key) ( #56269 )
...
Signed-off-by: Sören Dréano <gailenstorm@gmail.com >
2026-09-11 16:54:35 +00:00
Francisco Javier Arceo and OpenAI Codex
6a2a2bb02b
[Frontend] Add stateless /v1/responses/render endpoint ( #50195 )
...
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-07 12:50:42 +00:00
Francisco Javier Arceo
c00091e026
[Frontend] Gate scale-out endpoints behind opt-in flag ( #54579 )
...
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com >
2026-09-02 14:31:14 +08:00
lzhan011 and Cyrus Leung
ce7391712b
[Bugfix][Security] Bound embedding densification before to_dense() ( #54632 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-01 08:45:14 +00:00
Juan Pérez de Algaba
0d7d5ed0b2
fix(security): enforce VLLM_MAX_AUDIO_CLIP_FILESIZE_MB on all audio paths ( #53561 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-08-24 21:37:02 +00:00
Juan Pérez de Algaba
9f295fe8ce
[Docs][Security] Document multimodal media UUID security implications ( #53582 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-08-24 07:19:18 -07:00
Clinton Thomas and Lucas Bourtoule
2ec6f0d71e
Reject oversized media before fully downloading it ( #51896 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
2026-08-24 01:15:09 +00:00
Russell Bryant
6d8cd88e8d
[Docs] document cache salting for prefix cache timing side-channel mitigation ( #39082 )
...
Signed-off-by: Russell Bryant <rbryant@redhat.com >
2026-08-21 09:06:49 -07:00
Connor Carpenter and Nick Hill
f473870ecf
[Rust Frontend][gRPC] Add RL lifecycle control ( #51316 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-14 11:32:39 -07:00
Juan Pérez de Algaba and Cursor
640a09086d
Fix DoS via sample-rate forgery bypassing audio decode duration guard ( #49948 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-08-10 15:49:17 +00:00
Isotr0py
1180b604f8
[Multimodal] Expose mm hash algothrim selection to cli args ( #49686 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-07-31 05:25:19 +00:00
Juan Pérez de Algaba
89d97d9a16
docs(security): document Ray cluster trust model and env var propagation ( #50397 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-07-30 11:52:55 +00:00
Harry Mellor
3e74c60b9c
[Docs] Use gen-files for generated docs content ( #49587 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-25 14:43:21 +00:00
Harry Mellor
84d26b9ee3
[Model] Remove Plamo2 ( #49729 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-24 13:54:24 -07:00
Martin Hickey
4d4e04f452
[Render] Add round trip parity test and docs for derender ( #48617 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-07-17 05:02:46 +00:00
83762b77b0
[Frontend] Add /abort_requests to the RLHF dev API router ( #47173 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-12 14:21:02 +08:00
Martin Hickey
f7fc0ca993
[Frontend] Add endpoint plugins framework ( #47454 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-07-08 10:00:41 +08:00
5c4db60f01
docs(security): document gRPC interface as insecure for private use only ( #45903 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Russell Bryant <russell.bryant@gmail.com >
Co-authored-by: Russell Bryant <russell.bryant@gmail.com >
Co-authored-by: Russell Bryant <rbryant@redhat.com >
2026-07-01 12:39:57 +00:00
Juan Pérez de Algaba
364ee36af1
fix(security): prevent image decompression bomb OOM denial of service ( #47010 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-06-30 09:39:22 +00:00
AgenticSpark and Copilot
bf292b5f6b
[Docs] Remove BambaForCausalLM from supported hybrid models list ( #46071 )
...
Signed-off-by: liejiang <jianglie2023@gmail.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-06-26 08:02:50 +00:00
d14e551a53
[Model] Remove MiniMaxText01, MiniMaxVL01, MiniMaxForCausalLM ( #45993 )
...
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-22 15:20:46 +08:00
Mohammad Miadh Angkad
da03e549b3
[UX] Add a persistent cache for FlashInfer autotuning ( #42537 )
...
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com >
2026-05-18 20:25:37 -07:00
39c67d714e
fix: add API key authorization to /v2 endpoints ( #42594 )
...
Signed-off-by: DustHunter <dusthunter@126.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-05-16 01:29:27 +00:00
Vinay R Damodaran and Russell Bryant
5497ffbf7c
Add documentation about vLLM FIPS compliance ( #42190 )
...
Signed-off-by: Vinay Damodaran <vrdn@hey.com >
Signed-off-by: Vinay R Damodaran <vrdn@hey.com >
Co-authored-by: Russell Bryant <russell.bryant@gmail.com >
2026-05-11 18:17:02 +00:00
Russell Bryant
5a0a8fc1ea
[Docs] add cache directory security guidance ( #38920 )
...
Signed-off-by: Russell Bryant <rbryant@redhat.com >
2026-05-06 16:54:29 -07:00
Russell Bryant
f05f3664c3
[Doc] Add missing API endpoints to security documentation ( #40532 )
...
Signed-off-by: Russell Bryant <rbryant@redhat.com >
2026-04-28 21:53:19 +00:00
wang.yuqi
a8208e6a81
[Examples] Resettle features examples. ( #40995 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-28 00:33:41 -07:00
Juan Pérez de Algaba
57861ae48d
(security) Fix SSRF in batch runner download_bytes_from_url ( #38482 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-03-30 07:10:01 +00:00
Juan Pérez de Algaba and Russell Bryant
b111f8a61f
fix(security): Add VLLM_MAX_N_SEQUENCES environment variable and enforce limit ( #37952 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Russell Bryant <rbryant@redhat.com >
Co-authored-by: Russell Bryant <rbryant@redhat.com >
2026-03-27 09:02:10 -04:00
Kunshang Ji
53ec16a705
[Hardware] Replace torch.cuda.device_count/current_device/set_device API ( #36145 )
...
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-03-12 07:57:47 -07:00
Russell Bryant
d460a18fc6
[Docs] Expand --allowed-media-domains security guidance with threat details ( #36506 )
...
Signed-off-by: Russell Bryant <rbryant@redhat.com >
2026-03-09 17:43:42 +00:00
Harry Mellor
a0f44bb616
Allow markdownlint to run locally ( #36398 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-08 20:05:24 -07:00
Kunshang Ji
66a2209645
[Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize ( #36085 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-03-05 10:36:39 +00:00
Russell Bryant
636ee223ac
[Docs] Document security risks of GPT-OSS Python tool ( #35139 )
...
Signed-off-by: Russell Bryant <rbryant@redhat.com >
2026-03-04 20:27:31 +00:00
Manrique Vargas
28028dff2f
fix(docs): use static rdzv backend in multi-node troubleshooting script ( #34784 )
...
Signed-off-by: machov <mv1742@nyu.edu >
2026-03-04 17:15:35 +00:00
160424a937
[Bugfix] Fix CUDA compatibility path setting for both datacenter and consumer NVIDIA GPUs ( #33992 )
...
Signed-off-by: Seungmin Kim <8457324+ehfd@users.noreply.github.com >
Signed-off-by: Andrew Mello <19512127+88plug@users.noreply.github.com >
Co-authored-by: 88plug <19512127+88plug@users.noreply.github.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-02-25 18:15:51 -08:00
Mark McLoughlin and Harry Mellor
5cc7c4452e
[Metrics] Add Prometheus counters for Model FLOPs Utilization (MFU) ( #30950 )
...
Export the existing Model FLOPs Utilization (MFU) metrics via Prometheus.
`--enable-mfu-metrics` is required for these to be exposed.
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
2026-02-23 15:01:07 +00:00
Isotr0py
19fab44152
[Doc] Update Encoder-Decoder models support doc with Florence-2 ( #34581 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-02-15 04:18:57 -08:00
Nicolò Lucchesi
81a90e5277
[Docs] Add bart-plugin to docs ( #33905 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-02-05 12:20:25 +00:00
