iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,069 Commits 3 Branches 0 Tags
d7f5722d7b487dd6794f638043375a95ebecfef7
Commit Graph
115 Commits
Author SHA1 Message Date
mevince cd1daf58a9 [Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector (#57251) 
Signed-off-by: Vincent <vincexxchan@gmail.com>
 2026-09-26 17:17:11 -07:00
Juan Pérez de AlgabaandCursor 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-26 11:10:35 +00:00
LucainandClaude Opus 5 6b858751f6 [docs] Fix legacy hf CLI references (vllm) (#57958) 
Signed-off-by: Wauplin <lucainp@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-21 15:32:27 +00:00
Kevin H. LuuOpenAI CodexTyler Michael Smith
 		caf56ce9da [CI] Pass the scale-out endpoint flag in EC E2E (#56819) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-09-15 12:25:30 +08:00
Martin Hickey 934b1fcbcb [Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-14 10:02:14 +00:00
Sören Dréano 9a35c081e8 [Docs] Correct API key authentication scope (/inference does NOT bypass the API key) (#56269) 
Signed-off-by: Sören Dréano <gailenstorm@gmail.com>
 2026-09-11 16:54:35 +00:00
Francisco Javier ArceoandOpenAI Codex 6a2a2bb02b [Frontend] Add stateless /v1/responses/render endpoint (#50195) 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-07 12:50:42 +00:00
Francisco Javier Arceo c00091e026 [Frontend] Gate scale-out endpoints behind opt-in flag (#54579) 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
 2026-09-02 14:31:14 +08:00
lzhan011andCyrus Leung ce7391712b [Bugfix][Security] Bound embedding densification before to_dense() (#54632) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-01 08:45:14 +00:00
Juan Pérez de Algaba 0d7d5ed0b2 fix(security): enforce VLLM_MAX_AUDIO_CLIP_FILESIZE_MB on all audio paths (#53561) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-08-24 21:37:02 +00:00
Juan Pérez de Algaba 9f295fe8ce [Docs][Security] Document multimodal media UUID security implications (#53582) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-08-24 07:19:18 -07:00
Clinton ThomasandLucas Bourtoule 2ec6f0d71e Reject oversized media before fully downloading it (#51896) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
 2026-08-24 01:15:09 +00:00
Russell Bryant 6d8cd88e8d [Docs] document cache salting for prefix cache timing side-channel mitigation (#39082) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-08-21 09:06:49 -07:00
Connor CarpenterandNick Hill f473870ecf [Rust Frontend][gRPC] Add RL lifecycle control (#51316) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-14 11:32:39 -07:00
Juan Pérez de AlgabaandCursor 640a09086d Fix DoS via sample-rate forgery bypassing audio decode duration guard (#49948) 
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-10 15:49:17 +00:00
Isotr0py 1180b604f8 [Multimodal] Expose mm hash algothrim selection to cli args (#49686) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-31 05:25:19 +00:00
Juan Pérez de Algaba 89d97d9a16 docs(security): document Ray cluster trust model and env var propagation (#50397) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-07-30 11:52:55 +00:00
Harry Mellor 3e74c60b9c [Docs] Use gen-files for generated docs content (#49587) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-25 14:43:21 +00:00
Harry Mellor 84d26b9ee3 [Model] Remove Plamo2 (#49729) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-24 13:54:24 -07:00
Martin Hickey 4d4e04f452 [Render] Add round trip parity test and docs for derender (#48617) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-17 05:02:46 +00:00
aoshen02Claude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		83762b77b0 [Frontend] Add /abort_requests to the RLHF dev API router (#47173) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-12 14:21:02 +08:00
Martin Hickey f7fc0ca993 [Frontend] Add endpoint plugins framework (#47454) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-08 10:00:41 +08:00
Juan Pérez de AlgabaRussell BryantRussell Bryant
 		5c4db60f01 docs(security): document gRPC interface as insecure for private use only (#45903) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
Co-authored-by: Russell Bryant <russell.bryant@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2026-07-01 12:39:57 +00:00
Juan Pérez de Algaba 364ee36af1 fix(security): prevent image decompression bomb OOM denial of service (#47010) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-30 09:39:22 +00:00
AgenticSparkandCopilot bf292b5f6b [Docs] Remove BambaForCausalLM from supported hybrid models list (#46071) 
Signed-off-by: liejiang <jianglie2023@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-26 08:02:50 +00:00
Tiezhen WANGClaudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d14e551a53 [Model] Remove MiniMaxText01, MiniMaxVL01, MiniMaxForCausalLM (#45993) 
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-22 15:20:46 +08:00
Mohammad Miadh Angkad da03e549b3 [UX] Add a persistent cache for FlashInfer autotuning (#42537) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-05-18 20:25:37 -07:00
DustHunterQwen-Coderwang.yuqi
 		39c67d714e fix: add API key authorization to /v2 endpoints (#42594) 
Signed-off-by: DustHunter <dusthunter@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-16 01:29:27 +00:00
Vinay R DamodaranandRussell Bryant 5497ffbf7c Add documentation about vLLM FIPS compliance (#42190) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Signed-off-by: Vinay R Damodaran <vrdn@hey.com>
Co-authored-by: Russell Bryant <russell.bryant@gmail.com>
 2026-05-11 18:17:02 +00:00
Russell Bryant 5a0a8fc1ea [Docs] add cache directory security guidance (#38920) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-05-06 16:54:29 -07:00
Russell Bryant f05f3664c3 [Doc] Add missing API endpoints to security documentation (#40532) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-04-28 21:53:19 +00:00
wang.yuqi a8208e6a81 [Examples] Resettle features examples. (#40995) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-28 00:33:41 -07:00
Juan Pérez de Algaba 57861ae48d (security) Fix SSRF in batch runner download_bytes_from_url (#38482) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-03-30 07:10:01 +00:00
Juan Pérez de AlgabaandRussell Bryant b111f8a61f fix(security): Add VLLM_MAX_N_SEQUENCES environment variable and enforce limit (#37952) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2026-03-27 09:02:10 -04:00
Kunshang Ji 53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00
Russell Bryant d460a18fc6 [Docs] Expand --allowed-media-domains security guidance with threat details (#36506) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-03-09 17:43:42 +00:00
Harry Mellor a0f44bb616 Allow markdownlint to run locally (#36398) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-08 20:05:24 -07:00
Kunshang Ji 66a2209645 [Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-05 10:36:39 +00:00
Russell Bryant 636ee223ac [Docs] Document security risks of GPT-OSS Python tool (#35139) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-03-04 20:27:31 +00:00
Manrique Vargas 28028dff2f fix(docs): use static rdzv backend in multi-node troubleshooting script (#34784) 
Signed-off-by: machov <mv1742@nyu.edu>
 2026-03-04 17:15:35 +00:00
Seungmin Kim88plugMichael Goin
 		160424a937 [Bugfix] Fix CUDA compatibility path setting for both datacenter and consumer NVIDIA GPUs (#33992) 
Signed-off-by: Seungmin Kim <8457324+ehfd@users.noreply.github.com>
Signed-off-by: Andrew Mello <19512127+88plug@users.noreply.github.com>
Co-authored-by: 88plug <19512127+88plug@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-25 18:15:51 -08:00
Mark McLoughlinandHarry Mellor 5cc7c4452e [Metrics] Add Prometheus counters for Model FLOPs Utilization (MFU) (#30950) 
Export the existing Model FLOPs Utilization (MFU) metrics via Prometheus.

`--enable-mfu-metrics` is required for these to be exposed.

Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-02-23 15:01:07 +00:00
Isotr0py 19fab44152 [Doc] Update Encoder-Decoder models support doc with Florence-2 (#34581) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-15 04:18:57 -08:00
Nicolò Lucchesi 81a90e5277 [Docs] Add bart-plugin to docs (#33905) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-02-05 12:20:25 +00:00
Wentao Ye d88a1df699 [Deprecation] Deprecate profiling envs (#33722) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-04 05:58:21 +00:00
jma99_2333andRoger Wang 22d9a056d5 Support clear mm and encoder cache (#33452) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-01-31 15:22:25 +00:00
Andrew Bennett f243abc92d Fix various typos found in docs (#32212) 
Signed-off-by: Andrew Bennett <potatosaladx@meta.com>
 2026-01-13 03:41:47 +00:00
Harry Mellor 0b544e6476 [Docs] Fix some snippets (#31378) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-26 12:47:41 +00:00
Isotr0pyandyoukaichao 2532f437ee [Doc] Add troubleshooting for Triton PTX error about undefined gpu-name (#31338) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-12-25 02:26:34 -08:00
Didier DurandandWentao Ye 1a55cfafcb [Doc]: fixing typos in various files (#30540) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Signed-off-by: Didier Durand <2927957+didier-durand@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-12-14 02:14:37 -08:00