iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,316 Commits 3 Branches 0 Tags
bcee730b1a9d25f0fd283a0ef6c19133ebeebf4f
Commit Graph
2785 Commits
Author SHA1 Message Date
Sundri Lai bcee730b1a [Docs] Remove references to removed env vars (#59530) 
Signed-off-by: Sundri Lai <laiyanting.neu@gmail.com>
 2026-10-01 08:26:14 +00:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
vx120Cuiping Chengaoshen02Claude Opus 5.5
 		70be6aae19 [Feature][Spec Decode] Support sampling mask replay for MRV2 MTP (#59359) 
Signed-off-by: Cuiping Cheng <chengcuipingswu@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Cuiping Cheng <chengcuipingswu@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 13:51:39 +08:00
Guy StoneandClaude Fable 5 37d6174096 [Metrics] Add --custom-histogram-buckets to override histogram bucket families (#48867) 
The core engine histograms ship with default bucket boundaries tuned for
typical serving workloads. The `--custom-histogram-buckets` option replaces
the boundaries of one or more bucket families. Use it, for example, to track
sub-300ms latency SLOs with the request-phase histograms, whose smallest
default boundary is 0.3s.

Signed-off-by: Guy Stone <guys@spotify.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-10-01 01:03:39 +00:00
vx120aoshen02Claude Fable 5.1
 		a47e4a5568 [Doc] Score centering via top-k processed logprobs (#59361) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 08:57:00 +08:00
Benjamin ChislettandCopilot 8fb16ea5a6 [Docs] Add PR checklist skill for coding agents (#57084) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-09-30 14:47:28 -07:00
Nils Matteson 51048b13ce [Docs] Clarify snapshot runtime image support (#59127) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-09-30 20:31:21 +00:00
Thien Tranandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8873ac83bb [Docs] Add @gau-nernst to CODEOWNERS and committers (#59369) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-30 16:23:40 +00:00
Raphaël RiallandSimon VeitnerLucas Wilkinson
 		0f8b398158 [watermarking] add context deduplication support to speculative decoding (#56807) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-30 12:22:10 -04:00
Wei ZhaoandCodex 279012f94d [Docs] Add @wzhao18 to NVIDIA integration and kv offloading code owners (#59456) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-30 16:12:12 +00:00
JulienDarveBugen ZhaoClaude Opus 5.5
 		f28a508162 [Feature][Rust Frontend] Add Shutdown control RPC (#59316) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-30 12:49:25 +00:00
Harry MellorandClaude Opus 5.5 b22494cc0c [Docs] Reinstate docs build gate for PRs (#59411) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 13:41:53 +01:00
Rukhaiya2004andLi, Jiang <jiang1.li@intel.com> 72e7874fa6 [Hardware][Power] Enable W4A16 (AWQ & GPTQ) quantization on POWER10 using VSX (#59149) 
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 19:44:48 +08:00
yuvalluriaandClaude Sonnet 4.6 863475eb9c docs: add gemma-2, SmolLM2, Qwen2.5-Coder, Llama-3.2-1B to batch invariance tested models (#54441) 
Signed-off-by: Yuval Luria <yluria@redhat.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-09-30 11:13:23 +00:00
cjackal 0a30bc3f9a [Model] Extend device-side mm normalization to GLM4V/GLM5Next (#55389) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-09-30 15:44:46 +08:00
Louie Tsai 1b77cc3d67 [Benchmark] Add sweep warmup and failure recovery (#57305) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-09-30 11:16:41 +08:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
Matthew BonanniandClaude Opus 5.5 2798f66860 [Bugfix][HiSparse] Resolve MTP verification rows with a union residency kernel (#59235) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 03:01:45 +00:00
Shimi BandielandClaude 4dc57d44c4 [Frontend] Attach resolved logprobs to streaming derender chunks (#55029) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-29 18:43:44 +00:00
Isotr0py 31876d8ea1 [Bugfix] Fix TorchCodec audio IO correctness (#58364) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-29 11:34:48 -05:00
Yixin DongCodexyuchuanBugen ZhaoClaude Opus 5.5
 		dfc8e0f3e2 [Frontend] Support strict MiMo-V2.6 tool calling (#58019) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ubospica <ubospica@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-29 12:47:47 +00:00
Harry MellorandClaude Opus 5.5 998490cd9f [Docs] Speed up docs build ~5x (#58705) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 12:16:33 +01:00
Andreas KaratzasandOpenAI Codex 3acddf01e7 [Bugfix][ROCm] Keep Mooncake bootstrap ports bound during startup (#58967) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-29 04:14:26 -05:00
Jakub ZakrzewskiandOpenAI Codex 70dc122f81 [MM] Enable device normalization for Llama Nemotron VL Embed/Rerank (#57928) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-29 08:33:52 +00:00
Jakub ZakrzewskiOpenAI CodexOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d86cf91240 [MM][Mistral3] Image preprocessing optimization (#57531) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-29 06:53:38 +00:00
cjackalandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 491f44adfa [Model] Extend device-side mm normalization to Qwen3VL/Qwen3.5/Qwen4Next (#51289) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-29 11:44:54 +08:00
Shijie LyuShijie LyuClaude
 		c6843d397c [Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo (#55771) 
Signed-off-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-29 10:36:22 +08:00
qizixi 6d3ea3c2c9 [KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill (#50499) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
 2026-09-28 18:50:46 +02:00
linitra24andJee Jee Li 953f90d25f [LoRA] Support variable num_labels for sequence classification (#57766) 
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-28 14:58:35 +00:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
Louie Tsai ccfd1cea75 [CPU] Include vLLM Recipes tooling in release image to deploy models using vLLM Recipes (#58796) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-09-28 16:46:53 +08:00
Rehan KhanandLi, Jiang <jiang1.li@intel.com> fd2f3fcfa5 [CPU] Add video inferencing via torchcodec on s390x (#58693) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-28 14:16:59 +08:00
HarshilClaudeChauncey
 		af7f9488c2 [Benchmark] Add Responses API backend to vllm bench serve (#54628) 
Signed-off-by: QHarshil <harshil_c@hotmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-28 05:04:11 +00:00
jz-yoloandjz-yolo 4a190dc518 [Model] Enable LoRA support for RobertaForSequenceClassification (#58884) 
Signed-off-by: jz-yolo <jz-yolo@users.noreply.github.com>
Signed-off-by: Jane Zhu <jane.zhu@slack-corp.com>
Co-authored-by: jz-yolo <jz-yolo@users.noreply.github.com>
 2026-09-28 03:50:12 +00:00
Chao Lei 386ac2573e [Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout (reopens #55763) (#58919) 2026-09-27 19:47:36 +00:00
Lio Einaudiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 0c87a197b8 [Bugfix] Disable sequence parallelism / async TP under batch invariance and add a TP regression test (#56377) 
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-26 22:19:08 -04:00
mevince cd1daf58a9 [Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector (#57251) 
Signed-off-by: Vincent <vincexxchan@gmail.com>
 2026-09-26 17:17:11 -07:00
Juan Pérez de AlgabaandCursor 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-26 11:10:35 +00:00
qizixiOpenAI CodexClaude Opus 5.5Nick Hill
 		a4eb3f25d6 [Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-26 17:01:17 +08:00
Ke Wenmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>aoshen02
 		b6d2b4fc25 [RL] Add sharding-aware NCCL M2N weight transfer (#51520) 
Signed-off-by: Ke Wen <kwen@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-26 01:38:53 +00:00
Yousafandsfeng33 83db8c8390 [Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957) 
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-09-25 20:43:08 +00:00
Andrii SkliarAndrii SkliarCodexBenjamin Chislett
 		73a78e6f1f [SpecDecode] Add LiLiCorr drafter (#57934) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-09-25 10:08:28 -04:00
Isotr0py 24928a4d62 [UX][Frontend] Introduce vllm preload cli for fast restart (#56680) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 06:56:39 +00:00
wang.yuqiIsotr0pyIsotr0py
 		8a269919f4 [MM] Add Triton kernel for mm_input_normal. (#56798) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 04:31:20 +00:00
Zijing LiuYifan QiaoCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7a9406690e [KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode (#58472) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 23:55:47 +00:00
fxmarty-amdmgoinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		90a9515006 [Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 12:48:14 -07:00
Misha Goin 58aa2f1ef1 [Refactor] Move auxiliary files out of the repository root (#58572) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-24 12:38:56 -07:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
Andreas KaratzasandCodex 153f2ba1b9 [Compile][CI] Honor Triton cache overrides and add AMD timeout headroom (#58474) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-24 01:27:07 -05:00
Lio Einaudi 711fc55c10 [Perf] Use breakable CUDA graphs (no torch.compile) by default under VLLM_BATCH_INVARIANT so the tuned matmul configs see the runtime M (#57586) 
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
 2026-09-23 14:39:38 -04:00