iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,521 Commits 3 Branches 0 Tags
5145e35e50ca7fa2bf381eaacdb30e234d774d1c
Commit Graph
2816 Commits
Author SHA1 Message Date
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
Chinmay-Kulkarni-AMDandCursor Agent 533d81b4b0 [ZenCPU][Test] Add zentorch/torch version pin consistency test (#52888) 
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-10-06 13:12:19 +08:00
Robert ShawClaude Opus 5.5Matthew Bonanni
 		2e3154aa18 [HiSparse] Log steady-state max concurrency and expose host-tier utilization gauges (#58949) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-05 14:20:32 -07:00
Raphaël RiallandandSimon Veitner c32495d58f [watermark] compatibility validation (#56801) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
 2026-10-05 14:37:13 -04:00
Giancarlo Delfin 88f3c601b3 [Docs] Add reviewer area of interest for TheEpicDolphin (#60092) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-10-05 18:35:19 +00:00
Sohaib Ahmed 64cb683842 [Model] Add Nemotron 3.5 ASR transcription support (#59827) 
Signed-off-by: Sohaib-Ahmed21 <sohaibahmed1919@gmail.com>
 2026-10-05 16:28:09 +00:00
Jee Jee Li f9c9e8ac24 [LoRA] Remove tensorizer (#60024) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-10-05 07:01:56 -07:00
cjackal 0e468adb43 [Model] Extend device-side mm normalization to Kimi K2.5 / K3 (#59278) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-10-05 21:28:13 +08:00
Harshal JanjaniHarry MellorClaude Opus 5
 		34051ad714 feat: Add video support for the Transformers backend (#57441)
pre-commit / pre-run-check (push) Canceled after 0s
Details
pre-commit / pre-commit (push) Canceled after 0s
Details 
Signed-off-by: Harshal Janjani <harshaljanjani@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-05 00:51:15 -07:00
Shenglei FuClaude Opus 5.5Wentao Ye
 		1388100560 [Kernel][LoRA] Add deterministic split-K=8 LoRA shrink for batch invariance (#59377) 
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-04 18:46:16 -04:00
Lio EinaudiYifan DaCodeBuddyClaude Fable 5.1Wentao Ye
 		89439db727 [CI][Docs] Fix decode/prefill consistency test prefix construction; add GLM-4-9B and Phi-4 to batch-invariant tested models (#53692) 
Signed-off-by: Yifan Da <1363818765@qq.com>
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Yifan Da <1363818765@qq.com>
Co-authored-by: CodeBuddy <noreply@codebuddy.dev>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-04 11:55:18 -04:00
aoshen02andClaude Opus 5.5 5e56e9af2a [Feature] Release the CUDA graph pool on sleep (#59160) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 01:13:10 +00:00
aoshen02aoshen02Claude Opus 5.5
 		e319f86f15 [Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-03 14:17:09 +00:00
Martin Hickey bc21cba967 [Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-03 19:38:25 +08:00
Nils Matteson 9fdb174678 [Bugfix] Avoid InfiniBand state in TP1 snapshots (#59699) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-10-03 03:59:17 +00:00
Shang WangClaude Opus 5.5Tyler Michael SmithShengqi Chen
 		8e443871f0 [CI/Build][NVIDIA] Build Rubin images on the public nvidia/cuda base image (#59288) 
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-10-03 00:12:33 +00:00
yifanFenggClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Wentao Ye
 		6e517b15c1 [Docs] Add ERNIE 4.5 to batch invariance tested models (#58476) 
Signed-off-by: yifanFengg <evanfengff@gmail.com>
Signed-off-by: yifanFengg <yifanfen@amazon.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-02 20:37:32 +00:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
Misha GoinandClaude 1ab18599ca [Agents] Add pre-commit check for agent files and skills (#59657) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-02 09:38:22 -07:00
+8
Lucas WilkinsonClaude Opus 5.5JartXTaneem IbrahimbnellnmItay EtelissimondanielssonRoberto L. CastroNetanel HaberItay AlroyFelix MartyVarun Sundar RabindranathMicah WilliamsoncjackalSummer YangRohan138kliuaeMisha Goin
 		7b6be2f1f4 [Docs] Add Reviewers page (#59459) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-02 16:11:31 +00:00
Rajeev Warrierandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 30e956fad3 [Docs] Fix stale W4A16 NVFP4 default kernel in ModelOpt docs (#59135) 
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-02 13:21:43 +00:00
Harry MellorandClaude Opus 5.5 6e4efafc6e [Model] Migrate GPT-NeoX, Phi, Seed-OSS and Jais2 to the Transformers modeling backend (#59701) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 11:18:35 +00:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		4056c8ac1f [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-01 23:04:21 -07:00
Teng MaandTeng Ma 1a9eaa3a68 [Doc] Document ECMooncakeConnector for EPD disaggregation (#59196) 
Signed-off-by: Teng Ma <stmatengss@gmail.com>
Co-authored-by: Teng Ma <stmatengss@users.noreply.github.com>
 2026-10-02 12:10:33 +08:00
Martin Hickey fcfacc3499 [Bugfix][Frontend] Seed derender detokenization from the prompt (#59046) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-02 03:17:56 +00:00
Harry MellorandClaude Opus 5.5 8faacd707f [Model] Migrate Glm, Arcee, CWM and Mellum to the Transformers modeling backend (#59679) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 03:15:14 +00:00
Yoni GozlanCursorBugen ZhaoClaude Opus 5.5
 		9bd993564f [Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 02:33:00 +00:00
Hyunjae Wooandaoshen02 5a661487e2 [Feature] Add native ModelExpress weight transfer backend (#58399) 
Signed-off-by: Hyunjae Woo <hwoo@nvidia.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-02 02:29:35 +00:00
Flora Feng 785b990116 [Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT (#58623) 
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-02 02:25:48 +00:00
Flora Feng a5105dba05 [Frontend] Port Step-3.5 parsers to the streaming parser engine (#59321) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-01 22:18:37 +00:00
Zijing Liu ea84051819 [KVConnector][NIXL] Count completion notifications that arrive after KV expiry (#58875) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
 2026-10-01 21:30:58 +00:00
Sundri Lai bcee730b1a [Docs] Remove references to removed env vars (#59530) 
Signed-off-by: Sundri Lai <laiyanting.neu@gmail.com>
 2026-10-01 08:26:14 +00:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
vx120Cuiping Chengaoshen02Claude Opus 5.5
 		70be6aae19 [Feature][Spec Decode] Support sampling mask replay for MRV2 MTP (#59359) 
Signed-off-by: Cuiping Cheng <chengcuipingswu@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Cuiping Cheng <chengcuipingswu@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 13:51:39 +08:00
Guy StoneandClaude Fable 5 37d6174096 [Metrics] Add --custom-histogram-buckets to override histogram bucket families (#48867) 
The core engine histograms ship with default bucket boundaries tuned for
typical serving workloads. The `--custom-histogram-buckets` option replaces
the boundaries of one or more bucket families. Use it, for example, to track
sub-300ms latency SLOs with the request-phase histograms, whose smallest
default boundary is 0.3s.

Signed-off-by: Guy Stone <guys@spotify.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-10-01 01:03:39 +00:00
vx120aoshen02Claude Fable 5.1
 		a47e4a5568 [Doc] Score centering via top-k processed logprobs (#59361) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 08:57:00 +08:00
Benjamin ChislettandCopilot 8fb16ea5a6 [Docs] Add PR checklist skill for coding agents (#57084) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-09-30 14:47:28 -07:00
Nils Matteson 51048b13ce [Docs] Clarify snapshot runtime image support (#59127) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-09-30 20:31:21 +00:00
Thien Tranandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8873ac83bb [Docs] Add @gau-nernst to CODEOWNERS and committers (#59369) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-30 16:23:40 +00:00
Raphaël RiallandSimon VeitnerLucas Wilkinson
 		0f8b398158 [watermarking] add context deduplication support to speculative decoding (#56807) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-30 12:22:10 -04:00
Wei ZhaoandCodex 279012f94d [Docs] Add @wzhao18 to NVIDIA integration and kv offloading code owners (#59456) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-30 16:12:12 +00:00
JulienDarveBugen ZhaoClaude Opus 5.5
 		f28a508162 [Feature][Rust Frontend] Add Shutdown control RPC (#59316) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-30 12:49:25 +00:00
Harry MellorandClaude Opus 5.5 b22494cc0c [Docs] Reinstate docs build gate for PRs (#59411) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 13:41:53 +01:00
Rukhaiya2004andLi, Jiang <jiang1.li@intel.com> 72e7874fa6 [Hardware][Power] Enable W4A16 (AWQ & GPTQ) quantization on POWER10 using VSX (#59149) 
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 19:44:48 +08:00
yuvalluriaandClaude Sonnet 4.6 863475eb9c docs: add gemma-2, SmolLM2, Qwen2.5-Coder, Llama-3.2-1B to batch invariance tested models (#54441) 
Signed-off-by: Yuval Luria <yluria@redhat.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-09-30 11:13:23 +00:00
cjackal 0a30bc3f9a [Model] Extend device-side mm normalization to GLM4V/GLM5Next (#55389) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-09-30 15:44:46 +08:00
Louie Tsai 1b77cc3d67 [Benchmark] Add sweep warmup and failure recovery (#57305) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-09-30 11:16:41 +08:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
Matthew BonanniandClaude Opus 5.5 2798f66860 [Bugfix][HiSparse] Resolve MTP verification rows with a union residency kernel (#59235) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 03:01:45 +00:00
Shimi BandielandClaude 4dc57d44c4 [Frontend] Attach resolved logprobs to streaming derender chunks (#55029) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-29 18:43:44 +00:00