iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,824 Commits 3 Branches 0 Tags
3709632ff2944a5f2ecdacc84ede5cd134b7ae08
Commit Graph
330 Commits
Author SHA1 Message Date
wang.yuqi c1692c85f1 [Misc] mv chat_utils.py to renderers. (#60522) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-10 06:50:34 +00:00
AlecCodexBugen ZhaoClaude Opus 5.5
 		8ea646b827 [Rust Frontend][gRPC] Remove GenerateStream producer task so disconnects abort requests (#60922) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-10 04:59:35 +00:00
Bugen ZhaoandClaude Opus 5.5 35e920d671 [Rust Frontend] Support --stream-interval and per-request stream_interval (#60589) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 14:10:42 -07:00
Bugen ZhaoandClaude Opus 5.5 5dc9bcd8cd [Rust Frontend] Pass reasoning controls through render -> generate (#60546) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:04:49 -07:00
Bugen ZhaoandClaude Opus 5.5 ce4603cad6 [Bugfix][Rust Frontend] Pass through raw characters in byte-level decode (#59861) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:04:00 -07:00
Bugen ZhaoandClaude Opus 5.5 9a585c62aa [Rust Frontend] Box rarely-set fields of per-token output types (#60588) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:01:16 -07:00
AlecBugen ZhaoClaude Opus 5.5
 		51ef239768 [Rust Frontend] Carry reasoning controls over gRPC (#59836) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:11:19 +08:00
devshah-cohereandBugen Zhao b49c921a27 [Bugfix][Rust Frontend] Always include token_ids in completion stream choices (#60286) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: devshah-cohere <dev.shah@cohere.com>
 2026-10-08 19:11:55 +08:00
Alec 29ae81fe83 [Rust Frontend] Preserve explicit zero sampling values in gRPC requests (#60115) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
 2026-10-08 07:27:47 +00:00
Bugen ZhaoOpenAI CodexClaude Opus 5.5
 		f226af0e11 [Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners (#54113) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 06:25:27 +00:00
Alec 14e3902d51 [Frontend][Rust] Add gRPC forbidden token sequences and cache usage (#59837) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
 2026-10-07 07:46:44 -07:00
Bugen ZhaoandClaude Opus 5.5 bc8dd4ad27 [Rust Frontend] Share SchemaRoot between argument coercion and grammars (#59408) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 08:28:27 +00:00
Bugen ZhaoandClaude Opus 5.5 7436a7f119 [Rust Frontend] Backtrack from safe text at a complete marker (#59563) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 07:45:15 +00:00
Cameron QuiliciCam QuiliciNick HillYifan Qiao
 		e37e51dd24 [Metrics] Expose cached prompt tokens by cache tier (#56318) 
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-10-06 14:52:05 -04:00
Yin LiYin LiClaude
 		7b665f7a58 [Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) (#58181) 
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-06 17:22:47 +00:00
Rodrigo Garcia 3faf214051 [vllm-bench, feature] Added support for Mooncake-style, timed-traces replay. (#55937) 2026-10-06 20:39:35 +08:00
Wentao Ye 3e182185aa [Deprecation] Deprecate sonet dataset as scheduled (#60129) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-06 08:15:12 -04:00
Bugen ZhaoandClaude Opus 5.5 6bbad6acdf [Rust Frontend] Pass tool defer_loading through to chat templates (#60203) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-06 18:00:15 +08:00
Bugen ZhaoandClaude Opus 5.5 b3082d3044 [Rust Frontend] Remove the PyO3 tool-parser bridge (#59744) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
aoshen02aoshen02Claude Opus 5.5
 		e319f86f15 [Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-03 14:17:09 +00:00
Misha GoinandClaude 1ab18599ca [Agents] Add pre-commit check for agent files and skills (#59657) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-02 09:38:22 -07:00
AlecBugen ZhaoClaude Opus 5.5
 		9d2a6f52b2 [Rust Frontend] Expose gRPC port in Python vllm serve (#59659) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 08:14:37 +00:00
ai-jzai-jzBugen Zhao
 		e5d33a36e7 [Bugfix][Rust Frontend] Preserve whitespace in GLM string arguments (#59654) 
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 10:10:35 +08:00
Idder Ghanbaja f3b77eff64 [Rust][Benchmark] Warn when temperature is left to the server default (#59247) 2026-10-01 21:03:03 +08:00
Bugen ZhaoandClaude Opus 5.5 24fb5fc08c [Rust Frontend] Add hf parser for response templates (#59005) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 08:16:38 +00:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
Bugen ZhaoandClaude Opus 5.5 b2d3a3b9ad [Rust Frontend] Split argument grammars into schema resolution and per-model rendering (#59395) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 05:43:18 +00:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ac9126e58a [Rust Frontend] Snapshot structural-tag grammars as readable outlines (#59393) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 05:09:43 +00:00
Bugen ZhaoandClaude Opus 5.5 36229e1337 [Rust Frontend] Token-aware marker parsing for Kimi K3 (#58358) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 04:21:50 +00:00
Bugen ZhaoandClaude Opus 5.5 ec5ef92c87 [Rust Frontend] Anchor tokens after pending UTF-8 bytes at their own first byte (#58357) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <bugen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 04:21:50 +00:00
ZhenchengLinClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Roy Wang
 		31ec1c3aa1 [Bugfix][Rust Frontend] Stop vllm-bench chat latency at the last token (#59251) 
Signed-off-by: zhenchenglin <zhl132@ucsd.edu>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Roy Wang <jasonailu87@gmail.com>
 2026-10-01 02:04:19 +00:00
Juan Pérez de AlgabaandCursor cff08b461e [Security] Fix chat template resource-exhaustion DoS (GHSA-4hhp-h66f-… (#50300) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-30 08:32:00 -07:00
Juan Pérez de AlgabaandCursor eef8b15d7c [Security] Bump pyjwt and rand for remaining GHSAs (#59427) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-30 14:26:16 +00:00
JulienDarveBugen ZhaoClaude Opus 5.5
 		f28a508162 [Feature][Rust Frontend] Add Shutdown control RPC (#59316) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-30 12:49:25 +00:00
Juan Pérez de AlgabaandCursor a4c4bb418e [Security] Bump remaining Dependabot packages (excl. ignored) (#59315) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-30 05:16:31 -07:00
Ronaldaoshen02Claude Opus 5.5aoshen02
 		713ec07655 [Frontend][RL] Track HTTP weight operation outcomes and concurrency (#55781) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 09:19:43 +00:00
Bugen ZhaoandClaude Opus 5.5 87228cff01 [Rust Frontend] Replay roundtrip output grammars through XGrammar (#59143) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 07:29:01 +00:00
Ronaldaoshen02Claude Opus 5.5
 		e5f3d08d72 [Frontend][RL] Align sleep-mode API responses and operation metrics (#52864) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 23:16:38 -07:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
Bugen ZhaoandClaude Sonnet 5.5 8039d3cbfd [Rust Frontend] Use the MiMo structural-tag builder (#59148) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
 2026-09-30 01:34:46 +00:00
Bugen ZhaoandClaude Opus 5.5 afac509a33 [Bugfix][Rust Frontend] Skip engine-derived metrics under --disable-log-stats (#59205) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 11:01:54 -07:00
ReidBugen ZhaoClaude Opus 5.5
 		3f3fbe286f [Bugfix][Rust Frontend] Account for new requests in DP routing (#58956) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-29 10:24:28 +00:00
Yashasvi AsthanaClaudeBugen Zhao
 		9af952c555 [Bugfix][Rust Frontend] Support raise_exception in chat templates (#59020) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yashasvi Asthana <yashasvi.asthana@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-29 08:01:52 +00:00
Bugen ZhaoandClaude Opus 5.5 e05095c9e1 [Rust Frontend] Feed parser tests and benchmarks attributed input (#59048) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 06:23:54 +00:00
Bugen ZhaoandClaude Opus 5.5 28f6739576 [Rust Frontend] Relax schema-aware tool argument conversion (#59004) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 05:40:41 +00:00
Nick HillandRobert Shaw 42dbe4578c [Core] Rework scheduler skipped_waiting queue (#58947) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-09-28 20:03:16 +00:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
Juan Pérez de AlgabaandCursor 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-26 11:10:35 +00:00
zhecClaudeBugen Zhao
 		ddd6fbca14 [Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: zhec <chengyunfei@ruc.edu.cn>
 2026-09-26 02:18:58 +00:00