iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,539 Commits 3 Branches 0 Tags
049507aa76200090f8bc6ca4e48ab0e09f2f0d29
Commit Graph
303 Commits
Author SHA1 Message Date
Rodrigo Garcia 3faf214051 [vllm-bench, feature] Added support for Mooncake-style, timed-traces replay. (#55937) 2026-10-06 20:39:35 +08:00
Wentao Ye 3e182185aa [Deprecation] Deprecate sonet dataset as scheduled (#60129) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-06 08:15:12 -04:00
Bugen ZhaoandClaude Opus 5.5 6bbad6acdf [Rust Frontend] Pass tool defer_loading through to chat templates (#60203) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-06 18:00:15 +08:00
Bugen ZhaoandClaude Opus 5.5 b3082d3044 [Rust Frontend] Remove the PyO3 tool-parser bridge (#59744) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
aoshen02aoshen02Claude Opus 5.5
 		e319f86f15 [Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-03 14:17:09 +00:00
Misha GoinandClaude 1ab18599ca [Agents] Add pre-commit check for agent files and skills (#59657) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-02 09:38:22 -07:00
AlecBugen ZhaoClaude Opus 5.5
 		9d2a6f52b2 [Rust Frontend] Expose gRPC port in Python vllm serve (#59659) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 08:14:37 +00:00
ai-jzai-jzBugen Zhao
 		e5d33a36e7 [Bugfix][Rust Frontend] Preserve whitespace in GLM string arguments (#59654) 
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 10:10:35 +08:00
Idder Ghanbaja f3b77eff64 [Rust][Benchmark] Warn when temperature is left to the server default (#59247) 2026-10-01 21:03:03 +08:00
Bugen ZhaoandClaude Opus 5.5 24fb5fc08c [Rust Frontend] Add hf parser for response templates (#59005) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 08:16:38 +00:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
Bugen ZhaoandClaude Opus 5.5 b2d3a3b9ad [Rust Frontend] Split argument grammars into schema resolution and per-model rendering (#59395) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 05:43:18 +00:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ac9126e58a [Rust Frontend] Snapshot structural-tag grammars as readable outlines (#59393) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 05:09:43 +00:00
Bugen ZhaoandClaude Opus 5.5 36229e1337 [Rust Frontend] Token-aware marker parsing for Kimi K3 (#58358) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 04:21:50 +00:00
Bugen ZhaoandClaude Opus 5.5 ec5ef92c87 [Rust Frontend] Anchor tokens after pending UTF-8 bytes at their own first byte (#58357) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <bugen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 04:21:50 +00:00
ZhenchengLinClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Roy Wang
 		31ec1c3aa1 [Bugfix][Rust Frontend] Stop vllm-bench chat latency at the last token (#59251) 
Signed-off-by: zhenchenglin <zhl132@ucsd.edu>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Roy Wang <jasonailu87@gmail.com>
 2026-10-01 02:04:19 +00:00
Juan Pérez de AlgabaandCursor cff08b461e [Security] Fix chat template resource-exhaustion DoS (GHSA-4hhp-h66f-… (#50300) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-30 08:32:00 -07:00
JulienDarveBugen ZhaoClaude Opus 5.5
 		f28a508162 [Feature][Rust Frontend] Add Shutdown control RPC (#59316) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-30 12:49:25 +00:00
Ronaldaoshen02Claude Opus 5.5aoshen02
 		713ec07655 [Frontend][RL] Track HTTP weight operation outcomes and concurrency (#55781) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 09:19:43 +00:00
Bugen ZhaoandClaude Opus 5.5 87228cff01 [Rust Frontend] Replay roundtrip output grammars through XGrammar (#59143) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 07:29:01 +00:00
Ronaldaoshen02Claude Opus 5.5
 		e5f3d08d72 [Frontend][RL] Align sleep-mode API responses and operation metrics (#52864) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 23:16:38 -07:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
Bugen ZhaoandClaude Sonnet 5.5 8039d3cbfd [Rust Frontend] Use the MiMo structural-tag builder (#59148) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
 2026-09-30 01:34:46 +00:00
Bugen ZhaoandClaude Opus 5.5 afac509a33 [Bugfix][Rust Frontend] Skip engine-derived metrics under --disable-log-stats (#59205) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 11:01:54 -07:00
ReidBugen ZhaoClaude Opus 5.5
 		3f3fbe286f [Bugfix][Rust Frontend] Account for new requests in DP routing (#58956) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-29 10:24:28 +00:00
Yashasvi AsthanaClaudeBugen Zhao
 		9af952c555 [Bugfix][Rust Frontend] Support raise_exception in chat templates (#59020) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yashasvi Asthana <yashasvi.asthana@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-29 08:01:52 +00:00
Bugen ZhaoandClaude Opus 5.5 e05095c9e1 [Rust Frontend] Feed parser tests and benchmarks attributed input (#59048) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 06:23:54 +00:00
Bugen ZhaoandClaude Opus 5.5 28f6739576 [Rust Frontend] Relax schema-aware tool argument conversion (#59004) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 05:40:41 +00:00
Nick HillandRobert Shaw 42dbe4578c [Core] Rework scheduler skipped_waiting queue (#58947) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-09-28 20:03:16 +00:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
Juan Pérez de AlgabaandCursor 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-26 11:10:35 +00:00
zhecClaudeBugen Zhao
 		ddd6fbca14 [Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: zhec <chengyunfei@ruc.edu.cn>
 2026-09-26 02:18:58 +00:00
Bugen Zhaojthomson04Claude Opus 5.5
 		055f150df5 [Perf][Rust Frontend] Make histogram observations lock-free (#58574) 
Co-authored-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-25 10:25:10 +08:00
Harry MellorandClaude Opus 5.5 d636456b1f [Frontend] Remove the slow tokenizer mode (#58545) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 08:48:28 -07:00
ReidBugen ZhaoClaude Opus 5.5
 		e430f7421f [Bugfix][Rust Frontend] Prevent MM timing from enabling debug tracing (#58378) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 09:22:23 +00:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e028554a1f [Rust Frontend] Support --sse-keep-alive-interval (#58306) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 06:19:14 +00:00
Biswa PandaandCodex cf78797cb7 [Rust Frontend] Pass vision preprocessing context for Nemotron-H (#57634) 
Pass the remaining engine context-length budget to model-owned vision processors through VisionPreprocessingContext. Preserve Nemotron batched engine fields and recognize llm_config as a text_config alias.

Use the merged upstream llm-multimodal revision f0985ef65967615db2c79279aa07818499301bfd.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 14:08:26 +08:00
Bugen ZhaoandClaude Opus 5.5 a1b6763df6 [Rust Frontend] Accept custom chat roles for HF templates (#58311) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 13:43:34 +08:00
Bugen ZhaoandClaude Opus 5.5 a5603f21ae [Rust Frontend] Recognize new frontend-owned serve args as unsupported or no-op (#58330) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 13:43:24 +08:00
Bugen ZhaoandCodex 81d7293c21 [Rust Frontend] Construct model-owned vision processors through specs (#58109) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-22 10:01:53 +00:00
Bugen Zhao a6c47fbbf4 [Rust Frontend] Separate multimodal instrumentation from request timing (#58084) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-22 06:57:56 +00:00
Bugen ZhaoandCodex 496c6472cb [Rust Frontend] Build full-output grammars from initialized reasoning parsers (#57340) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-22 06:11:55 +00:00
Bugen ZhaoandClaude Fable 5.1 d50723df04 [Rust Frontend] Introduce parser-owned output grammar interfaces (#55269) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-22 05:35:07 +00:00
karthikKarthik Gangulafactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>Isotr0py
 		f84325c48c [Frontend][Rust] Add mm-processor benchmark for Rust frontend (#51922) 
Signed-off-by: Karthik Gangula <gangula-karthik@users.noreply.github.com>
Signed-off-by: gangula-karthik <gkarthik923@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Karthik Gangula <gangula-karthik@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 05:08:35 +00:00
Bugen ZhaoandCodex ff3c9cba3f [Rust Frontend] Add MiMo V2.5 parser support (#57933) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-22 04:27:09 +00:00
Karen Chung 986e217158 [Feature] Add first-class KV hints request envelope for programmatic KV management (#53423) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-09-21 09:56:11 -07:00
Hongxin Xuaoshen02Codex
 		10e6a7f210 [Frontend] Expose per-request spec decode metrics in generate API (#43310) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
 2026-09-20 12:21:33 +00:00
hangangqiangmukeshbhandarkaraoshen02Claude Fable 5.1aoshen02
 		90531daf6d [Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-18 18:11:05 -07:00
Bugen ZhaoCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c58532c86b [Frontend] Upgrade XGrammar to 0.2.7 and Rust structural tags to 0.3.0 (#57272) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-18 11:44:05 +00:00