iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,944 Commits 3 Branches 0 Tags
4e4eab73ed2b4e00931a4636e926b8df1cb1fe50
Commit Graph
347 Commits
Author SHA1 Message Date
Yoni GozlanandBugen Zhao 657a2e9417 [Rust Frontend] Accept whole-number floats for integer tool arguments (#61299) 
Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-13 12:22:48 +08:00
xinnywinneBugen ZhaoClaude Opus 5.5
 		1bc6e072b0 [Rust Frontend] Add per-request timing metrics to chat and completions (#59314) 
Signed-off-by: wxin <wxin@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 15:39:10 -07:00
Martin Hickey 7039c7f17b [Bugfix][Rust Frontend] Omit unset sampling params in render output (#61220) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-12 19:31:03 +00:00
Liana KolevaandBugen Zhao aa1d44df16 [Bugfix][Rust Frontend] Alias qwen3_xml to qwen3_coder and add qwen3_json for the Qwen JSON format (#60136) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-12 20:20:53 +08:00
Shimi BandielHyeonki HongMimirClaudeBugen Zhao
 		8d7b93fec2 [Frontend] Report stop_reason on /inference/v1/generate and pass it through derender (#60251) 
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>
Signed-off-by: Shimi Bandiel <shimib@google.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Hyeonki Hong <hhk7734@gmail.com>
Co-authored-by: Mimir <mimir@nousresearch.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-12 19:37:53 +08:00
ReidBugen ZhaoClaude Opus 5.5
 		32b05addc1 [Bugfix][Rust Frontend] Restore DP wave delivery after XPUB reconnects (#60249) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:12:43 +08:00
ReidandBugen Zhao f5cda48c90 [Bugfix][Rust Frontend] Export corrupted request metric (#59937) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-12 19:12:39 +08:00
0xsenseiBugen ZhaoClaude Opus 5.5
 		3b9e462774 [Bugfix][Rust Frontend] Correct DP routing after request removal (#59631) 
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:12:35 +08:00
Zupeng WangOpenAI CodexBugen ZhaoClaude Opus 5.5
 		21445806bd [Rust Frontend] Support logprob_token_ids on OpenAI endpoints (#59549) 
Signed-off-by: Wang Zupeng <zupenwang@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:12:30 +08:00
Bugen ZhaoandClaude Opus 5.5 5a09062632 [Rust Frontend] Compose answer constraints into parser-owned output grammars (#60781) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 09:43:48 +00:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0466bd36ea [Rust Frontend] Build the Kimi K3 full-output grammar from its parser mode (#59190) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-12 09:43:47 +00:00
Shuolei Wangaoshen02Bugen ZhaoClaude Opus 5.5
 		4163a80142 [Rust Frontend] Return weight-version spans on final outputs (#61070) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 15:56:59 +08:00
aoshen02 bd5bbce6e4 [RL] Release weight-transfer communicators without stopping inference (#61060) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
 2026-10-11 21:15:14 +08:00
aoshen02andOpenAI Codex 1789f09588 [Frontend] Explain disabled scale-out endpoints with opt-in hints (#61042) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-11 08:58:07 +00:00
Juan Pérez de AlgabaandCyrus Leung fcae5c9887 [Security] Restrict default CORS origins (#61009) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 08:26:02 +00:00
Shuolei Wangandaoshen02 b89852aad0 [RL][Frontend] Return token-level weight-version spans on final outputs (#59582) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-11 11:33:29 +08:00
Wentao Ye 276fbcff27 [Deprecation] Remove scheduled deprecated items (#60923) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-10 15:45:33 -04:00
wang.yuqi c1692c85f1 [Misc] mv chat_utils.py to renderers. (#60522) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-10 06:50:34 +00:00
AlecCodexBugen ZhaoClaude Opus 5.5
 		8ea646b827 [Rust Frontend][gRPC] Remove GenerateStream producer task so disconnects abort requests (#60922) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-10 04:59:35 +00:00
Bugen ZhaoandClaude Opus 5.5 35e920d671 [Rust Frontend] Support --stream-interval and per-request stream_interval (#60589) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 14:10:42 -07:00
Bugen ZhaoandClaude Opus 5.5 5dc9bcd8cd [Rust Frontend] Pass reasoning controls through render -> generate (#60546) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:04:49 -07:00
Bugen ZhaoandClaude Opus 5.5 ce4603cad6 [Bugfix][Rust Frontend] Pass through raw characters in byte-level decode (#59861) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:04:00 -07:00
Bugen ZhaoandClaude Opus 5.5 9a585c62aa [Rust Frontend] Box rarely-set fields of per-token output types (#60588) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:01:16 -07:00
AlecBugen ZhaoClaude Opus 5.5
 		51ef239768 [Rust Frontend] Carry reasoning controls over gRPC (#59836) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:11:19 +08:00
devshah-cohereandBugen Zhao b49c921a27 [Bugfix][Rust Frontend] Always include token_ids in completion stream choices (#60286) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: devshah-cohere <dev.shah@cohere.com>
 2026-10-08 19:11:55 +08:00
Alec 29ae81fe83 [Rust Frontend] Preserve explicit zero sampling values in gRPC requests (#60115) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
 2026-10-08 07:27:47 +00:00
Bugen ZhaoOpenAI CodexClaude Opus 5.5
 		f226af0e11 [Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners (#54113) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 06:25:27 +00:00
Alec 14e3902d51 [Frontend][Rust] Add gRPC forbidden token sequences and cache usage (#59837) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
 2026-10-07 07:46:44 -07:00
Bugen ZhaoandClaude Opus 5.5 bc8dd4ad27 [Rust Frontend] Share SchemaRoot between argument coercion and grammars (#59408) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 08:28:27 +00:00
Bugen ZhaoandClaude Opus 5.5 7436a7f119 [Rust Frontend] Backtrack from safe text at a complete marker (#59563) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 07:45:15 +00:00
Cameron QuiliciCam QuiliciNick HillYifan Qiao
 		e37e51dd24 [Metrics] Expose cached prompt tokens by cache tier (#56318) 
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-10-06 14:52:05 -04:00
Yin LiYin LiClaude
 		7b665f7a58 [Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) (#58181) 
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-06 17:22:47 +00:00
Rodrigo Garcia 3faf214051 [vllm-bench, feature] Added support for Mooncake-style, timed-traces replay. (#55937) 2026-10-06 20:39:35 +08:00
Wentao Ye 3e182185aa [Deprecation] Deprecate sonet dataset as scheduled (#60129) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-06 08:15:12 -04:00
Bugen ZhaoandClaude Opus 5.5 6bbad6acdf [Rust Frontend] Pass tool defer_loading through to chat templates (#60203) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-06 18:00:15 +08:00
Bugen ZhaoandClaude Opus 5.5 b3082d3044 [Rust Frontend] Remove the PyO3 tool-parser bridge (#59744) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
aoshen02aoshen02Claude Opus 5.5
 		e319f86f15 [Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-03 14:17:09 +00:00
Misha GoinandClaude 1ab18599ca [Agents] Add pre-commit check for agent files and skills (#59657) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-02 09:38:22 -07:00
AlecBugen ZhaoClaude Opus 5.5
 		9d2a6f52b2 [Rust Frontend] Expose gRPC port in Python vllm serve (#59659) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 08:14:37 +00:00
ai-jzai-jzBugen Zhao
 		e5d33a36e7 [Bugfix][Rust Frontend] Preserve whitespace in GLM string arguments (#59654) 
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 10:10:35 +08:00
Idder Ghanbaja f3b77eff64 [Rust][Benchmark] Warn when temperature is left to the server default (#59247) 2026-10-01 21:03:03 +08:00
Bugen ZhaoandClaude Opus 5.5 24fb5fc08c [Rust Frontend] Add hf parser for response templates (#59005) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 08:16:38 +00:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
Bugen ZhaoandClaude Opus 5.5 b2d3a3b9ad [Rust Frontend] Split argument grammars into schema resolution and per-model rendering (#59395) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 05:43:18 +00:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ac9126e58a [Rust Frontend] Snapshot structural-tag grammars as readable outlines (#59393) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 05:09:43 +00:00
Bugen ZhaoandClaude Opus 5.5 36229e1337 [Rust Frontend] Token-aware marker parsing for Kimi K3 (#58358) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 04:21:50 +00:00
Bugen ZhaoandClaude Opus 5.5 ec5ef92c87 [Rust Frontend] Anchor tokens after pending UTF-8 bytes at their own first byte (#58357) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <bugen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 04:21:50 +00:00
ZhenchengLinClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Roy Wang
 		31ec1c3aa1 [Bugfix][Rust Frontend] Stop vllm-bench chat latency at the last token (#59251) 
Signed-off-by: zhenchenglin <zhl132@ucsd.edu>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Roy Wang <jasonailu87@gmail.com>
 2026-10-01 02:04:19 +00:00
Juan Pérez de AlgabaandCursor cff08b461e [Security] Fix chat template resource-exhaustion DoS (GHSA-4hhp-h66f-… (#50300) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-30 08:32:00 -07:00