iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,105 Commits 3 Branches 0 Tags
869278cbb02fc04a5f10d2eaba2bb251939e1eab
Commit Graph
112 Commits
Author SHA1 Message Date
zhecClaudeBugen Zhao
 		ddd6fbca14 [Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: zhec <chengyunfei@ruc.edu.cn>
 2026-09-26 02:18:58 +00:00
ReidBugen ZhaoClaude Opus 5.5
 		e430f7421f [Bugfix][Rust Frontend] Prevent MM timing from enabling debug tracing (#58378) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 09:22:23 +00:00
Biswa PandaandCodex cf78797cb7 [Rust Frontend] Pass vision preprocessing context for Nemotron-H (#57634) 
Pass the remaining engine context-length budget to model-owned vision processors through VisionPreprocessingContext. Preserve Nemotron batched engine fields and recognize llm_config as a text_config alias.

Use the merged upstream llm-multimodal revision f0985ef65967615db2c79279aa07818499301bfd.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 14:08:26 +08:00
Bugen ZhaoandClaude Opus 5.5 a1b6763df6 [Rust Frontend] Accept custom chat roles for HF templates (#58311) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 13:43:34 +08:00
Bugen ZhaoandCodex 81d7293c21 [Rust Frontend] Construct model-owned vision processors through specs (#58109) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-22 10:01:53 +00:00
Bugen Zhao a6c47fbbf4 [Rust Frontend] Separate multimodal instrumentation from request timing (#58084) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-22 06:57:56 +00:00
Bugen ZhaoandClaude Fable 5.1 d50723df04 [Rust Frontend] Introduce parser-owned output grammar interfaces (#55269) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-22 05:35:07 +00:00
karthikKarthik Gangulafactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>Isotr0py
 		f84325c48c [Frontend][Rust] Add mm-processor benchmark for Rust frontend (#51922) 
Signed-off-by: Karthik Gangula <gangula-karthik@users.noreply.github.com>
Signed-off-by: gangula-karthik <gkarthik923@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Karthik Gangula <gangula-karthik@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 05:08:35 +00:00
Bugen ZhaoandCodex ff3c9cba3f [Rust Frontend] Add MiMo V2.5 parser support (#57933) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-22 04:27:09 +00:00
Karen Chung 986e217158 [Feature] Add first-class KV hints request envelope for programmatic KV management (#53423) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-09-21 09:56:11 -07:00
Biswa PandaandBugen Zhao 95d723700b [Rust Frontend] Return sampling masks over gRPC (#56777) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
 2026-09-18 08:26:17 +00:00
lazymioClaude Opus 5Bugen Zhao
 		dee37d8911 [Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-18 04:32:39 +00:00
Bugen Zhao e52da038d1 [Rust Frontend] Normalize native renderer reasoning controls (#56998) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-16 18:15:05 -07:00
Bugen Zhao bfd713bf8b [Rust Frontend] Support HF config overrides --hf-overrides (#56931) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-16 02:52:55 +00:00
Yeonwoo SungandBugen Zhao 142020c6c2 [Bugfix][Rust Frontend] Honor add_generation_prompt in DeepSeek V3.2/V4/V4.1 renderers (#56593) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: YeonwooSung <neos960518@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-15 10:03:26 +00:00
ReidBugen ZhaoCodex
 		7b1ea3f524 [Bugfix][Rust Frontend][Multimodal] Align DeepSeek V4.1 and Kimi K3 media with rendered placeholders (#56366) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-14 10:37:59 +00:00
Bugen Zhao eed1f3d0c6 [Rust Frontend] Support generation blocks in HF chat template (#56378) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-12 04:56:55 +00:00
Bugen ZhaoandCodex 485421b1c3 [Rust Frontend] Honor HF revisions, offline mode, and cache directory (#56386) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-12 03:39:17 +00:00
Biswa PandaBugen ZhaoCodex
 		31759ccb6d [Rust Frontend][Multimodal] Accept preprocessed multimodal gRPC features (#55047) 
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-12 02:29:49 +00:00
JinYan SuClaude Fable 5.1Bugen ZhaoCodex
 		79c137c6ad [Bugfix][Rust Frontend][Renderer] Align DeepSeek tool-call arguments with deepseek-recipe (#56260) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: JinYan Su <751080330@qq.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-11 10:19:55 +00:00
Bugen ZhaoandIsotr0py b47b01cf33 [Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 01:15:06 -07:00
Bugen ZhaoCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		41cffa9af0 [Rust Frontend] Strongly type wire dtypes and multimodal modalities (#56174) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-10 02:11:22 +00:00
Bugen Zhao 285cbce6bd [Rust Frontend] Resolve unified and split parser selections consistently (#56018) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-09 18:19:30 -07:00
Bugen ZhaoandCodex fc6b6e1feb [Rust Frontend] Correctly parse whitespace framing in model output (#55417) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-09 13:40:13 +10:00
Bugen Zhao d6bce42983 [Rust Frontend] Report reasoning tokens in chat completion usage (#54883) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-03 16:47:24 -07:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 21a2211040 [Rust Frontend] Use token-attributed text in reasoning and unified parsers (#54884) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-04 02:05:05 +08:00
Reid 60857baa53 [Bugfix][Rust Frontend][Renderer] Align DeepSeek V4 historical developer message handling (#54854) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-09-02 20:43:57 +00:00
Bugen Zhao b911fe85c4 [Rust Frontend] Attribute decoded text to tokens (#52910) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-01 22:25:31 +00:00
ReidBugen ZhaoOpenAI Codex
 		d4329ba53d [Bugfix][Rust Frontend] Fix adjacent DeepSeek V4 user content rendering (#53281) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-01 00:23:24 +00:00
Connor Carpenter 7a67941c1d [Rust Frontend][gRPC] Add audio and video media inputs (#53760) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
 2026-08-30 11:31:37 +08:00
Cheng JiangchengvjiangJee Jee LiBugen ZhaorussellfengJee Jee Li
 		b2f685834a [Hy4] support Hy4-preview model (#54160) 
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: russellfeng <russellfeng@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-29 12:07:28 +08:00
RoachZhaoClaude Opus 5Bugen ZhaoOpenAI Codex
 		4c6c9d569f [Rust Frontend] Take the raw buffer in mm tensor lowering when possible (#53528) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: roachsinai <roachsinai@qq.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-29 01:05:14 +00:00
Pranav ThakurandBugen Zhao 085e9bb07e [Rust Frontend] Add support for truncate_prompt_tokens and truncation_side (#48584) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: pranavthakur0-0 <pranavthakurdew@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-29 00:16:37 +00:00
Jiahao LiangandChauncey 59d7fc92ea [Bugfix][DeepSeek V4] Handle trailing system messages in prompt rendering (#51262) 
Signed-off-by: jiahaoliang <gzliangjiahao@gmail.com>
Signed-off-by: Jiahao Liang <gzliangjiahao@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-08-25 07:49:44 +00:00
ReidandBugen Zhao 2adf4b9e2a [Rust Frontend] Fix Kimi K3 reasoning_effort="none" handling (#53043) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-08-21 02:47:07 +00:00
Bugen Zhao df3b3422b4 [Rust Frontend] Add HY3 unified parser and local XGrammar structural-tag builder (#53054) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-20 19:13:54 -07:00
Bugen Zhao f32b17b6d6 [Rust Frontend] Support --generation-config vllm (#53044) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-20 17:31:25 -07:00
Sagemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Bugen Zhao
 		727274a75b [Rust Frontend] Fix Qwen parser auto-detection (#51169) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-08-20 09:39:05 +00:00
Woosuk KwonBugen ZhaoOpenAI Codex
 		e0e5a7fb28 [Rust Frontend] Fix GLM-5.2 chat template rendering parity (#51426) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-18 03:16:54 +00:00
WillZZZyCyrus LeungClaude
 		3962042304 [Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift (#46747) (#46747) 
Signed-off-by: William Zhao <williamzh@meta.com>
Signed-off-by: WillZZZy <93454470+WillZZZy@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-12 12:06:57 +08:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f1e921b6d6 [Rust Frontend] Support dynamic tools from developer messages (#51144) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-10 18:30:38 -07:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> d8c70f2243 [Rust Frontend] Upgrade MiniJinja to 2.22 & remove method lookup workaround (#51235) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 00:23:24 +00:00
Sage 865781e627 [Rust Frontend] Add standalone Rust renderer (#50289) 2026-08-06 11:23:26 +00:00
SageandBugen Zhao 5bf32605a9 [Rust Frontend] Deduplicate request preprocessing for /tokenize (#50448) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-05 08:27:35 +00:00
Bugen Zhao 7743486190 [Frontend] DeepSeek V4 0731 reasoning effort prompts & mappings (#50580) 2026-08-04 08:51:32 -07:00
Connor CarpenterandBugen Zhao e98a8774cb [Rust Frontend][gRPC] Add multimodal image inference (#50368) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-04 09:34:31 +00:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 0b1c151cbc [Rust Frontend] Align tool rendering for Kimi K3 (#50540) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-04 08:03:46 +00:00
Karen Chung f57123aa2d feat(frontend): session id plumbing into requests (#48048) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-08-03 22:18:58 +00:00
Bugen Zhao ab98034d4c [Frontend][Bugfix] Use default tool call IDs for Kimi K3 for conversation-level uniqueness (#50420) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-31 10:02:17 +08:00
ReidandBugen Zhao 242c591d5a [Rust Frontend] Send multimodal tensors in auxiliary frames (#49341) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-29 12:21:09 +00:00