iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
21,990 Commits 3 Branches 0 Tags
6ec2f84e1a42cdca59779ffae20aa62c8beda993
Commit Graph
270 Commits
Author SHA1 Message Date
Bugen Zhaojthomson04Claude Opus 5.5
 		055f150df5 [Perf][Rust Frontend] Make histogram observations lock-free (#58574) 
Co-authored-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-25 10:25:10 +08:00
Harry MellorandClaude Opus 5.5 d636456b1f [Frontend] Remove the slow tokenizer mode (#58545) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 08:48:28 -07:00
ReidBugen ZhaoClaude Opus 5.5
 		e430f7421f [Bugfix][Rust Frontend] Prevent MM timing from enabling debug tracing (#58378) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 09:22:23 +00:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e028554a1f [Rust Frontend] Support --sse-keep-alive-interval (#58306) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 06:19:14 +00:00
Biswa PandaandCodex cf78797cb7 [Rust Frontend] Pass vision preprocessing context for Nemotron-H (#57634) 
Pass the remaining engine context-length budget to model-owned vision processors through VisionPreprocessingContext. Preserve Nemotron batched engine fields and recognize llm_config as a text_config alias.

Use the merged upstream llm-multimodal revision f0985ef65967615db2c79279aa07818499301bfd.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 14:08:26 +08:00
Bugen ZhaoandClaude Opus 5.5 a1b6763df6 [Rust Frontend] Accept custom chat roles for HF templates (#58311) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 13:43:34 +08:00
Bugen ZhaoandClaude Opus 5.5 a5603f21ae [Rust Frontend] Recognize new frontend-owned serve args as unsupported or no-op (#58330) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 13:43:24 +08:00
Bugen ZhaoandCodex 81d7293c21 [Rust Frontend] Construct model-owned vision processors through specs (#58109) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-22 10:01:53 +00:00
Bugen Zhao a6c47fbbf4 [Rust Frontend] Separate multimodal instrumentation from request timing (#58084) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-22 06:57:56 +00:00
Bugen ZhaoandCodex 496c6472cb [Rust Frontend] Build full-output grammars from initialized reasoning parsers (#57340) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-22 06:11:55 +00:00
Bugen ZhaoandClaude Fable 5.1 d50723df04 [Rust Frontend] Introduce parser-owned output grammar interfaces (#55269) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-22 05:35:07 +00:00
karthikKarthik Gangulafactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>Isotr0py
 		f84325c48c [Frontend][Rust] Add mm-processor benchmark for Rust frontend (#51922) 
Signed-off-by: Karthik Gangula <gangula-karthik@users.noreply.github.com>
Signed-off-by: gangula-karthik <gkarthik923@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Karthik Gangula <gangula-karthik@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 05:08:35 +00:00
Bugen ZhaoandCodex ff3c9cba3f [Rust Frontend] Add MiMo V2.5 parser support (#57933) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-22 04:27:09 +00:00
Karen Chung 986e217158 [Feature] Add first-class KV hints request envelope for programmatic KV management (#53423) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-09-21 09:56:11 -07:00
Hongxin Xuaoshen02Codex
 		10e6a7f210 [Frontend] Expose per-request spec decode metrics in generate API (#43310) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
 2026-09-20 12:21:33 +00:00
hangangqiangmukeshbhandarkaraoshen02Claude Fable 5.1aoshen02
 		90531daf6d [Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-18 18:11:05 -07:00
Bugen ZhaoCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c58532c86b [Frontend] Upgrade XGrammar to 0.2.7 and Rust structural tags to 0.3.0 (#57272) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-18 11:44:05 +00:00
Biswa PandaandBugen Zhao 95d723700b [Rust Frontend] Return sampling masks over gRPC (#56777) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
 2026-09-18 08:26:17 +00:00
lazymioClaude Opus 5Bugen Zhao
 		39e33db7f3 [Frontend] Fix the parsing of missing string= in DeepSeek V4 (#56271) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
 2026-09-18 05:11:24 +00:00
lazymioClaude Opus 5Bugen Zhao
 		dee37d8911 [Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-18 04:32:39 +00:00
Dakai AnAlanFokCoaoshen02OpenAI Codex
 		092bdd6d57 [Frontend][Core] Add release_kv_cache_memory() API (#44890) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-18 07:12:12 +08:00
aoshen02 b978df3299 Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302) 2026-09-16 23:31:59 -07:00
aoshen02andClaude Fable 5.1 8538017f4b [Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-17 05:58:56 +00:00
AlecandCodex 28ec03bc6e [Bugfix][Rust Frontend] Accept appended EngineCoreOutput fields (#56533) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
 2026-09-17 01:54:43 +00:00
Bugen Zhao e52da038d1 [Rust Frontend] Normalize native renderer reasoning controls (#56998) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-16 18:15:05 -07:00
aoshen02 0983aef8da [Frontend] Return prompt metadata from /inference/v1/generate (#53187) 2026-09-16 15:29:07 -07:00
AlecandCodex 9639cbde04 [Rust Frontend] Expose local DP size in gRPC Control metadata (#57116) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-16 11:28:08 -07:00
JulienDarveBugen ZhaoOpenAI Codex
 		f37c550bf6 [Feature][Frontend] Expose effective attention block size for DCP (#56538) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-16 08:29:43 +00:00
IslamandBugen Zhao c16bb6068f [Bugfix][Rust Frontend] Tolerate NaN-corrupted logprobs in engine-core (#51026) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Islam <islam.almersawi@openinnovation.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-16 03:10:58 +00:00
Bugen Zhao bfd713bf8b [Rust Frontend] Support HF config overrides --hf-overrides (#56931) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-16 02:52:55 +00:00
ReidandBugen Zhao 0673de9800 [Rust Frontend][Metrics] Add per-request preemption histogram (#57033) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-09-16 02:51:17 +00:00
Bugen ZhaoandCodex 78eaaad2c7 [Rust Frontend] Add iteration token histogram (#56990) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-15 12:12:15 +00:00
Yeonwoo SungandBugen Zhao 142020c6c2 [Bugfix][Rust Frontend] Honor add_generation_prompt in DeepSeek V3.2/V4/V4.1 renderers (#56593) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: YeonwooSung <neos960518@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-15 10:03:26 +00:00
Kevin H. LuuandOpenAI Codex 772e4f05fa [Bugfix][Rust] Fix HF multi-turn dataset integration (#56915) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-14 18:04:46 -07:00
JinYan SuOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f0a61bd438 [Rust][Benchmark] Support HF ShareGPT datasets in multi-turn mode (#51104) 
Signed-off-by: xiaguan <751080330@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-14 16:48:23 -07:00
Kevin H. LuuandSherlock d320a21431 [Bugfix][Rust Frontend] Restore prost test dependency (#56866) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Sherlock <sherlock@raft.ai>
 2026-09-14 14:37:44 -07:00
Nicolò LucchesiandCodex 1713a09491 [Rust Frontend] Forward per-request watermarking controls (#56338) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-14 23:17:02 +08:00
ReidBugen ZhaoCodex
 		7b1ea3f524 [Bugfix][Rust Frontend][Multimodal] Align DeepSeek V4.1 and Kimi K3 media with rendered placeholders (#56366) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-14 10:37:59 +00:00
Martin Hickey 934b1fcbcb [Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-14 10:02:14 +00:00
AlecandOpenAI Codex e6b4e47d2d [Bugfix][Rust Frontend] Preserve selected-token logprob mode (#56406) 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
 2026-09-14 07:05:33 +00:00
Zupeng WangBugen ZhaoCodex
 		7632f767f0 [Rust Frontend] Support HTTP RL weight synchronization (#56567) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Wang Zupeng <zupenwang@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-14 03:37:04 +00:00
Clinton ThomasandLucas Bourtoule 22f6e4eccb [Frontend][Rust] Reject empty structured-output values (#54821) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
 2026-09-12 19:12:25 +04:00
Bugen Zhao eed1f3d0c6 [Rust Frontend] Support generation blocks in HF chat template (#56378) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-12 04:56:55 +00:00
Bugen ZhaoandCodex 485421b1c3 [Rust Frontend] Honor HF revisions, offline mode, and cache directory (#56386) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-12 03:39:17 +00:00
Biswa PandaBugen ZhaoCodex
 		31759ccb6d [Rust Frontend][Multimodal] Accept preprocessed multimodal gRPC features (#55047) 
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-12 02:29:49 +00:00
AlecandCodex 988d9b6777 [Rust Frontend][gRPC] Surface engine generation errors (#56405) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-11 15:31:55 -07:00
JinYan SuClaude Fable 5.1Bugen ZhaoCodex
 		79c137c6ad [Bugfix][Rust Frontend][Renderer] Align DeepSeek tool-call arguments with deepseek-recipe (#56260) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: JinYan Su <751080330@qq.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-11 10:19:55 +00:00
JulienDarveOpenAI CodexBugen Zhaomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		69db1c26b4 [CI/Build][Rust Frontend] Publish vllm-proto on crates.io (#56365) 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-11 16:28:52 +08:00
Bugen ZhaoandIsotr0py b47b01cf33 [Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 01:15:06 -07:00
Juan Pérez de AlgabaBugen ZhaoOpenAI Codex
 		3735c2d5f5 [Security][Rust Frontend] Normalize HTTP method labels in metrics (#56058) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-10 10:54:40 +08:00