iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,341 Commits 3 Branches 0 Tags
patches
Commit Graph
66 Commits
Author SHA1 Message Date
AlecClaude Opus 5.5Bugen Zhao
 		467b024f21 [Rust Frontend] Report vllm-proto schema version in ServerInfo (#61533) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-14 16:21:48 +08:00
AlecCodexBugen ZhaoClaude Opus 5.5
 		8ea646b827 [Rust Frontend][gRPC] Remove GenerateStream producer task so disconnects abort requests (#60922) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-10 04:59:35 +00:00
AlecBugen ZhaoClaude Opus 5.5
 		51ef239768 [Rust Frontend] Carry reasoning controls over gRPC (#59836) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:11:19 +08:00
Bugen ZhaoOpenAI CodexClaude Opus 5.5
 		f226af0e11 [Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners (#54113) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 06:25:27 +00:00
Bugen ZhaoandClaude Opus 5.5 b3082d3044 [Rust Frontend] Remove the PyO3 tool-parser bridge (#59744) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
Bugen ZhaoandClaude Opus 5.5 24fb5fc08c [Rust Frontend] Add hf parser for response templates (#59005) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 08:16:38 +00:00
Bugen ZhaoandClaude Opus 5.5 36229e1337 [Rust Frontend] Token-aware marker parsing for Kimi K3 (#58358) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 04:21:50 +00:00
Juan Pérez de AlgabaandCursor eef8b15d7c [Security] Bump pyjwt and rand for remaining GHSAs (#59427) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-30 14:26:16 +00:00
Juan Pérez de AlgabaandCursor a4c4bb418e [Security] Bump remaining Dependabot packages (excl. ignored) (#59315) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-30 05:16:31 -07:00
Bugen ZhaoandClaude Sonnet 5.5 8039d3cbfd [Rust Frontend] Use the MiMo structural-tag builder (#59148) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
 2026-09-30 01:34:46 +00:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
Bugen Zhaojthomson04Claude Opus 5.5
 		055f150df5 [Perf][Rust Frontend] Make histogram observations lock-free (#58574) 
Co-authored-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-25 10:25:10 +08:00
Biswa PandaandCodex cf78797cb7 [Rust Frontend] Pass vision preprocessing context for Nemotron-H (#57634) 
Pass the remaining engine context-length budget to model-owned vision processors through VisionPreprocessingContext. Preserve Nemotron batched engine fields and recognize llm_config as a text_config alias.

Use the merged upstream llm-multimodal revision f0985ef65967615db2c79279aa07818499301bfd.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-24 14:08:26 +08:00
Bugen ZhaoandCodex 81d7293c21 [Rust Frontend] Construct model-owned vision processors through specs (#58109) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-22 10:01:53 +00:00
Bugen ZhaoandClaude Fable 5.1 d50723df04 [Rust Frontend] Introduce parser-owned output grammar interfaces (#55269) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-22 05:35:07 +00:00
karthikKarthik Gangulafactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>Isotr0py
 		f84325c48c [Frontend][Rust] Add mm-processor benchmark for Rust frontend (#51922) 
Signed-off-by: Karthik Gangula <gangula-karthik@users.noreply.github.com>
Signed-off-by: gangula-karthik <gkarthik923@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Karthik Gangula <gangula-karthik@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 05:08:35 +00:00
Bugen ZhaoCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c58532c86b [Frontend] Upgrade XGrammar to 0.2.7 and Rust structural tags to 0.3.0 (#57272) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-18 11:44:05 +00:00
Biswa PandaandBugen Zhao 95d723700b [Rust Frontend] Return sampling masks over gRPC (#56777) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
 2026-09-18 08:26:17 +00:00
Alec d2a2a82954 [Rust Frontend] Bump vllm-proto to 0.3.0 (#57233) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
 2026-09-17 03:41:55 +00:00
Bugen Zhao bfd713bf8b [Rust Frontend] Support HF config overrides --hf-overrides (#56931) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-16 02:52:55 +00:00
Kevin H. LuuandSherlock d320a21431 [Bugfix][Rust Frontend] Restore prost test dependency (#56866) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Sherlock <sherlock@raft.ai>
 2026-09-14 14:37:44 -07:00
Biswa PandaBugen ZhaoCodex
 		31759ccb6d [Rust Frontend][Multimodal] Accept preprocessed multimodal gRPC features (#55047) 
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-12 02:29:49 +00:00
JulienDarveOpenAI CodexBugen Zhaomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		69db1c26b4 [CI/Build][Rust Frontend] Publish vllm-proto on crates.io (#56365) 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-11 16:28:52 +08:00
Bugen ZhaoandIsotr0py b47b01cf33 [Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 01:15:06 -07:00
Bugen Zhao b911fe85c4 [Rust Frontend] Attribute decoded text to tokens (#52910) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-01 22:25:31 +00:00
Isotr0py dc5cf437cf [Rust Frontend] Enable Qwen4-exp multimodal support (#54813) 2026-09-01 20:23:55 +00:00
Jorge Menjivar 225aec4809 [Rust Frontend] Migrate to new tekken crate (#53056) 
Signed-off-by: Jorge Menjivar <jorge@menjivar.ai>
 2026-09-01 02:35:21 +00:00
Connor CarpenterBugen Zhaomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1f9444a34f [Rust Frontend][gRPC] Add LoRA lifecycle control (#52840) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-25 04:13:19 +00:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Roy Wang df344bf36f [Rust][Benchmark] Load HF datasets from parquet shards via hf-hub, fixing truncated-cache sampling (#51570) 2026-08-21 05:07:43 +00:00
Bugen ZhaoOpenAI CodexNick Hill
 		83c5d59209 [Rust Frontend] Replace external protoc with pure Rust lib protox (#52892) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 18:43:51 -07:00
Bugen ZhaoNick HillAndreas Karatzas
 		c89d692cb5 [Build] Propagate vLLM version to Rust binaries (#52593) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-17 23:37:37 -07:00
Woosuk KwonBugen ZhaoOpenAI Codex
 		e0e5a7fb28 [Rust Frontend] Fix GLM-5.2 chat template rendering parity (#51426) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-18 03:16:54 +00:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> d8c70f2243 [Rust Frontend] Upgrade MiniJinja to 2.22 & remove method lookup workaround (#51235) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 00:23:24 +00:00
Connor CarpenterandBugen Zhao e98a8774cb [Rust Frontend][gRPC] Add multimodal image inference (#50368) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-04 09:34:31 +00:00
Bugen Zhao 2899dca843 [Model] Add Kimi K3 support: Rust frontend [1/2] (#50104) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-28 14:31:13 -07:00
Bugen Zhao d552a68645 [Rust Frontend] Extract shared tracing setup logic into vllm-tracing (#50129) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-28 19:31:15 +00:00
Bugen Zhao eb44b3aaa4 [Rust][Benchmark] Use async HTTP clients (#49295) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-21 16:53:57 +08:00
Bugen Zhao 8688a06d67 [Rust][Benchmark] Use tracing for logs (#48937) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-21 05:18:13 +00:00
Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 1940c8441e [Rust Frontend][gRPC] Add engine-aware health reporting (#48992) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
 2026-07-21 10:54:39 +08:00
Bugen Zhao e2d7adeb64 [Rust Frontend] Bump xgrammar-structural-tag and enable local extension (#49161) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-20 16:22:24 +01:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> c01618fdc8 [Rust][Benchmark] Integrate vllm-bench to vllm-rs & vllm CLI (#48930) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-20 09:31:25 +00:00
Roy WangandBugen Zhao 9354f22204 [Rust][Benchmark] Port in vllm-bench (#48107) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-17 14:25:29 +08:00
Woosuk KwonBugen ZhaoGiancarlo DelfinIsotr0pyIsotr0pyJee Jee LiRoger WangYifan QiaoClaude Fable 5OpenAI Codex
 		6570c9800c [Model] Add Inkling model support [1/N] (#48799) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-15 23:40:07 -07:00
Bugen ZhaoIsotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e281ac663a [Rust Frontend] Integrate MM audio support (#48554) 
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-15 15:00:17 +08:00
Bugen Zhao 074bdd0d99 [Rust Frontend] Integrate MM video support (#47959) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-10 08:15:33 +00:00
Isotr0py 400a9c386d [Rust Frontend] Bump llm-multimodal version (#47530) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-03 09:48:36 +00:00
Bugen Zhao 7a327f0b4f [Rust Frontend] Simplify unit tests with shared TestTokenizer (#47125) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-30 15:34:43 +01:00
Tahsin TunanBugen Zhaomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b8cb75b149 [Rust Frontend] Add static HTTPS and mTLS support for HTTP and gRPC (#45890) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-30 01:45:59 +00:00
Bugen Zhao 5274c1181d [Rust Frontend] Add Harmony Renderer for GPT-OSS (#46800) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-29 03:39:04 +00:00