MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
Ting SUN
eb8de8d1a1
[Bugfix][OpenAI] Respect allowed_token_ids in beam search ( #47630 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-17 08:50:40 +00:00
Willow Lopez and Artem Perevedentsev
ad8a6065f4
[Bugfix] Validate guidance tokenizer compatibility before EngineCore ( #52298 )
...
Signed-off-by: Oxygen56 <jiangth99@163.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 21:59:02 +03:00
Mai Hoàng Đức and Flora Feng
5cd8a2626b
[Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls ( #60036 )
...
Signed-off-by: Mazukiri <maihoangduc04@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 00:06:59 +00:00
Raphaël Rialland and Simon Veitner
c32495d58f
[watermark] compatibility validation ( #56801 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
2026-10-05 14:37:13 -04:00
e319f86f15
[Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860 ) ( #56984 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-03 14:17:09 +00:00
b721a4c709
[Feature] Add fixed-token prefill scoring ( #54335 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-28 07:50:09 -07:00
1f0bc49ee6
[Structured Outputs] Parse Lark grammars natively in the xgrammar backend ( #58321 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-25 04:35:10 +00:00
Tony Lin and Flora Feng
e4340e41c9
[Feat][XPU] VLLM_BATCH_INVARIANT support for Dense/MoE models ( #55881 )
...
Signed-off-by: Tony Lin <tony.lin@intel.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-22 11:06:54 -04:00
e548298f81
[Bugfix][Structured Outputs] Reject empty structural_tag at request validation ( #47450 )
...
Signed-off-by: linnea-lin-00638949 <15521435947@163.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-09-22 16:09:39 +03:00
5302d1fe40
[Model Runner V2] Support custom logits processors ( #56497 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-19 17:46:49 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Alexander Lee and OpenAI Codex
10e8d5614c
[Bugfix][Beam Search] Respect skip_special_tokens during decoding ( #56211 )
...
Signed-off-by: lxy-alexander <lxy_alexander@outlook.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-14 05:49:44 +00:00
ea40bb9e90
[feature] Watermarked generation and detection (Gumbel-max algorithm) ( #54053 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 21:00:15 -07:00
Kevin H. Luu and Codex
5db652225f
[Bugfix] Validate extension integers before engine serialization ( #55606 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-08 16:30:06 +08:00
Andreas Karatzas
a9dc631429
[Bugfix] Reject empty bad-word tokenizations ( #53433 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-08-31 12:07:17 -07:00
jack and QwertyJack
5b0e5b69ac
[Bugfix][Frontend] Validate stop_token_ids against vocab size ( #54196 )
...
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
2026-08-29 16:25:35 +00:00
Naveenraj Kamalakannan and Benjamin Chislett
d5cadcee86
[Feature][DSpark]: Logprobs adaptive verification ( #52242 )
...
Signed-off-by: Naveenraj Kamalakannan <therealnaveenkamal@gmail.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-08-25 10:22:33 -04:00
a1c5b1fd9f
[Core][V1] Support trace_decode_token_ids for deterministic decode replay ( #46701 )
...
Signed-off-by: ianzxliu <ianzxliu@tencent.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com >
Co-authored-by: ianzxliu <ianzxliu@tencent.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-20 15:12:43 +08:00
Jeffrey Wang and Claude Opus 5
41f12a0daf
[Bugfix] Raise VLLMValidationError from structured output validators ( #52394 )
...
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-16 01:04:51 -07:00
Junhao Shen and Claude Opus 4.8
38f097fab8
[Bugfix] Reject NUL byte in structured_outputs.regex ( #51796 )
...
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-08-14 00:08:36 +00:00
7553aac77b
[Frontend] Add routed-experts prompt offset ( #51906 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-13 08:09:00 -07:00
7f7a32cfec
[Spec Decode] DSpark confidence-scheduled verification ( #47808 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com >
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-12 12:27:48 -07:00
b64a2708b0
Bound generation inputs before expensive work ( #51447 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 22:03:50 +08:00
Zach Zhu
58f9659397
[Frontend][Core] Standardize request error handling with VLLMError hierarchy ( #49665 )
...
Signed-off-by: Zach Zhu <zzqshu@126.com >
2026-07-29 04:20:40 +00:00
Walter Beller-Morales and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8040ef2426
[Frontend] expose stream_interval as req sampling param ( #49754 )
...
Signed-off-by: walterbm <walter.beller.morales@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-27 11:30:37 +08:00
langzhao-netizen and Claude
8b8af2caf7
[Frontend] Expose logprob_token_ids on Python OpenAI endpoints ( #43463 )
...
Signed-off-by: Lang Zhao <lang.zhao@galileo.ai >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-13 14:40:21 -07:00
Guan-Ming Chiu
ed051fab54
[Bugfix] Reject sampling params unsupported by diffusion models ( #45418 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
2026-07-07 11:25:36 +00:00
Ting SUN
ac521f6237
[Bugfix][Structured Outputs] Reject degenerate structured_outputs that crash EngineCore ( #45346 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-06-30 22:41:33 +00:00
Muhammad Fawaz and Mahad Durrani
97b5ce5c39
[Bugfix] Raise VLLMValidationError for non-integer logit_bias keys ( #46612 )
...
Signed-off-by: muhammadfawaz1 <135441198+muhammadfawaz1@users.noreply.github.com >
Co-authored-by: Mahad Durrani <114791389+mahadrehmann@users.noreply.github.com >
2026-06-30 06:18:59 +00:00
Peter Pan
ebb0a71ad0
[Bugfix] Reject out-of-range temperature values in SamplingParams ( #44965 )
...
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io >
2026-06-14 23:12:44 -07:00
WEI CHENG CHIU and Claude
521b88c29e
[Bugfix] Reject structured outputs for diffusion decoders with a clear error ( #45468 )
...
Signed-off-by: Wayne Chiu <waynehacking8@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-06-13 12:04:01 -07:00
Guan-Ming (Wesley) Chiu
c7aa3d2630
[Core] Support structured outputs for beam search ( #35022 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <guanmingchiu@gmail.com >
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
2026-06-12 06:56:25 -07:00
Juan Pérez de Algaba
d598d23973
[Security] Reject non-finite temperature and repetition_penalty values ( #45116 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-06-11 01:12:14 -07:00
Andreas Karatzas
53b88d1dfc
[CI] Reject out-of-vocabulary before they reach the GPU logprob path ( #44042 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-06-02 22:27:52 -05:00
linzm1007 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
739af5c7e1
[Reasoning] [Bugfix] Reject invalid thinking_token_budget values ( #43402 )
...
Signed-off-by: linzm1007 <linzm1007@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-26 03:37:30 -07:00
c7560af424
[RFC] Replace shared-memory routed experts with ModelRunnerOutput transfer and HTTP support ( #39568 )
...
Signed-off-by: xhx1022 <1737006628@qq.com >
Signed-off-by: arlenxu <arlenxu@tencent.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: arlenxu <arlenxu@tencent.com >
Co-authored-by: Junjie Zhang <junj.jay.zhang@gmail.com >
2026-05-14 14:12:30 +00:00
Andreas Karatzas
d628a3c5cb
[ROCm][CI] Skip ROCm batch invalid-input test pending torch fix ( #41572 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-13 18:50:47 +08:00
d7af6b34d8
[Model Runner V2] Bug fix: logprob dtype int64/int32 issue ( #41761 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-11 21:55:43 +00:00
Wentao Ye and Nick Hill
51295793a2
[Model Runner V2] Add logprob_token_ids support ( #40559 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-05-01 10:02:03 -07:00
Julien Denize
c0722f22de
[Mistral Grammar] Fix tool and reasoning parsing ( #39217 )
...
Signed-off-by: juliendenize <julien.denize@mistral.ai >
2026-04-15 21:05:04 -07:00
Julien Denize
fef56c1855
[Mistral Grammar] Support Grammar Factory ( #38150 )
...
Signed-off-by: juliendenize <julien.denize@mistral.ai >
2026-04-06 10:28:51 -04:00
2e56975657
Generative Scoring ( #34539 )
...
Signed-off-by: Vedant Jhaveri <vjhaveri@linkedin.com >
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-03-31 16:02:11 -07:00
Juan Pérez de Algaba and Russell Bryant
b111f8a61f
fix(security): Add VLLM_MAX_N_SEQUENCES environment variable and enforce limit ( #37952 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Russell Bryant <rbryant@redhat.com >
Co-authored-by: Russell Bryant <rbryant@redhat.com >
2026-03-27 09:02:10 -04:00
4731884796
[Feature] limit thinking tokens (hard limit) ( #20859 )
...
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com >
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com >
Signed-off-by: Chauncey <chaunceyjiang@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-24 09:53:07 -07:00
Harry Mellor
f83b933b84
[CI] Bump mypy version to 1.19.1 ( #36104 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-10 09:18:28 -07:00
Zhuohan Li
04b67d8f62
Remove unused disable_fallback field ( #36546 )
2026-03-09 20:56:54 -07:00
aykoppol
25e02647c2
[Core] Add optional flags to check for repetitive token patterns in engine output ( #35451 )
...
Signed-off-by: aykoppol <aykoppol+git@gmail.com >
2026-03-03 12:23:25 +08:00
Wentao Ye and Cyrus Leung
e113a30113
[Deprecation] Deprecate code in 0.17 as scheduled ( #35441 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-02-28 17:32:37 +00:00