MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
Cy Cheng and Cy
d74f4180cd
[Bugfix][Frontend] Detokenize echoed prompts after truncation ( #61876 )
...
Signed-off-by: Cy <15118312+cycancode@users.noreply.github.com >
Co-authored-by: Cy <15118312+cycancode@users.noreply.github.com >
2026-10-17 05:20:38 +00:00
maithilijoshi20 and Cyrus Leung
92dc3dd393
[Bugfix] Honor OPT token embedding width for prompt embeddings ( #61736 )
...
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-16 04:41:50 +00:00
wang.yuqi
0f0bc4dbb1
[Test] Split the chat_utils tests. ( #61765 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-15 08:26:22 +00:00
Ganesh R and Cyrus Leung
47232f3c7f
[Bugfix][CI] Fix renderers tests broken by tool name removal from warn logs ( #61652 )
...
Signed-off-by: Ganesh R <ganesh.r@amd.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-14 18:14:58 +00:00
wang.yuqi
669f85e686
[Misc] Break chat_utils into more maintainable modules. ( #60972 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-13 15:08:07 +00:00
RAHUL KUMAR and Claude Sonnet 5.5
c5422f4c46
[Test][Renderer] Cover the reasoning -> reasoning_content mirror in chat message parsing ( #56129 )
...
Signed-off-by: pctablet505 <55033230+pctablet505@users.noreply.github.com >
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com >
2026-10-13 03:58:32 +00:00
wang.yuqi
c1692c85f1
[Misc] mv chat_utils.py to renderers. ( #60522 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-10 06:50:34 +00:00
1a3a7a8f01
[Bugfix] Treat max_tokens as output upper bound, not input-space reservation ( #57035 )
...
Signed-off-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com >
Signed-off-by: YuvrajSinghBhadoria2 <yuvrajsinghbhado2030@gmail.com >
Signed-off-by: Yuvraj Singh Bhadoria <yuvrajsinghbhado2030@gmail.com >
Co-authored-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-10-10 01:15:22 +08:00
Matthew Bonanni and Claude Opus 5.5
a21a3f2ec7
[CI] Add supports_multimodal_inputs to test_executor_replace's mock config ( #59476 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 19:04:47 +00:00
Juan Pérez de Algaba and Cursor
cff08b461e
[Security] Fix chat template resource-exhaustion DoS (GHSA-4hhp-h66f-… ( #50300 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-30 08:32:00 -07:00
Harry Mellor and Claude Opus 5.5
fc2c801ad9
Disable mypy arg-type and assignment checks in tests ( #59428 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 15:18:50 +00:00
Harry Mellor and Claude Opus 5.5
d636456b1f
[Frontend] Remove the slow tokenizer mode ( #58545 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 08:48:28 -07:00
Ting SUN and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a33b3bac5d
[Bugfix][Qwen2.5-VL] Honor video fps for temporal M-RoPE ( #47736 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-22 05:56:12 +00:00
Cyrus Leung
7853700d4c
[MM] Move get_dummy_processor_inputs into MM processor ( #57967 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 09:39:20 -07:00
Cyrus Leung
c8824a1e42
[MM] Move supports_multimodal_inputs and cache out of registry ( #57913 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 07:43:07 -07:00
Dilber
27b7757f63
[Bugfix][Frontend] Bound the prompt after multimodal expansion ( #57076 )
...
Signed-off-by: Dilber P Shakir <dilbersha@gmail.com >
2026-09-20 08:52:27 +00:00
Cyrus Leung
751f6807d9
[Refactor] Use dynamic MM cache in processor ( #57674 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-19 15:38:00 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
323a29d8f2
[Bugfix] Unreadable prompt_embeds payload should be a 400, not a 500 ( #55451 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-16 05:53:36 +00:00
Tianyu Guo
fa1b3b1922
[Bugfix][EPD] Preserve explicit multimodal UUIDs with caches disabled ( #56432 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-09-13 10:08:30 +00:00
56d001faf0
[Bugfix] Detect OpenAI content format when message.content is passed through macro parameters ( #53824 )
...
Signed-off-by: wuhangxian <1391938827@qq.com >
Signed-off-by: wuhangxian <wuhangxian@users.noreply.github.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: wuhangxian <wuhangxian@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 20:10:28 +00:00
frankie and Trae Agent
9cc7793e32
[Bugfix] Gracefully handle unsupported reasoning_effort in chat templates ( #54022 )
...
Signed-off-by: frankie <wangyongsheng686@gmail.com >
Co-authored-by: Trae Agent <noreply@trae.ai >
2026-09-06 21:11:35 -07:00
lucamotz and OpenAI Codex
f4eccdadef
[Bugfix][Multimodal] Bound renderer warmup to the prefill token budget ( #55448 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-05 16:43:28 +00:00
Adababy and shaolila
2902ca17e3
[Bugfix] complete VLLMValidationError migration in chat_utils.py ( #50254 )
...
Signed-off-by: shaolila <shaolila@buaa.edu.cn >
Co-authored-by: shaolila <shaolila@buaa.edu.cn >
2026-09-05 17:56:12 +08:00
Ning Xie and Cyrus Leung
2a336d8239
[warmup] overlap renderer warmup and engine core initialization ( #54557 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-03 19:44:21 +00:00
Jan Kwieciński
1f1f628859
[Feat][MM Hashing] include media_io_kwargs in multi-modal hashes ( #54241 )
...
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com >
2026-09-01 13:17:01 +00:00
lzhan011 and Cyrus Leung
ce7391712b
[Bugfix][Security] Bound embedding densification before to_dense() ( #54632 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-01 08:45:14 +00:00
waizuichougou
8905633687
[Bugfix][Frontend] Preserve token offset origins after left text pre-trimming ( #54692 )
...
Signed-off-by: waizuichougou <2082431897@qq.com >
2026-09-01 08:02:56 +00:00
Hotragn Pettugani
28bf75c9a9
[Bugfix][Frontend] Truncate prompt_is_token_ids with the prompt ( #54509 )
...
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in >
2026-08-31 09:29:01 +00:00
Hotragn Pettugani
8fd9eb85d5
[Bugfix][Frontend] Truncate prompt_token_offsets with the prompt ( #54407 )
...
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in >
2026-08-31 06:13:37 +00:00
Hotragn Pettugani
687db59744
[Bugfix][Frontend] Truncate pooling prompts before padding them ( #54364 )
...
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in >
2026-08-30 22:12:03 -07:00
Kevin H. Luu
d04d3e0a07
[Bugfix] Revert renderer warmup overlap to avoid fork deadlock ( #54023 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-08-27 16:36:33 -07:00
Ning Xie
ca90b9e7d4
[warmup] overlap renderer warmup and engine core initialization ( #52764 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
2026-08-27 11:42:08 +08:00
7ca336929c
[Model] Remove ten deprecated model architectures ( #53608 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Xianbao QIAN <xianbao.qian@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-24 17:06:00 -07:00
andrewbcohere
f94dcde5c5
Fix Cohere ChatV2 citation and tool handling issues ( #52175 )
...
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com >
2026-08-21 15:23:17 -07:00
Cyrus Leung
4666a8ba9e
[Refactor] Remove InputPreprocessor ( #53064 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-08-20 08:39:48 +00:00
wang.yuqi
b8165e5e58
[Frontend] Consolidate entrypoint exception handler ( #52261 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-08-14 09:27:24 +00:00
andrewbcohere and Cyrus Leung
9c110fa522
[Frontend] Cohere chat v2 api support ( #47189 )
...
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-07-31 22:53:54 -07:00
Recoordinate and latent-9
0b5b49dde8
[Bugfix][Frontend] Raise VLLMValidationError for user-facing errors in chat_utils.py ( #50491 )
...
Signed-off-by: latent-9 <296084221+latent-9@users.noreply.github.com >
Co-authored-by: latent-9 <296084221+latent-9@users.noreply.github.com >
2026-07-31 11:54:43 +00:00
Bugen Zhao
f5a7cce9b6
[Model] Add Kimi K3 support: Python frontend [2/2] ( #50093 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-29 01:06:01 -07:00
Zach Zhu
58f9659397
[Frontend][Core] Standardize request error handling with VLLMError hierarchy ( #49665 )
...
Signed-off-by: Zach Zhu <zzqshu@126.com >
2026-07-29 04:20:40 +00:00
6570c9800c
[Model] Add Inkling model support [1/N] ( #48799 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-15 23:40:07 -07:00
Juan Pérez de Algaba
793cf79c89
[Bugfix][Security] Fix concurrent sparse invariant race bypassing CVE remediation ( #48583 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-07-14 11:08:24 +00:00
Hoang Nguyen and Claude Opus 4.8
038ec293b1
[Bugfix] Return 400 instead of 500 when multimodal data is sent to a text-only model ( #48473 )
...
Signed-off-by: Hoang Nguyen Tien <hoang.nguyentien.2601@gmail.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-14 08:15:43 +00:00
412414d8e0
Remove PersimmonForCausalLM and FuyuForCausalLM model architectures ( #48096 )
...
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-07-09 04:59:08 -07:00
Harry Mellor
f1445f6dbd
[CI] Bump huggingface-hub from v1.10.2 to v1.22.0 ( #47551 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-04 07:45:45 -07:00
Juan Pérez de Algaba and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c231d1f290
fix(security): bound tokenizer work when explicit truncation_side is set ( #47007 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-30 23:08:51 +08:00