MOVZX and Hinata
5f3c7bd8ab
[Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
...
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
the live server process maps none of them
- Delete tests that only build architectures this fork no longer
registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
import at module level
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 19:31:07 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
Shuolei Wang and aoshen02
b3a7e2ff6e
[RL][Frontend] Return weight-version spans on OpenAI completions and chat ( #61347 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-17 02:08:07 +00:00
maithilijoshi20 and Cyrus Leung
92dc3dd393
[Bugfix] Honor OPT token embedding width for prompt embeddings ( #61736 )
...
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-16 04:41:50 +00:00
Chester
c599b5971f
[Bugfix][Frontend] Return all logprobs when completions requests logprobs=-1 ( #55717 )
...
Signed-off-by: Chester <42577861+superdiaodiao@users.noreply.github.com >
2026-10-12 17:05:55 +00:00
premsurawut and Claude Opus 5.5
c811943137
[Bugfix][Frontend] Reject beam search with echo and logprobs in /v1/completions ( #60876 )
...
Signed-off-by: surawut.j <surawut.jirasaktavee@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 18:06:26 +00:00
Vineeth Sai Varikuntla and Flora Feng
112a282915
[Bugfix][Frontend] Keep streamed completion text_offset in step with tokens ( #60668 )
...
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:43 -07:00
Jee Jee Li
f9c9e8ac24
[LoRA] Remove tensorizer ( #60024 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-05 07:01:56 -07:00
Stefan Wang and Robert Shaw
b0eb87fe49
[Bugfix][Frontend] Return streaming errors before the first token ( #40986 )
...
Signed-off-by: 1fanwang <1fannnw@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-04 16:42:49 +00:00
Robert Shaw and Robert Shaw
dfab504333
[Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template ( #58754 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2026-09-26 14:00:56 +00:00
d95a896ea6
[Feature][Frontend] Add DeepSeek-V4 FIM completion rendering ( #44229 )
...
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-23 13:02:00 +00:00
Cyrus Leung
c8824a1e42
[MM] Move supports_multimodal_inputs and cache out of registry ( #57913 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 07:43:07 -07:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
18553c5c9f
[Kernel] Use Murmur3 RNG for Gumbel sampling ( #51367 )
...
Signed-off-by: freyfwt <110148731+freyfwt@users.noreply.github.com >
Signed-off-by: freyfwt <freytian1996@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-09-15 23:52:26 -07:00
wang.yuqi and Cyrus Leung
eeb549a74d
[Frontend] Move engine/protocol.py out openai folder ( #54492 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-31 10:13:50 +00:00
7cfb97e337
[Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses ( #48915 )
...
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com >
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-20 15:42:49 -07:00
wang.yuqi
eac636a7fa
[Frontend] Move api_server.py out openai folder ( #52131 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-08-19 09:43:46 +00:00
Yufeng He
5c9ff5366b
[Bugfix] Accept logprobs=-1 in the Completion API ( #46175 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
2026-08-18 08:22:13 +00:00
7553aac77b
[Frontend] Add routed-experts prompt offset ( #51906 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-13 08:09:00 -07:00
Nick Hill
77b519912b
[CI][Bugfix] Fix test_shutdown_on_engine_failure startup deadlock ( #51050 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-04 20:29:35 -05:00
Zach Zhu
58f9659397
[Frontend][Core] Standardize request error handling with VLLMError hierarchy ( #49665 )
...
Signed-off-by: Zach Zhu <zzqshu@126.com >
2026-07-29 04:20:40 +00:00
hcl
530ee36a0d
fix(openai): reject non-numeric logprobs with 400 instead of 500 ( #49144 )
...
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
2026-07-20 10:04:50 +00:00
langzhao-netizen and Claude
8b8af2caf7
[Frontend] Expose logprob_token_ids on Python OpenAI endpoints ( #43463 )
...
Signed-off-by: Lang Zhao <lang.zhao@galileo.ai >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-13 14:40:21 -07:00
sungbin1015 and Claude
04a703e397
[Frontend] Support bad_words in the /v1/completions endpoint ( #46793 )
...
Signed-off-by: sungbin1015 <sbin@solbox.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-08 09:51:17 +00:00
Juan Pérez de Algaba
675f4295cd
fix(security): bound completion prompt list to prevent unbounded engine fan-out ( #47845 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-07-07 22:48:20 +00:00
86db6c3070
[Frontend] add per-request timing metrics field to response body of Chat/Completions APIs ( #46768 )
...
Signed-off-by: Nicholas Edelman <nedelman@nvidia.com >
Signed-off-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: GPT-5.5 <noreply@cursor.com >
2026-07-06 17:48:29 -07:00
Harry Mellor
f1445f6dbd
[CI] Bump huggingface-hub from v1.10.2 to v1.22.0 ( #47551 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-04 07:45:45 -07:00
wang.yuqi
3483240b7e
[Frontend] Consolidate scale out entrypoints ( #44512 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-06-29 03:18:53 -07:00
wang.yuqi
31ca9504b1
[Frontend] Split ServingRender into renderer and entrypoint. ( #44285 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-06-23 11:19:09 +00:00
42ae5e7ac6
[Bugfix] Fix --enable-prompt-tokens-details omitting zero cached tokens ( #44383 )
...
Signed-off-by: Sasindharan Sankar <sasindharansankar@email.com >
Co-authored-by: Sasindharan Sankar <sasindharansankar@email.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-06-11 20:37:42 -07:00
Andreas Karatzas
54d0c36fff
[CI] Stabilize OpenAI schema fuzzing for malformed structural tags ( #44131 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-06-01 19:56:15 -07:00
Andreas Karatzas
5940590855
[ROCm][CI] Stabilize 400 error return code for invalid schema inputs ( #43016 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-24 10:06:49 +00:00
ruizhang and Rui Zhang
60af5c16ee
[Frontend] Add truncation side to OpenAI endpoints ( #43260 )
...
Signed-off-by: Rui Zhang <rza21.bc@gmail.com >
Signed-off-by: Rui Zhang <rui.zhang@globalrelay.net >
Co-authored-by: Rui Zhang <rui.zhang@globalrelay.net >
2026-05-21 20:32:31 -07:00
Nick Hill and Bugen Zhao
f2ace1d57d
[Frontend][RFC] Rust front-end integration ( #40848 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-05-21 12:24:48 +08:00
Andreas Karatzas
fb1ac806c5
[ROCm][CI] Stabilize ROCm shutdown and distributed compile CI ( #41573 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-10 03:47:40 +00:00
Luis 🚀 and Andrew Sansom
14043dfecd
feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API ( #40720 )
...
Signed-off-by: Luis Robaina <luis@protopia.ai >
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com >
Signed-off-by: LuisRobaina <luis@protopia.ai >
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com >
2026-05-01 10:05:55 +08:00
Andreas Karatzas
95995bbef8
[ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset ( #38503 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-04-25 05:25:20 +00:00
Chenguang Zheng and OpenAI Codex
07351e0883
[Feature] Warm up readonly multimodal processor during renderer startup ( #40797 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-04-25 03:57:41 +00:00
wang.yuqi
66c079ae83
[Frontend][4/n] Improve pooling entrypoints | pooling. ( #39153 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-09 10:09:45 +00:00
Bvicii and Cyrus Leung
999dfc1622
[Bugfix] Offload blocking tokenizer ops to shared thread pool to unblock event loop ( #34789 )
...
Signed-off-by: Bvicii <yizhanhuang2002@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-03-26 22:17:00 -07:00
Cyrus Leung
2e225f7bd2
[Renderer] Consolidate factory methods ( #38218 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-26 12:19:22 +00:00
Flora Feng
b4c1aef21c
[Refactor] Relocate tests from tests/v1/entrypoints/ to tests/entrypoints/ ( #37500 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-03-20 02:50:34 -07:00
Flora Feng
9040151fe1
[V0 Deprecation] Deprecate --disable-frontend-multiprocessing ( #37612 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-03-20 11:31:43 +08:00
Isotr0py
a836524d20
[Chore] Replace all base64 usages with faster pybase64 package ( #37290 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-03-17 14:44:19 +00:00
Flora Feng
384dc7f77b
[Refactor] Relocate completion and chat completion tests ( #37125 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-03-17 11:31:23 +08:00