36 Commits
Author SHA1 Message Date
MOVZXandHinata 5aa2ac64f8 [Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.

- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
  scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
  CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
  CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
  render routes, and vllm/config/speech_to_text.py. GenerationTask is
  generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
  Multi, Offloading and Example connectors stay; production uses the
  CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
  arch gate, declarations and op registrations. The CUDA build never
  defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
  fail with "The model ... does not exist. Valid aliases: ...".

Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 21:06:19 +07:00
MOVZXandHinata 5f3c7bd8ab [Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
  target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
  and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
  code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
  server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
  the live server process maps none of them
- Delete tests that only build architectures this fork no longer
  registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
  import at module level

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 19:31:07 +07:00
MOVZXandHinata 2066d3143f [Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 17:10:56 +07:00
MOVZXandHinata bd70a61c20 [Mini-VLLM] Remove orphan subsystems: snapshot, CPU workers, dead decoders
- Delete the snapshot package, CLI command, and its launch tests
- Delete CPU workers, cpu_communicator, and the hw_agnostic layer package
- Delete the EC CPU connector, keeping the EC base glue intact
- Delete dead composite/triton_flashinfer backends and third_party/flashmla
- Delete dead video decoders, video-bound tests, and orphan utils
- Remove the deep_gemm_mega_moe option and dead allreduce fusion pair

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 09:27:06 +07:00
MOVZXandHinata 54a396586e [Mini-VLLM] Repair test suite after SM86 cut
- Restore example connector, test helpers, and dropped patch tests
- Guard MLA metadata import in the example connector
- Point the PLE test at common.ops.ple after upstream #60021 move
- Cut tests for removed features: tools, DRY, epd proxy, STT batch,
  diffusion scheduler, INC/Quark PLE cases, mamba-2 mixer

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 01:06:30 +07:00
aa0c433a3e [Frontend] Accept --reasoning-parser none as the unset default (#61525)
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
2026-10-14 15:50:35 +08:00
Juan Pérez de AlgabaandCyrus Leung fcae5c9887 [Security] Restrict default CORS origins (#61009)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-10-11 08:26:02 +00:00
fa312090cb [Frontend] Accept each connection on the least loaded API worker (#60723)
Signed-off-by: hao-aaron <aaronhao@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 04:40:13 +00:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 02:30:52 -07:00
f226af0e11 [Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners (#54113)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 06:25:27 +00:00
bnkaiteamandFlora Feng 3722645539 [Bugfix][Frontend] Accept successful 2xx responses for batch output uploads (#60507)
Signed-off-by: bnkaiteam <bnkaiteam@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
2026-10-08 02:32:35 +00:00
09e0ce3d82 [Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server (#54835)
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-10-06 18:46:24 +08:00
sungbin1015andClaude 1e5d0ea888 [CI/Build] Fix incorrect CLI arg validation tests in test_cli_args.py (#46779)
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 05:58:47 +00:00
Nils Matteson 28556f8600 [Bugfix] Log CRIU failure details before snapshot cleanup (#59661)
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
2026-10-03 04:02:27 +00:00
Nils Matteson 9fdb174678 [Bugfix] Avoid InfiniBand state in TP1 snapshots (#59699)
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
2026-10-03 03:59:17 +00:00
9d2a6f52b2 [Rust Frontend] Expose gRPC port in Python vllm serve (#59659)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-10-02 08:14:37 +00:00
Mark McLoughlinandOpenAI Codex 91fcec1235 [Core][Logging] Add built-in JSON formatter (#58739)
Add `--logging-config.formatter=json` to enable the python-json-logger
formatter without having to use a complete Python logging dictConfig.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-10-01 09:11:46 +01:00
jackLei 21c08ac458 [CI] Deflake shutdown wait-timeout test by synchronizing on request admission (#58125)
Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com>
2026-10-01 00:10:03 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 15:18:50 +00:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 834ad456d3 [Refactor] Remove dead tests code (#58916)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-30 04:39:03 +00:00
5dd693122c [MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:27:45 +01:00
Louie Tsai 208bb38c80 [Bugfix][Rust Frontend] Fix startup with config-only model (#58643)
Signed-off-by: louie-tsai <louie.tsai@intel.com>
2026-09-29 10:11:53 +00:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205)
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
2026-09-25 11:36:59 +01:00
Nils MattesonTaneem IbrahimelehayymAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
13058faa49 [Frontend] Add reusable TP1 initialized-engine snapshots (#51360)
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-21 23:18:49 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
Chauncey f0fbc98c48 [Frontend] Share max_num_queued_reqs across API server processes (#54746)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-09-15 14:50:26 +00:00
wang.yuqi 67111973ee [Frontend] Move grpc_server to launchers. (#56746)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-09-14 05:03:08 +00:00
Francisco Javier ArceoandOpenAI Codex 6a2a2bb02b [Frontend] Add stateless /v1/responses/render endpoint (#50195)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-07 12:50:42 +00:00
Adababyandshaolila 2902ca17e3 [Bugfix] complete VLLMValidationError migration in chat_utils.py (#50254)
Signed-off-by: shaolila <shaolila@buaa.edu.cn>
Co-authored-by: shaolila <shaolila@buaa.edu.cn>
2026-09-05 17:56:12 +08:00
wang.yuqiandCyrus Leung eeb549a74d [Frontend] Move engine/protocol.py out openai folder (#54492)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
2026-08-31 10:13:50 +00:00
Moe Huzaifa 3e833933dc [Bugfix][Frontend] Keep credentials out of the Rust frontend launch log (#53738)
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com>
2026-08-25 16:32:12 +00:00
wang.yuqi 735daf8ef6 [Frontend] Move cli_args.py and dp_supervisor.py out openai folder (#53659)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-08-25 07:02:32 +00:00
wang.yuqi 9cef631f30 [Frontend] Move run_batch.py out openai folder (#53500)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-08-24 15:04:10 +00:00
Andreas KaratzasandOpenAI Codex cb58bb9c1e [CI] Harden RemoteVLLMServer GPU cleanup checks (#52282)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-08-19 14:17:17 -05:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-08-19 09:43:46 +00:00