160 Commits
Author SHA1 Message Date
MOVZXandHinata 61fbe0b865 [Mini-VLLM] Remove unreferenced definitions found by static scan
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.

- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
  reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
  unused BailingLinearAttentionBackend, and the sparse flash-attn entry
  points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
  cpu_mla_extend, cpu_mla_get_scheduler_metadata,
  gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
  gptq_quantize_weights, get_and_maybe_dequant_weights,
  marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
  ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
  input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
  run_dp_sharded_vision_model, fast_topk, get_layer_index,
  is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
  selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
  split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
  is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
  create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
  vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
  neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.

Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 21:39:56 +07:00
MOVZXandHinata 5f3c7bd8ab [Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
  target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
  and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
  code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
  server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
  the live server process maps none of them
- Delete tests that only build architectures this fork no longer
  registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
  import at module level

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 19:31:07 +07:00
MOVZXandHinata 2066d3143f [Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 17:10:56 +07:00
Harry MellorandClaude Opus 5.5 a8260cbc5c [CI] Bump Transformers version to 5.19.0 (#60381)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:01:41 -07:00
Harry Mellor 9e0b6fcf77 [CI] Bump Transformers version to 5.18.0 (#59621)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-10-04 00:42:35 +00:00
djramic 8882b83b34 [Bugfix] Bump tokenizers to 0.23.2 for duplicate-pattern support (#59796)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
2026-10-02 18:33:50 +00:00
Isotr0pyandHarry Mellor 58b3298457 [Misc] Add Transformers version upper bound in requirements (#59614)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-10-02 08:27:11 +00:00
Martin Hickey fcfacc3499 [Bugfix][Frontend] Seed derender detokenization from the prompt (#59046)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-10-02 03:17:56 +00:00
Harry MellorandClaude Opus 5 e3cae8d2ac [Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor (#57387)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-02 02:25:18 +00:00
Juan Pérez de AlgabaandCursor a4c4bb418e [Security] Bump remaining Dependabot packages (excl. ignored) (#59315)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-30 05:16:31 -07:00
Juan Pérez de AlgabaandCursor eec2a86b1e [Security] Bump nltk, aiohttp, pillow, and datamodel-code-generator (#59249)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-29 16:00:36 -07:00
dfc8e0f3e2 [Frontend] Support strict MiMo-V2.6 tool calling (#58019)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ubospica <ubospica@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-29 12:47:47 +00:00
Anton Peganovmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>OpenAI Codex
73859fec58 [Frontend] Switch Python Harmony dependency to oss-harmony (#55128)
Signed-off-by: Anton Peganov <apeganov@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-28 00:43:56 +04:00
Bugen ZhaoCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c58532c86b [Frontend] Upgrade XGrammar to 0.2.7 and Rust structural tags to 0.3.0 (#57272)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-18 11:44:05 +00:00
coderfornowandLi, Jiang <jiang1.li@intel.com> 88afb77700 [CPU][s390x] Pin protobuf to 7.36.1 and drop C++ extension removal workaround (#54978)
Signed-off-by: coderfornow <Ritik.Dhiranan@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-09-17 16:35:17 +08:00
LucainandClaude Opus 5 b87339888d [httpx migration] Import httpx from huggingface_hub (#56460)
Signed-off-by: Lucain Pouget <lucain@huggingface.co>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 16:29:31 +01:00
cjackalmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Flora Feng
bf339618d1 [Misc] Bump openai to >=2.25.0 to support namespace tools types (#49104)
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
2026-09-08 14:11:12 -07:00
Chauncey 339e16cbb6 [Bugfix] Support MCP SDK 2.x tool input schemas (#53870)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-09-01 11:02:39 -04:00
Cyrus Leung 0b19ebcacd [MM] Simplify _apply_hf_processor_main (#53275)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-08-22 01:49:37 -07:00
Andreas KaratzasandOpenAI Codex 90984ddbed [CI] Upgrade huggingface-hub to 1.28.0 (#52797)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-08-18 13:35:44 -05:00
74c94b9f29 [CI] Upgrade huggingface-hub to 1.27.0 (#51422)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-08-10 08:59:15 +01:00
397094da17 Resolve revision to commit_hash once per model load, via huggingface_hub's resolve_revision (#49990)
Signed-off-by: Wauplin <lucainp@gmail.com>
Signed-off-by: Lucain <lucainp@gmail.com>
Signed-off-by: Lucain Pouget <lucain@huggingface.co>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-08-05 13:49:56 +00:00
Julien Denize 1a20d23dab [PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947) 2026-07-30 15:38:00 +02:00
Russell BryantandClaude 96d2ceda4b [Security] Replace diskcache to eliminate pickle deserialization (#44549)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-14 20:29:24 -07:00
Rehan KhanandLi, Jiang <jiang1.li@intel.com> 9e57de7197 [CPU] Create Proper Numa topology for s390x (#40714)
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-07-13 12:58:43 +08:00
Julien Denize 3d3b96488f Migrate Voxtral to mistral-common 1.11.5 audio API (#46705)
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
2026-06-26 11:06:31 -07:00
Juan Pérez de AlgabaandIsotr0py f6ba720963 (security) Upgrade Starlette to >= 1.0.1 to fix CVE-2026-48710 (#45675)
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-06-18 12:35:13 -07:00
0a1c5034f5 [Model] Add MiniMax M3 support (#45381)
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-06-16 01:01:25 +08:00
Isotr0py 6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-06-12 12:02:21 -07:00
Chaunceycjackalmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2043258dec [Frontend] Support strict mode for tool calling (#45003)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-12 07:51:48 +00:00
Harry Mellor 18d87a87dc Deprecate Transformers v4 support (#45161)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-06-11 11:04:01 +08:00
Harry Mellor c66b19800b [CI] Bump mistral-common (#44649)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-06-05 14:18:50 +00:00
Michael Goin 06ee2d8433 [Quant] Support compressed-tensors WNA8O8Int linears and WNInt embeddings (#44340)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-06-04 07:40:33 -07:00
Michael Goin cd0ff26e7a [CI] Add DSV4-Flash to gsm8k moe-refactor/config-b200.txt (#42111)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-05-19 20:21:01 -07:00
Chao-Ju Chenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 0d4d334eaa Bump llguidance to 1.7 (#42150)
Signed-off-by: RickyChen / 陳昭儒 <ricky.chen@infinirc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-14 20:35:27 -04:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 1d694e78c9 [Examples][last/6] Resettle examples. (#41084)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-07 19:42:12 -07:00
Julien DenizeandRoger Wang 16e336491e [Mistral Tokenizer] allow more leniency in apply_chat_template (#41658)
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
2026-05-05 19:56:15 -07:00
8c57b6e7bc Bump model-hosting-container-standards to >= 0.1.14 (#39755)
Signed-off-by: EC2 Default User <ec2-user@ip-172-31-20-13.us-west-2.compute.internal>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-13.us-west-2.compute.internal>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
2026-05-05 19:09:57 -04:00
844df54269 feat: update xgrammar==0.2.0 to use structural tags for strict tool calling + reasoning for more models (#40894)
Signed-off-by: Yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Ubospica <ubospica@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Ubospica <ubospica@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
2026-05-04 12:45:24 -07:00
Neil SchemenauerandSimon Mo 21792520e7 [Build] Add Python 3.14 to supported version list. (#34770)
Signed-off-by: Neil Schemenauer <nas@arctrix.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
2026-04-24 10:24:05 -07:00
Isotr0pyandRoger Wang 67eb6083e3 Revert "[Misc] Move pyav and soundfile to common requirements" (#40276)
Co-authored-by: Roger Wang <hey@rogerw.io>
2026-04-21 09:08:06 -07:00
Isotr0py 617d1c2ff1 [Misc] Move pyav and soundfile to common requirements (#39997)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-04-16 08:52:37 -07:00
R3hankhan 4b7ca37bd4 [CPU][IBM Z][Dockefile][Docs] Fix s390x builds for torch 2.11 and update docs for s390x (#39910)
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
2026-04-15 22:26:21 -07:00
03f8d3a548 Update to transformers v5 (#30566)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
2026-04-15 16:29:15 -07:00
Julien Denize fef56c1855 [Mistral Grammar] Support Grammar Factory (#38150)
Signed-off-by: juliendenize <julien.denize@mistral.ai>
2026-04-06 10:28:51 -04:00
Chauncey b322b197f1 [Build] Bump python openai version (#32316)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-03-18 18:20:10 +08:00
Brian DellabettaandDipika Sikka fa75204b16 bump compressed-tensors version to 0.14.0.1 (#36988)
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
2026-03-17 15:36:19 -04:00
Russell Bryant b3debb7e77 [Build] Upgrade xgrammar to get a security fix (#36168)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
2026-03-15 03:13:48 +00:00
Julien Denizerootgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
e42b49bd69 Mistral common v10 (#36971)
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Co-authored-by: root <root@h200-bar-196-227.slurm-bar-compute.tenant-slurm.svc.cluster.local>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-03-14 07:26:43 -07:00
Chang SuandNick Hill 507ddbe992 feat(grpc): extract gRPC servicer into smg-grpc-servicer package, add --grpc flag to vllm serve (#36169)
Signed-off-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
2026-03-10 03:29:59 -07:00