MOVZX and Hinata
2b544b7ba0
[Mini-VLLM] Cut dead subsystems across serving, connectors, parsers
...
- Remove P/D transfer connectors: nixl, nixl_chat, mooncake x2, hf3fs,
moriio, hisparse, example and hidden-states connectors
- Reduce weight_transfer to the base API used by model loading
- Remove kv_offload tiering, simple_kv_offload, ec cpu/example,
snapshot command, engine snapshot server
- Remove transformers model fallback, dsv4 tokenizers, dead platforms
cpu/tpu/xpu, flashmla third_party, dsa, megamoe/flydsl, dead attn
ops, dead layers (short_conv, fused_embed_norm, indexer_topk,
relu2 gemma_ar, lightning_attn, minimax_rms_norm), gemm_rs_ar
- Keep MoE LoRA op (imported eagerly by punica_gpu); drop fp8 variant
- Rebuild parser hubs around Qwen3 adapters only (36 dead adapters)
- Remove dead tests with the code they covered
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 02:19:03 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
MOVZX and Hinata
e61dcb9143
[Mini-vLLM] SM86-only cut: models, backends, Rust, tests
...
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-18 19:19:56 +07:00
alp and mottopanikeiku
2268d83bd9
[Bugfix][Multimodal] Honor RGB PNG transparency when loading images ( #61986 )
...
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-16 23:09:12 +00:00
a9fe30e9f6
[Bugfix][Multimodal] Recover PyNvVideoCodec decoder slots after failures ( #59597 )
...
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com >
Signed-off-by: danielafrimi <45691845+danielafrimi@users.noreply.github.com >
Signed-off-by: Daniel Afrimi <dafrimi@oci-hsg-cs-001-login-02.cm.cluster >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Daniel Afrimi <dafrimi@oci-hsg-cs-001-login-02.cm.cluster >
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com >
2026-10-15 09:17:19 +03:00
Juan Pérez de Algaba and OpenAI Codex
7173c64e2e
[Security] Bound decoded and downloaded media per request ( #61226 )
...
Signed-off-by: juanpas97 <juan.perezalgaba@gmail.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-13 10:17:14 +00:00
092c35dd15
[Model] Support Cloudflare/clef-omni on /v1/systemone ( #61064 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-12 21:27:34 +08:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
Vadim Gimpelson and Claude
ee406adf0c
[Bugfix] Fix ZeroDivisionError in Qwen3-VL video sampling when the source fps is unknown ( #60501 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 06:28:44 +00:00
Juan Pérez de Algaba
710ac56e69
[Security] Restrict Pillow image formats on untrusted media paths ( #60022 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
2026-10-05 10:40:15 +00:00
0872ddf4b9
[Bugfix][Multimodal] Normalize single-channel audio to 1D ( #56691 )
...
Signed-off-by: Ylean <61880605+Woolgathererer@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-04 06:11:41 +00:00
Harry Mellor and Claude Opus 5
e3cae8d2ac
[Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor ( #57387 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-02 02:25:18 +00:00
Harry Mellor and Claude Opus 5.5
fc2c801ad9
Disable mypy arg-type and assignment checks in tests ( #59428 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 15:18:50 +00:00
Cyrus Leung
677e3f5d50
[CI/Build] Fix pre-commit ( #59426 )
2026-09-30 13:45:28 +00:00
3d5625cc3c
[Security] Authenticate shared-memory multimodal cache handles ( #59357 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-30 05:16:26 -07:00
5225f1bacb
[MyPy][3/N] Fix MyPy errors in test groups (part 3) ( #55939 )
...
Signed-off-by: Taimys <ffy2281693949@gmail.com >
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Co-authored-by: Taimys <ffy2281693949@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 11:06:07 +00:00
Isotr0py
31876d8ea1
[Bugfix] Fix TorchCodec audio IO correctness ( #58364 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-29 11:34:48 -05:00
Daniel M. García-Ocaña Hernández
295a0b8d64
[Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution ( #56372 )
...
Signed-off-by: Daniel M. García-Ocaña Hernández <danielgarciaocana@gmail.com >
2026-09-29 03:58:39 +00:00
Tianyu Guo and Cyrus Leung
3ee2906f47
[EPD] Support metadata-only audio inputs ( #57887 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-23 09:03:50 +08:00
Cyrus Leung
c8824a1e42
[MM] Move supports_multimodal_inputs and cache out of registry ( #57913 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 07:43:07 -07:00
Juan Pérez de Algaba and Cursor
7268f6e38e
[Security] Prefer fresh multimodal payloads over a stale receiver cache ( #57833 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-21 12:50:48 +00:00
Cyrus Leung
751f6807d9
[Refactor] Use dynamic MM cache in processor ( #57674 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-19 15:38:00 +00:00
a8d1aa9c99
[Bugfix][Multimodal] Frame the multi-modal hash digest input ( #54283 )
...
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-18 21:06:48 -07:00
Kevin H. Luu and Kimi
f2b6152ec2
[CI] Add flaky rerun markers / timeout skips to sibling tests lacking them ( #57359 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-18 03:40:32 -07:00
092bdd6d57
[Frontend][Core] Add release_kv_cache_memory() API ( #44890 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: AlanFokCo <alanfok2868@gmail.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com >
Co-authored-by: AlanFokCo <alanfok2868@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-18 07:12:12 +08:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Ama Senevirathne and amasen02
9ca6dbba71
fix(multimodal): tolerate malformed EXIF metadata during hashing ( #56527 ) ( #56576 )
...
Signed-off-by: amasen02 <amasen02@users.noreply.github.com >
Co-authored-by: amasen02 <amasen02@users.noreply.github.com >
2026-09-16 07:56:50 +00:00
Isotr0py
22258a26bc
[Multimodal] Use GPU NVDEC for EPD encoder-only instance video media IO ( #53675 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-11 22:20:11 +08:00
295ac4e52e
[Bugfix][Multimodal] Parse decoded video frame lists as a single video ( #55326 )
...
Signed-off-by: waizuichougou <2082431897@qq.com >
Signed-off-by: waizuichougou <105572299+waizuichougou@users.noreply.github.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-11 12:16:05 +00:00
Jiatai Wang and OpenAI Codex
a36dfc93cb
[Bugfix][Multimodal] Restore cached audio inputs with UUIDs ( #56310 )
...
Signed-off-by: JiataiWang <wangjiatai@proton.me >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-10 19:20:03 +00:00
912d2e79fe
[multimodal][feat] add torchaudio backend to AudioResampler ( #52598 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 06:42:38 +00:00
Andreas Karatzas
f7f060d253
[Bugfix][Audio] Restore soundfile-first automatic decoding ( #55642 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-09-07 06:21:02 +00:00
lucamotz and OpenAI Codex
f4eccdadef
[Bugfix][Multimodal] Bound renderer warmup to the prefill token budget ( #55448 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-05 16:43:28 +00:00
Sun
c81ace1859
[Perf] Read VidCom2 frame budgets once ( #55331 )
...
Signed-off-by: levius <2114377220@qq.com >
2026-09-04 20:11:42 +00:00
6a039f465e
[feat] add torchcodec as audio loader and implement selective audio backend ( #51826 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: saltman155 <saltman155@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-04 16:16:10 +00:00
Juan Pérez de Algaba and Isotr0py
8b6de0eb9a
[Security] Cap GLMGA video sampling to prevent request-driven resource exhaustion ( #54935 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-04 12:02:41 +00:00
Jiangyun Zhu
98ed0856f3
[Model] add GLM-5.3-Flash support ( #53906 )
2026-09-03 09:40:35 -07:00
waizuichougou and Nick Hill
3f1af35e01
[Bugfix][Multimodal] Handle prefix-covered items in SHM worker cache ( #54994 )
...
Signed-off-by: waizuichougou <2082431897@qq.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-03 06:39:34 +00:00
waizuichougou
c6bca6e585
[Bugfix][Multimodal] Scope cache hash kwargs by modality ( #54918 )
...
Signed-off-by: waizuichougou <2082431897@qq.com >
2026-09-02 14:20:26 +00:00
Jan Kwieciński
1f1f628859
[Feat][MM Hashing] include media_io_kwargs in multi-modal hashes ( #54241 )
...
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com >
2026-09-01 13:17:01 +00:00
Jan Kwieciński
ec32f669bb
[Feature][MM_UUIDs] Allow empty video URLs when using multi-modal UUIDs ( #54220 )
...
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com >
2026-09-01 05:55:27 +00:00
Prudhvi Vuda
7292ee2791
[Bugfix][Multimodal] Honor modality-scoped mm_processor_kwargs ( #53808 )
...
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com >
2026-08-31 06:37:12 +00:00
Isotr0py
4fc943b867
[Multimodal] Deprecate PyAV video decoder backend ( #54231 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-08-29 15:13:28 +00:00
Moe Huzaifa
299ebd094a
[Bugfix][Multimodal] Reject malformed base64 audio with 400 instead of 500 ( #53744 )
...
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com >
2026-08-25 17:31:42 +00:00
Cyrus Leung
ecfa7bb373
[MM] Cache common token sequences ( #53560 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Signed-off-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-24 07:48:10 -07:00
Clinton Thomas and Lucas Bourtoule
2ec6f0d71e
Reject oversized media before fully downloading it ( #51896 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
2026-08-24 01:15:09 +00:00
Cyrus Leung and Qwen Code
ebcd606467
[MM] Simplify prompt updates: replace PromptSeq with list[int] ( #53372 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Co-authored-by: Qwen Code <noreply@alibaba-inc.com >
2026-08-23 14:50:32 +08:00
Yiqin and shenyiqin
cb09dd7488
[Core][Multimodal] Skip redundant placeholder scan when token match succeeds ( #52925 )
...
Co-authored-by: shenyiqin <shenyiqin1@huawei.com >
2026-08-20 15:04:29 +00:00
Prudhvi Vuda
de216b6e64
[Bugfix] Skip MM processor cache inserts larger than capacity ( #53016 )
...
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com >
2026-08-20 13:42:50 +00:00