Commit Graph
544 Commits
Author SHA1 Message Date
8a269919f4 [MM] Add Triton kernel for mm_input_normal. (#56798)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-25 04:31:20 +00:00
5b3f280a6d [Bugfix] Count unsplit Idefics3 image patches (#48760)
Signed-off-by: nightcityblade <nightcityblade@gmail.com>
Co-authored-by: nightcityblade <nightcityblade@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-24 11:36:26 -07:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 14:14:10 +00:00
Linze Shi 484c211fe8 [Bugfix][Pooling] Fix JinaVL label configuration and restore multimodal tests (#57347)
Signed-off-by: Linze-Shi <linzeshi0@gmail.com>
2026-09-24 13:51:25 +00:00
Tianyu GuoandCyrus Leung 3ee2906f47 [EPD] Support metadata-only audio inputs (#57887)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-23 09:03:50 +08:00
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> a33b3bac5d [Bugfix][Qwen2.5-VL] Honor video fps for temporal M-RoPE (#47736)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-22 05:56:12 +00:00
djramicandAndreas Karatzas 4f14516790 [ROCm][CI] Use ROCm backend for DeepSeek V4.1 ViT test (#57931)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-21 16:39:31 +00:00
Cyrus Leung 7853700d4c [MM] Move get_dummy_processor_inputs into MM processor (#57967)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-09-21 09:39:20 -07:00
Sahil PatelandCyrus Leung 4e61ad8ee1 [Bugfix] unskip InternViT test for transformers v5 compatibility (#55767)
Signed-off-by: sahil <sahil@example.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-21 09:38:25 -07:00
Cyrus Leung c8824a1e42 [MM] Move supports_multimodal_inputs and cache out of registry (#57913)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-09-21 07:43:07 -07:00
Oliver Holworthyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 9598487bc1 [Bugfix][Model] Mistral3: align placeholder grid with public processor (#53758)
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-21 06:36:59 +00:00
Tianyao Wu 59f1527280 [Bugfix] Fix Whisper engine crash on audio clips longer than 30s (#57769)
Signed-off-by: Tianyao Wu <rayroy31@gmail.com>
2026-09-21 05:44:40 +00:00
Andreas KaratzasandOpenAI Codex 76ee20c1ec [ROCm][Tests] Avoid pooling cleanup waits against live shared engines (#57864)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-21 05:21:35 +00:00
Isotr0py 27757dde02 [DSV4.1] Add encoder cuda graph support for deepseek-v4.1-flash (#56625)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-20 21:51:13 +08:00
Cyrus Leung 751f6807d9 [Refactor] Use dynamic MM cache in processor (#57674)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-09-19 15:38:00 +00:00
Jiang WuandJiang Wu 01c7bf8813 [Bugfix][Model] Fix Aria expert weight names and layout (#57487)
Signed-off-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com>
Co-authored-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com>
2026-09-18 21:06:30 -07:00
Kevin H. LuuandKimi 9d75dd4544 [CI] Deflake pooling shards with GPU teardown fixtures between tests (#57371)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
2026-09-19 11:30:02 +08:00
Harry MellorandClaude Opus 5 8a5cf54387 [Multimodal] Type dummy options per modality (#57576)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 17:29:11 +01:00
Cyrus Leung dbf4b89cf2 [Misc] Remove unnecessary Transformers version guards (#57556)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-09-18 15:01:35 +00:00
SunandCodex 42a33039eb [Perf] Fuse CohereASR relative attention score accumulation (#55190)
Signed-off-by: levius <2114377220@qq.com>
Co-authored-by: Codex <codex@openai.com>
2026-09-18 06:18:41 +00:00
jiangkuaixue123andCodex de24e51908 [MM][V2] Enable encoder-only ViT CUDA graph capture (#56922)
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-17 16:12:17 +08:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
Kevin H. LuuSherlockmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
af1c01499b [Bugfix] Make GPU sync checks safe under torch.compile (#56904)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Sherlock <sherlock@raft.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-15 21:37:08 -07:00
Abhinav VermaandClaude Opus 5 1e47ec00d2 [Bugfix][Gemma 4] Don't read fft_length when profiling unified audio (#56721)
Signed-off-by: Abhinav Verma <verma.abhinav275@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 14:57:32 +00:00
Nicolò LucchesiandJorge Gallego Feliciano 995e8581f4 [Model] Voxtral Realtime: add support for CUDAGraphMode.FULL_DECODE_ONLY (#51167)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Jorge Gallego Feliciano <jorge.gallego@mistral.ai>
2026-09-14 19:16:39 +00:00
linitra24 9d4d9aa5bc [LoRA][Refactor] Unify multimodal LoRA token count hooks (#55071)
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
2026-09-14 13:19:15 +00:00
YleanandCodex a987777755 [Bugfix][Gemma4] Keep image kwargs out of video preprocessing (#56652)
Signed-off-by: Ylean <61880605+Woolgathererer@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-13 09:07:12 +00:00
Taneem Ibrahim 1b29c508bc [Bugfix] Initialize data parser in Nano-Nemotron audio test (#56401)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-09-12 09:07:20 -04:00
Taneem Ibrahim a0914ab7d0 [Nano-Nemotron] Fix Nano-Nemotron precomputed multimodal embeddings (#56398)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-09-12 09:07:10 -04:00
Andreas KaratzasandOpenAI Codex 9dd969da09 [Model][ROCm] Enable DeepSeek V4 Vision (#55107)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-11 09:17:30 -07:00
e77daef89e [Model] Support DeepSeek-V4.1-Flash (#56214)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 02:11:19 -07:00
zexplorerhjCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
fe26c705f7 [Model] Add Bailing V3 VL support (#55921)
Signed-off-by: zexplorerhj <zhjoneson@163.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 02:46:56 -07:00
Yunzhu LuandIsotr0py 442d36031c [MM][CG] Enable encoder CUDA Graph for MiniCPM-V (#42785)
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-10 06:28:48 +00:00
Li, Jiang c3ccc0e957 [Model] Add DeepSeek-V4 CPU backend (#55355)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-09-10 14:14:48 +08:00
Qiming Zhang 6e1d051fc9 [CI][XPU] skip test_models_text on XPU (#56038)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
2026-09-10 11:03:09 +08:00
cjackalmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Isotr0py
22d95d1adc [Bugfix] Make mm_device_do_normalize encoder-cudagraph safe (#55370)
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-10 01:43:22 +08:00
Taneem Ibrahim 5af4cc33ec [Bugfix] Fix OpenPangu multimodal embedding merge (#55941)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-09-08 23:53:59 +00:00
Andreas Karatzas cb22234687 [CI] Reuse ColQwen3 models across pooling tests (#55889)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
2026-09-08 14:03:54 -07:00
Wentao Ye 6b15bea080 [Refactor] Remove utils dead code (#53941)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-08 11:22:09 -04:00
Kevin H. Luu 34b9899c8f [CI] Increase ColQwen3 pooling test memory budget on H200 MIG (#55878)
Signed-off-by: khluu <khluu000@gmail.com>
2026-09-08 11:39:01 +00:00
Sahil Patel 51da0ca66c [CI/Build] Unskip ColQwen3 multimodal pooling tests on Transformers v5 (#55588)
Signed-off-by: Sahil <sip4818@gmail.com>
2026-09-07 15:15:39 +00:00
Qiming Zhang 29af8bd672 [XPU][UT] skip GLM-5.3-Flash test on XPU (#55266)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
2026-09-04 14:45:06 +08:00
SunandCodex 848ab131bc [Perf] Accumulate Conformer attention scores with baddbmm (#55062)
Signed-off-by: levius <2114377220@qq.com>
Co-authored-by: Codex <codex@openai.com>
2026-09-03 04:17:52 -07:00
stefankoncarevic 2691c6cc53 [Bugfix][CI] Set cudagraph_mode=FULL for the Ernie4.5-VL ViT cudagraph test (#54957)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
2026-09-02 13:18:56 -05:00
Isotr0py 1356635d83 [New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support (#54566)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-02 11:09:09 -07:00
Hung Hoang (黄兴)andCyrus Leung 3b45d053b4 [Bugfix][Model] Fix CohereASR streaming audio-token estimate (unit + subsampling) (#53829)
Signed-off-by: hungh <hungh@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-02 08:55:16 -07:00
Fanli LinandKunshang Ji 82936c409d [Tests][XPU] Limit Qwen2-VL generation length to avoid flaky numerical divergence (#54172)
Signed-off-by: Lin, Fanli <fanli.lin@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-08-31 22:59:57 +08:00
Andreas Karatzas 399247cc88 [Bugfix][MM] Fix MiniCPM-o image processor reuse on Transformers v5 (#54501)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
2026-08-31 17:07:52 +08:00
Prudhvi Vuda 7292ee2791 [Bugfix][Multimodal] Honor modality-scoped mm_processor_kwargs (#53808)
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
2026-08-31 06:37:12 +00:00