Prudhvi Vuda
de216b6e64
[Bugfix] Skip MM processor cache inserts larger than capacity ( #53016 )
...
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com >
2026-08-20 13:42:50 +00:00
liuzhenwei
b1d9337e92
[EPD] Allow KV consumers to omit MM embeddings ( #52697 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-08-19 02:16:08 +00:00
wang.yuqi and Cyrus Leung
2fa490470d
[Model] Fused mm preprocess normalisation on the Device ( #50411 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-06 03:11:14 -07:00
Tianyu Guo
b30291cf0c
[EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU ( #50390 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-05 15:59:54 -07:00
Isotr0py
1180b604f8
[Multimodal] Expose mm hash algothrim selection to cli args ( #49686 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-07-31 05:25:19 +00:00
nvbfalk
60915c972c
[Feature] Add VidCom2 video token pruning ( #47750 )
...
Signed-off-by: Benedikt Falk <bfalk@nvidia.com >
2026-07-28 03:20:17 +00:00
Isotr0py
1750e443f2
[Misc] Move PyNvVideoCodec stuff out of gpu worker ( #49322 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-07-22 10:11:56 +08:00
Brandon Pelfrey and Roger Wang
af16446bf3
Vram semaphore infra ( #44465 )
...
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-06-26 17:32:51 -07:00
Zhanda Zhu and Yubo Gao
5d5c776444
[Perf] FP8 FlashInfer Attn for ViT ( #38065 )
...
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com >
Co-authored-by: Yubo Gao <ybgao-nvidia@users.noreply.github.com >
2026-04-27 13:44:15 +08:00
Harry Mellor and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
21d2b53f88
Remove need for explicit \n in docstring lists for --help formatting ( #38350 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-27 08:38:00 -07:00
Brandon Pelfrey and Nick Hill
80b70884eb
Add tensor IPC transfer mechanism for multimodal data ( #32104 )
...
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com >
Signed-off-by: Brandon Pelfrey <brandonpelfrey@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-03-21 20:10:20 +00:00
Cyrus Leung
987506bca6
[Refactor] Simplify dummy data generation ( #35025 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-22 20:55:27 -08:00
Roger Wang
1dae7b7843
[Bugfix] Exclude language_model_only key from MM AOT compile hash but include in model one ( #34508 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
2026-02-13 13:59:00 +00:00
Cyrus Leung
25e48a3aae
[Doc] Update usage of --limit-mm-per-prompt ( #34148 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-09 21:12:13 -08:00
Roger Wang
64a9c2528b
[UX] Add --language-model-only for hybrid models ( #34120 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
2026-02-09 14:57:33 +00:00
c4df59ad43
Add embedding input functionality for disabled modalities [remake] ( #32493 )
...
Signed-off-by: Reagan Lee <“reaganjlee@gmail.com ”>
Signed-off-by: Reagan Lee <reaganjlee@gmail.com >
Signed-off-by: Reagan Lee <96998476+reaganjlee@users.noreply.github.com >
Co-authored-by: Reagan Lee <“reaganjlee@gmail.com ”>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-08 04:57:16 -08:00
Harry Mellor
61e632aea1
Turn @config into a dataclass_transform ( #31541 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-02-03 17:40:59 +00:00
Hiroken. and Xingran Wang
1209b784f2
[Bugfix]: resolve torch.compile cache conflict between mm_encoder_tp_modes ( #32842 )
...
Signed-off-by: Hongjian Zhang <zhanghongjian@xiaohongshu.com >
Signed-off-by: Xingran Wang <wangxingran123456@outlook.com >
Co-authored-by: Xingran Wang <wangxingran123456@outlook.com >
2026-01-24 14:45:14 +00:00
Cyrus Leung
4753f3bf69
[Model] Use context managers for encoder- and LM-only mode ( #32605 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-01-20 11:43:38 +08:00
Matthew Bonanni
2612ba9285
[1/N][Attention] Restructure attention: move files ( #31916 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-01-09 13:10:24 -08:00
Cyrus Leung
e83b7e379c
Revert "[Renderer] Separate out RendererConfig from ModelConfig ( #30145 )" ( #30199 )
2025-12-07 00:00:22 -08:00
Cyrus Leung
27f4c2fd46
[Renderer] Separate out RendererConfig from ModelConfig ( #30145 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-12-06 23:15:42 -08:00
Matthew Bonanni
fc1d8be3dc
[Attention] Update attention imports ( #29540 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2025-11-27 11:19:09 -05:00
George D. Torres and Russell Bryant
56531b79cc
[Misc] Add backup hash algorithm for FIPS constrained environments ( #28795 )
...
Signed-off-by: George D. Torres <gdavtor@gmail.com >
Signed-off-by: George D. Torres <41129492+geodavic@users.noreply.github.com >
Signed-off-by: Russell Bryant <rbryant@redhat.com >
Co-authored-by: Russell Bryant <rbryant@redhat.com >
2025-11-26 00:50:22 +00:00
Roger Wang
0ff70821c9
[Core] Deprecate xformers ( #29262 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
2025-11-24 04:18:55 +00:00
Nick Hill
8832fff972
[BugFix] Fix mm_encoder_attn_backend arg type checking ( #28599 )
...
Signed-off-by: Nick Hill <nhill@redhat.com >
2025-11-13 03:06:03 +00:00
Matthew Bonanni and Luka Govedič
b30dfa03c5
[Attention] Refactor CUDA attention backend selection logic ( #24794 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2025-11-11 07:40:44 -05:00
Russell Bryant and DarkLight1337
58fab50d82
[Frontend] Require flag for loading text and image embeds ( #27204 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-10-22 15:52:02 +00:00
Roger Wang
c3a2c6ac5f
[MM][Core] Decouple ViT backend from LM backend ( #27061 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
2025-10-21 00:30:10 -07:00
Anand Roy and Harry Mellor
10214b6935
[FEATURE]: Use pydantic validation in multimodal.py config ( #26629 )
...
Signed-off-by: Anand Roy <86306690+andycandy@users.noreply.github.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-10-13 07:56:59 -07:00
Harry Mellor
8fcaaf6a16
Update Optional[x] -> x | None and Union[x, y] to x | y ( #26633 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-10-12 09:51:31 -07:00
Harry Mellor
d6953beb91
Convert formatting to use ruff instead of yapf + isort ( #26247 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-10-05 07:06:22 -07:00
Wenlong Wang and Harry Mellor
79aa244678
[Multi Modal] Configurable MM Profiling ( #25631 )
...
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-10-03 03:59:10 -07:00
Eugene Khvedchenya and Roger Wang
392edee34a
EVS Support (Video tokens pruning) ( #22980 )
...
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com >
Signed-off-by: Eugene Khvedchenya <ekhvedchenya@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2025-09-26 11:54:54 +08:00
Harry Mellor and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
c4afdb69cc
Move MultiModalConfig from config/__init__.py to config/multimodal.py ( #24659 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-09-15 17:43:16 +00:00
Prudhvi Vuda