 bnellnm
|
91be5f9be3
|
[MoE Refactor] Rename "naive" all2all backend (#36294)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-03-19 15:50:34 -04:00 |
|
|
bnellnm
|
9279c59a0e
|
[MoE Refactor] DefaultMoERunner simplifcation (#33049)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-03-19 15:07:44 -04:00 |
|
|
EdalatiAliandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
daa05bf340
|
[Bugfix] Fix AttributeError when serving MXFP8 models with DeepGEMM installed (#37358)
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-03-19 17:58:33 +00:00 |
|
|
Lucas Kabela
|
7769b58307
|
[torch.compile][BE][Multimodal] Remove requirement to set_model_tag to avoid cache conflict (#37345)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
|
2026-03-19 17:26:12 +00:00 |
|
|
Cyrus Leung
|
657855ab41
|
[Misc] Cleanup more configs and processors (#37560)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-19 15:45:23 +00:00 |
|
|
Wei Zhao
|
e27b8ba3d1
|
[Bug] Fix fp8 trtllm MoE modular kernel supported routing methods (#37346)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-03-19 11:43:06 -04:00 |
|
|
Cyrus Leung
|
9515c20868
|
[Misc] Clean up processing logic (#37541)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-19 13:30:20 +00:00 |
|
|
Cyrus Leung
|
7a6ebcbfcf
|
[Model] Remove unnecessary get_language_model (#37545)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-19 20:00:36 +08:00 |
|
|
Cyrus Leung
|
765e461065
|
[Bugfix] Fix Nemotron Parse loading (#37407)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-19 09:55:29 +00:00 |
|
|
Duyi-Wang
|
6a9cceb219
|
[Bugfix][ROCm] Fix MoRI + AITER FP8 dispatch compatibility for defer_input_quant (#37418)
Signed-off-by: Duyi-Wang <duyi.wang@amd.com>
|
2026-03-19 09:49:27 +00:00 |
|
|
Collin McCarthy
|
0b6d52629f
|
Support temporal compression for Nemotron-3-VL videos (#36808)
Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
|
2026-03-19 08:02:19 +00:00 |
|
|
Aaron Hao
|
5f82706a21
|
[BUG] Exclude SKIP_TENSORS from get_layer_size() + new weight sync example for dpep (#37334)
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
|
2026-03-19 00:45:10 +00:00 |
|
|
Michael Goin
|
9482b0b085
|
[Bugfix] Remove assertion for NVFP4 scale dynamic range (#37465)
Signed-off-by: Michael Goin <mgoin64@gmail.com>
|
2026-03-18 15:37:49 -07:00 |
|
|
Woosuk Kwon
|
5bc1da147f
|
[LoRA][BugFix] Fix skipped LoRA adapters for Mistral3 (#36928)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-18 22:34:19 +00:00 |
|
|
Wentao Ye
|
0d81a1fe61
|
[V0 Deprecation] Deprecate virtual engine (#37195)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-03-18 14:30:14 -07:00 |
|
|
Netanel Haber
|
6ae4c8d6fc
|
chunk parakeet into 30s clips to prevent OOMs on long audios (#36671)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
|
2026-03-18 14:22:24 -07:00 |
|
|
JartX
|
a913b612d8
|
[Bugfix] Fix ROCm crash in qwen3_next multi-stream events (#36795) (#37427)
Signed-off-by: JartX <sagformas@epdcenter.es>
|
2026-03-18 16:06:31 -04:00 |
|
|
Chengyu Fang
|
738d0a281f
|
[Bugfix] Fix incorrect use of merge_size in Qwen3-VL video timestamp calculation (#37439)
Signed-off-by: chengyufang <cnyvfang@outlook.com>
|
2026-03-18 11:36:34 -07:00 |
|
|
Cyrus Leung
|
7476d148db
|
[Model] Remove unnecessary processor definition for Nemotron Parse (#37456)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-18 18:25:13 +00:00 |
|
|
Cyrus Leung
|
f3732bd931
|
[Misc] Clean up model registry (#37457)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-18 18:24:44 +00:00 |
|
|
Wentao Ye
|
0ef7f79054
|
[Perf] Add tuned triton moe config for Qwen3.5 H200, 9.9% E2E throughput improvement (#37340)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-03-18 14:18:34 -04:00 |
|
|
Xin Yang
|
b1169d7be8
|
[Kernel] Add gpt-oss Router GEMM kernel (#37205)
Signed-off-by: Xin Yang <xyangx@amazon.com>
|
2026-03-18 08:15:56 -07:00 |
|
|
XLiu-2000andXuLiu
|
17808394bc
|
standardize load_weights using AutoWeightsLoader for kimi_linear and minimax_text_01 (#37371)
Signed-off-by: XuLiu <xuliu40@gmail.com>
Co-authored-by: XuLiu <xuliu40@gmail.com>
|
2026-03-18 15:05:37 +00:00 |
|
|
elvischenv
|
296839a1b0
|
[Perf] Eliminate padding and slicing op for GPT-OSS with Flashinfer MXFP4 MXFP8 MoE (#30647)
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
|
2026-03-18 15:01:26 +00:00 |
|
|
Wentao Ye
|
c373b5c00d
|
[Log] Reduce duplicate log (#37313)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-03-18 10:57:44 -04:00 |
|
|
Cyrus Leung
|
99267c23ca
|
[2/3] Refactor InternVL-based processors (#37324)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-18 22:22:19 +08:00 |
|
|
Shwetha Poojary
|
cef1f302d2
|
[Model] Enable LoRA support for tower and connector in H2OVL (#31696)
Signed-off-by: shwetha-s-poojary <shwetha.s-poojary@ibm.com>
|
2026-03-18 13:26:47 +00:00 |
|
|
|
17c47fb869
|
[Bugfix] Fix EP weight filter breaking EPLB and NVFP4 accuracy (#37322)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-03-18 18:30:29 +08:00 |
|
|
Xin Yang
|
f1740006e4
|
[Perf] Enable dual stream execution of input projection for Qwen3 (#36795)
Signed-off-by: Xin Yang <xyangx@amazon.com>
|
2026-03-18 11:13:27 +08:00 |
|
|
Roy WangandClaude Opus 4.6
|
761e0aa7a0
|
[Performance] Add --enable-ep-weight-filter CLI option (#37351)
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-03-18 09:36:55 +08:00 |
|
|
Michael Goin
|
09e4576f65
|
[Kernel] Add non-gated support for NVFP4 CUTLASS MoE (#37320)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-03-17 18:12:04 -04:00 |
|
|
Athrael Sojuandwang.yuqi
|
c0745a851a
|
[Model] Add ColQwen3.5 4.5B support (#36887)
Signed-off-by: Athrael Soju <athrael.soju@gmail.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-03-17 21:17:02 +00:00 |
|
|
Ekagra Ranjan
|
b5ca9c3557
|
[Models] Cohere ASR (#35809)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
|
2026-03-17 21:04:17 +00:00 |
|
|
Chao-Ju ChenandMichael Goin
|
245758992e
|
[Bugfix] Rescale NVFP4 weight scales to fix BF16 dequant underflow (#34577)
Signed-off-by: ricky-chaoju <ricky.chen@infinirc.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-03-17 20:48:42 +00:00 |
|
|
Cyrus Leung
|
51f0acda79
|
[Model] Remove unused handle_oov_mm_token (#37321)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-17 19:44:52 +00:00 |
|
|
Andrey Talman
|
68f783a727
|
[Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673)
Signed-off-by: atalman <atalman@fb.com>
|
2026-03-17 18:47:59 +00:00 |
|
|
Cyrus Leung
|
c781fbbab3
|
[Bugfix] Standardize custom HF Processor init (#37289)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-17 15:38:55 +00:00 |
|
|
Anton VlasjukandHarry Mellor
|
2ff0ad9694
|
[UltraVox] Fix output type (#37224)
Signed-off-by: vasqu <antonprogamer@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-03-17 14:51:17 +00:00 |
|
|
Cyrus Leung
|
f340324335
|
[1/2] Move InternVL-based processors (#37260)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-17 21:50:56 +08:00 |
|
|
sfbemerkandBenjamin Merkel
|
2660b9289c
|
Bugfix for offloading+prefetch for GLM-4.7-FP8 (#37178)
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
|
2026-03-17 21:22:09 +08:00 |
|
|
Augusto Yao
|
9c7cab5ebb
|
[Feature]: Support for multiple embedding types in a single inference call (#35829)
Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>
|
2026-03-17 17:05:42 +08:00 |
|
|
|
24b4272a8c
|
Fix infinite recursive search issue in quark.py (#32779)
Signed-off-by: Yanwen Lin <lyw1124278064@gmail.com>
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Signed-off-by: kimheesu <wlskaka4@gmail.com>
Co-authored-by: Yanwen Lin <lyw1124278064@gmail.com>
Co-authored-by: Kim Hee Su <wlskaka4@gmail.com>
|
2026-03-17 07:19:15 +00:00 |
|
|
Benjamin Chislett
|
8a680463fa
|
[Bugfix] Fix NemotronH MTP + Chunked Prefill (#35447)
|
2026-03-17 07:07:33 +01:00 |
|
|
Nick Cao
|
20b14095a4
|
[Bugfix] Fix loading Music Flamingo (#35535)
Signed-off-by: Nick Cao <ncao@redhat.com>
|
2026-03-17 05:24:40 +00:00 |
|
|
Kyuyeun Kim
|
0a0a1a198b
|
Add ability to replace oot ops when using lora (#37181)
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
|
2026-03-16 18:04:15 -07:00 |
|
|
Julien Denize
|
7961486a9b
|
Fix EagleMistralLarge3Model initialization (#37232)
Signed-off-by: juliendenize <julien.denize@mistral.ai>
|
2026-03-16 15:41:00 -07:00 |
|
|
EdalatiAli
|
e5b807607c
|
[Quant][Feature] Support online MXFP8 quantization for MoE and dense models (#35448)
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
|
2026-03-16 18:07:39 -04:00 |
|
|
|
fd4d96302a
|
Fix eplb nvfp4 experts hook (#37217)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Elvir Crncevic <elvir@anthropic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-03-16 22:03:54 +00:00 |
|
|
Lucas Kabela
|
714c6e0eab
|
[torch.compile][BE] Modify cudagraph callable to check for is_forward_context_set (#36288)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
|
2026-03-16 19:42:34 +00:00 |
|
|
haosdentandOr Ozeri
|
ca1954d58c
|
[Bugfix] Disable cross-layer KV cache for MLA attention backends (#37090)
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-03-16 19:03:10 +02:00 |
|