Pierangelo Di Pilato and Or Ozeri
65a7b46284
[KV-Offloading] Support workload identity for objectstore secondary tier ( #47063 )
...
Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-07-07 16:30:16 +03:00
93e2ab7111
Disable dynamic speculative decoding when DP is enabled ( #45963 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-07-07 12:52:41 +00:00
Lanze Liu
8b745527cd
[Bugfix] Fix UBatchWrapper CUDA graph key to sum all ubatches, not just first two ( #43161 )
...
Signed-off-by: Lanze Liu <lanzetech@gmail.com >
2026-07-07 12:42:31 +00:00
Nick Cao and Claude
920469974a
[UX] Log worker exit code when process dies unexpectedly ( #38641 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-07 12:36:29 +00:00
Ting SUN and Nick Hill
8b91cd5b20
[Bugfix][Core] Close underlying iterator in merge_async_iterators single-iterator fast path ( #44726 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-07 05:13:41 -07:00
Harry Mellor
dd94484577
Bump Transformers version to 5.10.4 ( #41359 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-07 05:13:28 -07:00
Shaun Kotek
7ff656cc8b
fix: ensure no double load of lm head in nemotron mtp ( #47440 )
...
Signed-off-by: Shaun Kotek - Nvidia <skotek@nvidia.com >
2026-07-07 12:01:45 +00:00
danielafrimi
0a2965b1b3
[BugFix] Fix ModelOpt mixed-precision quantization for sparse quantized_layers configs. ( #47318 )
...
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com >
Signed-off-by: <dafrimi@nvidia.com >
2026-07-07 11:45:13 +00:00
Harry Mellor
0ed05b6f82
[CI] Fix Transformers modeling backend LoRA test ( #47832 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-07 11:40:00 +00:00
Guan-Ming Chiu
ed051fab54
[Bugfix] Reject sampling params unsupported by diffusion models ( #45418 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
2026-07-07 11:25:36 +00:00
Ronen Schaffer and Or Ozeri
48fcfc926c
[KV Offload] Add ParentManager ABC for secondary tier callbacks ( #47274 )
...
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-07-07 13:51:18 +03:00
3354dba381
[Bugfix][KV offload] Store interior chunk-boundary blocks under MTP/Eagle ( #46972 )
...
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-07-07 13:16:52 +03:00
cbb5f045be
[ROCm][CI] Refresh ROCm base images when docker rocm_base changes ( #46904 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: Codex <codex@example.invalid >
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Codex <codex@example.invalid >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-07-07 03:10:50 -07:00
Juan Pérez de Algaba and Cursor
b3e85be663
fix: use configured max_logprobs instead of hardcoded 20 in derender validation ( #47834 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-07-07 09:42:47 +00:00
Summer Yang
d3e69fd671
[Perf] Use blocking CUDA events to avoid busy polling cuda driver lock ( #47081 )
...
Signed-off-by: Jingyi Yang <girasoleyang@gmail.com >
2026-07-07 09:36:10 +00:00
c85d72076a
[HARDWARE][POWER] optimize math functions of VSX power ( #47321 )
...
Signed-off-by: Akash Kaothalkar <akashkaothalkar@akashs-mbp.bl1-in.ibm.com >
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com >
Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com >
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com >
Co-authored-by: Akash Kaothalkar <akashkaothalkar@akashs-mbp.bl1-in.ibm.com >
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com >
2026-07-07 09:35:47 +00:00
Kyung Sub Lee (Daniel) and Claude Fable 5
c5b66233b2
[Bugfix][Spec Decode] Skip uniform spec-decode padding for diffusion models ( #47464 )
...
Signed-off-by: kl527 <kl527@cornell.edu >
Signed-off-by: Kyung Sub Lee (Daniel) <66861800+kl527@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-07 09:25:12 +00:00
Netanel Haber and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
066f02ae94
[MoE] FI autotuning: max bucket = max token count [e.g. DP_size*MNBT] ( #47427 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 12:08:36 +03:00
Jee Jee Li
5d23ca47ab
[Kernel] Applies routed_scaling_factor internally ( #47408 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-07-07 02:00:54 -07:00
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e55cc59e52
[Rust Frontend][CI] Unblock more end-to-end test cases ( #47735 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-07 08:27:20 +00:00
ba50b9763f
[Bugfix] Match the mapped filename in find_loaded_library ( #47586 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-07-07 08:06:29 +00:00
b4cfbc24d3
[Bugfix][Core] Fix host memory leak from undrained new_block_ids ( #44490 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-07-07 07:32:55 +00:00
Aritra Roy Gosthipaty
1e823dc01d
[docs update] Update usage of hf cli for cache list and removal ( #47830 )
...
Signed-off-by: Aritra Roy Gosthipaty <aritra.born2fly@gmail.com >
2026-07-07 07:09:18 +00:00
Juan Pérez de Algaba and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8e61b646e2
fix(security): add resource bounds validation to derender endpoints ( #47260 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 14:58:26 +08:00
e040899a00
[KV Offloading] Add basic offloading metrics ( #45958 )
...
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com >
Signed-off-by: Srinivas Krovvidi <194645829+Srinivasoo7@users.noreply.github.com >
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com >
Co-authored-by: Or Ozeri <or@ozery.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-07-07 09:26:28 +03:00
Micah Williamson and Andreas Karatzas
dd5c299fbe
[ROCm][Bugfix] Convert ModelOpt FP8 per-channel weights to e4m3fnuz on MI300/MI325 ( #47201 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-06 23:24:58 -07:00
xiangdong
6db31c8e76
[XPU][CI]Adjust memory request for tests in Intel GPU CI ( #47758 )
...
Signed-off-by: zengxian <xiangdong.zeng@intel.com >
2026-07-07 05:56:40 +00:00
cbe9c40f99
[Bugfix] Forward callable hf_overrides to the draft model config ( #45352 )
...
Signed-off-by: HumphreySun98 <humphreysun98@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-07-06 21:12:38 -07:00
Andreas Karatzas
2f71b2bd9f
[ROCm] Align mixed encoder-decoder KV cache views in V2 runner ( #47685 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-07 12:09:22 +08:00
Jee Jee Li and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
32ab064621
[UX] Add model_class_overrides for development and debugging ( #47148 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 11:43:08 +08:00
Guan-Ming Chiu
c64c356990
[Perf] Bound DiffusionGemma sampler transient via request-tiled logits ( #45672 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
2026-07-07 03:42:01 +00:00
Tahsin Tunan
34e6dfced8
[Rust Frontend] Stamp arrival_time at the frontend entry ( #47787 )
...
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com >
2026-07-07 03:27:10 +00:00
Reid
39a1d32b59
[Rust Frontend] Avoid extra copies for multimodal tensors ( #47581 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-07-07 03:09:55 +00:00
700e882eab
Add TorchCodec as a video decoding backend ( #46609 )
...
Signed-off-by: Nicolas Hug <contact@nicolas-hug.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-07-06 19:58:51 -07:00
jialoop-git and Claude Opus 4.8
a4f019fa25
fix(distributed): propagate distributed_timeout_seconds to NCCL device groups ( #45159 )
...
Signed-off-by: jialoop-git <joane8913456@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-07 02:52:51 +00:00
Rahul Vishwakarma
9dd2465896
feat(cpu): add CPU support for Mamba ShortConv ( #35059 )
...
Signed-off-by: Rahul Vishwakarma <Rahul.Vishwakarma2@ibm.com >
2026-07-07 10:47:08 +08:00
Reid
a46c9329e5
[Rust Frontend] Add DeepSeek V3.2 roundtrip fixture ( #47619 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-07-07 10:47:02 +08:00
Kyle Sayers
445321fab4
[Bugfix] [Quantization] Fix loading for CT DSV2 ( #47780 )
2026-07-07 02:28:00 +00:00
YiSheng5 and Kunshang Ji
69f3150981
[XPU] Fix PP accuracy on XPU device ( #47253 )
...
Signed-off-by: yisheng <yi.sheng@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-07 09:17:27 +08:00
86db6c3070
[Frontend] add per-request timing metrics field to response body of Chat/Completions APIs ( #46768 )
...
Signed-off-by: Nicholas Edelman <nedelman@nvidia.com >
Signed-off-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: GPT-5.5 <noreply@cursor.com >
2026-07-06 17:48:29 -07:00
akii96 and Andreas Karatzas
5769a7382c
[ROCm][CI][Bugfix] Fix flaky parallel tool-call streaming (test assertion + Mistral/Granite parsers) ( #47550 )
...
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-06 19:06:19 -04:00
Andreas Karatzas
8484ca5d45
[ROCm][CI] Adding Rust parity ( #47478 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-06 15:05:39 -07:00
Simon Danielsson and nnyrhila
482e5524fe
[Bugfix][ROCm] Fix memory access fault in AITER MLA backend for DPA+FP8 KV ( #47276 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: nnyrhila <niko.nyrhila@amd.com >
2026-07-06 21:30:02 +00:00
567a78432d
[Bugfix] Fix dp mtp hang ( #40589 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: sherryC41 <sherry.c.c41@gmail.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-07-06 21:08:17 +00:00
Jinzhen Lin and mgoin
d891b9bd51
[Quantization] add humming moe backend to all dense/moe oracles ( #41652 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-07-06 13:36:07 -07:00
04adc8843b
[Bugfix]Fix DeepSeek-V4 fp8_ds_mla KV cache reshape ( #47716 )
...
Co-authored-by: yy-fighting <23518844576@qq.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-07-06 12:56:44 -07:00
Harry Mellor
ae098abe3f
[CI] Fix some errors on main ( #47726 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-06 19:40:23 +00:00
Andrii Skliar and Andrii Skliar
b1384f5ec6
Enable B12x backend for non-gated MoEs (like Nemotron) ( #43328 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-07-06 12:40:07 -07:00
Nathaniel McVicar and Harry Mellor
b136cc2c2c
[Bugfix][Model] Add stability window to DiffusionGemma to match HF stability_threshold semantics ( #45965 )
...
Signed-off-by: Nathaniel McVicar <namcvica@microsoft.com >
Signed-off-by: Nathaniel McVicar <Nathaniel.McVicar@microsoft.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-06 19:39:12 +00:00
9fde043f54
[Kernel][Helion][1/N] Add Helion kernel for silu_and_mul_per_block_quant ( #43994 )
...
Signed-off-by: Sean Chen <seachen@redhat.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 00:19:01 +08:00
