 Ronen Schaffer
|
4bfa0f2b14
|
[KV Offload] Rename SecondaryTierManager.get_finished() to get_finished_jobs() (#43870)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-28 16:00:18 +00:00 |
|
|
Vadim Gimpelson
|
5d126dd155
|
[Bugfix] Exclude Ray DP from #42585's deferred port allocation (#43864)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
|
2026-05-28 15:55:14 +00:00 |
|
|
Angelo RuoccoandOr Ozeri
|
864990e8d9
|
Add token-offset based selective offload in OffloadConnector (#39983)
Signed-off-by: Angelo Ruocco <ang@zurich.ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
|
2026-05-28 14:11:02 +00:00 |
|
|
Rotem Shavitt
|
1b16f2ddc9
|
change name of fs_python secondary tier to fs. (#43600)
Signed-off-by: Rotem Shavitt <rshavitt@gmail.com>
|
2026-05-28 07:05:48 +00:00 |
|
|
Nick HillandWentao Ye
|
8c94938cfb
|
[MRV2][BugFix] Fix KV connector handling in spec decode case (#43719)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-05-27 06:37:56 +00:00 |
|
|
|
1fc2cee50a
|
[KVConnector][Mooncake] Wire reset_cache cascade end-to-end (#42694)
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-26 20:52:35 -07:00 |
|
|
Matthew Bonanni
|
aa6138169f
|
[MLA][Attention] Add OOT MLA prefill backend registration mechanism (#43325)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-05-26 19:56:09 -07:00 |
|
|
|
7e33081cee
|
[Attention] Make FlexAttention and FlashAttention use num-blocks first layouts (#42095)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
|
2026-05-26 19:55:56 -07:00 |
|
|
linzm1007andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
739af5c7e1
|
[Reasoning] [Bugfix] Reject invalid thinking_token_budget values (#43402)
Signed-off-by: linzm1007 <linzm1007@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-26 03:37:30 -07:00 |
|
|
Simon DanielssonandTJian
|
681d7dd38b
|
[Misc][Refactor][ROCm] Convert MoRI-related envvars to extra config args (#43303)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-05-26 03:33:35 -07:00 |
|
|
Ethan Feng
|
755043cf3c
|
[KV Transfer] Enable HMA by default for connectors that support it (#41847)
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
|
2026-05-26 12:28:51 +02:00 |
|
|
Dao007forever
|
c2a4005c70
|
[KV Connector] Propagate MooncakeStore load failures (#42788)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
|
2026-05-25 22:12:15 -07:00 |
|
|
Dao007foreverandClaude
|
7966fc7233
|
[KV Connector][Bugfix] MooncakeStore: don't double-apply Eagle prune in load_mask (#43516)
Signed-off-by: Dao Le <daole@inferact.ai>
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-05-25 22:11:57 -07:00 |
|
|
Nicolò Lucchesi
|
716d5294e6
|
[Misc] Print accuracy value for PD tests even on success (#43583)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-05-25 02:10:01 -07:00 |
|
|
Zhewen LiandZhewen Li
|
873758c13a
|
[KV Connector] Handle Mooncake finish after preemption (#43281)
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-05-25 01:58:38 -07:00 |
|
|
Yifan Qiao
|
81252d4e24
|
[Feat][KVConnector] Support DSV4 in SimpleCPUOffloadBackend (#42296)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-05-25 14:04:30 +08:00 |
|
|
Rotem ShavittandOr Ozeri
|
d0a100c87a
|
File system secondary tier implemented in python (#41735)
Signed-off-by: Rotem Shavitt <rshavitt@gmail.com>
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-05-24 18:14:44 +00:00 |
|
|
Or Ozeri
|
357fddf614
|
[kv_offload]: Add DSv4 support (#43142)
Signed-off-by: Or Ozeri <oro@il.ibm.com>
|
2026-05-24 11:10:12 +03:00 |
|
|
Dao007foreverandClaude
|
0902d8e62f
|
[KV Connector] Keep MooncakeStore full hits block-aligned (#43494)
Signed-off-by: Dao Le <daole@inferact.ai>
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-05-23 23:15:03 -07:00 |
|
|
Dao007forever
|
819c610f9b
|
[Mooncake] Add metrics for MooncakeStoreConnector operations (#43392)
|
2026-05-23 13:34:40 -07:00 |
|
|
|
5bb8d2767a
|
[Kernel] Batch invariant NVFP4 linear using cutlass (#39912)
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-05-23 09:41:12 -04:00 |
|
|
Gabriel Wu
|
82536acc54
|
Keep scheduler alive for delayed KV connector frees (#43433)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
|
2026-05-23 06:23:32 +00:00 |
|
|
Xiaogang Zhouandxiaogang.zhou
|
d19db10974
|
[Bugfix] Fix native Triton top-k/top-p kernel assumes contiguous logi… (#42739)
Signed-off-by: xiaogang.zhou <xiaogang.zhou@bytedance.com>
Co-authored-by: xiaogang.zhou <xiaogang.zhou@bytedance.com>
|
2026-05-22 22:56:16 -07:00 |
|
|
Ashwin Giridharanandabinggo
|
84e351555a
|
[Bugfix] Auto-raise max_num_batched_tokens for prefix-LM multimodal models (#43051)
Signed-off-by: Ashwin Giridharan <girida@amazon.com>
Co-authored-by: abinggo <107740309+abinggo@users.noreply.github.com>
|
2026-05-22 21:23:50 -07:00 |
|
|
Benjamin Chislettandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
4e2eba28be
|
[Perf] Optimize hidden state extraction logic (#37374)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-05-22 18:23:08 -04:00 |
|
|
Dao007foreverandClaude Opus 4.7
|
b21f3d56d4
|
[KV Connector] MooncakeStore: don't co-queue save with load to avoid double delayed-free (#43371)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-22 16:14:11 +00:00 |
|
|
Isotr0py
|
f0feb15e7f
|
[Multimodal] Simplify ViT CUDA graph interfaces (#41234)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-05-22 22:31:00 +08:00 |
|
|
Weida Hong
|
6bb8753db1
|
Correcting the mock classes for MM GC tests (#43321)
Signed-off-by: Weida Hong <wdhongtw@google.com>
|
2026-05-22 15:21:35 +08:00 |
|
|
haosdentandWentao Ye
|
5ea76fa89a
|
[CI] Fix test_lora_with_spec_decode on V2 model runner (#43314)
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-05-22 14:24:18 +08:00 |
|
|
Lanze Liu
|
39d5fa96a7
|
[Bugfix] Zero stale is_prefilling in padded CUDA graph rows for Mamba (#41873)
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
|
2026-05-21 15:42:42 -07:00 |
|
|
Francesco Fuscoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
b730c46352
|
[Perf] [Hybrid] Fused Triton kernel for GPU-side Mamba state postprocessing (#40172)
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-21 04:50:54 -07:00 |
|
|
haosdentandNickLucche
|
f2d5e3d3ae
|
[CI] Lower granite-4.0-h-tiny gsm8k threshold for Hybrid SSM NixlConnector PD accuracy tests (4 GPUs) (#43186)
Signed-off-by: haosdent <haosdent@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
|
2026-05-20 17:00:24 +00:00 |
|
|
rishitdholakia13andCursor
|
ded871201a
|
[Bug][Structured Outputs] Fix bug that leads to unconstrained generations with structural tags (#42452)
Signed-off-by: rishitdholakia13 <rishit+github@cohere.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-05-20 07:08:58 -07:00 |
|
|
Kebe
|
19cf334207
|
[Feature] Support manually enabling the cumem allocator (#33648)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2026-05-20 08:58:30 -04:00 |
|
|
Ronen Schaffer
|
4f940896a3
|
[KV Offload] Pass OffloadingSpec instead of VllmConfig to secondary tiers (#43076)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-20 03:32:08 +00:00 |
|
|
Benjamin Chislett
|
c628a93a64
|
[Perf][Bugfix] Update dflash aux layer indexing (#40727)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-05-19 20:15:57 -07:00 |
|
|
Nick Hill
|
b82e908b4c
|
[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-19 10:35:54 -04:00 |
|
|
zhanqiuhuandNicolò Lucchesi
|
129019f334
|
[CI] Add MTP + PD disagg test for Qwen3.5 (#42677)
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
|
2026-05-19 11:44:33 +02:00 |
|
|
Yifan Qiao
|
056bc2e166
|
[KVConnector][DSV4] HMA support for Mooncake store connector (#42828)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-05-19 01:07:46 -07:00 |
|
|
|
fab07e4d0f
|
[Bugfix][KV Connector] Fix SimpleCPUOffloadScheduler TOCTOU between Phase A and Phase B (#42289)
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: gemini-code-assist <noreply@google.com>
|
2026-05-18 21:22:33 -07:00 |
|
|
Ronen Schaffer
|
84747489de
|
Tier offload followup (#42529)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-18 19:41:58 +00:00 |
|
|
Netanel Haber
|
47829b1159
|
[Bugfix] mamba: run single-token extends as decodes (#42430)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
|
2026-05-18 15:26:00 +00:00 |
|
|
|
e5417657e5
|
[KV Connector][Offloading] Flush all pending jobs on last step (#42611)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: liranschour <liranschour@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-18 12:59:42 +00:00 |
|
|
roikoren755
|
737bfa3a43
|
[Bugfix][Hybrid][NemotronH] Fix mamba_cache_mode=all + speculative decoding crash (#41233)
Signed-off-by: Roi Koren <roik@nvidia.com>
|
2026-05-18 14:54:00 +03:00 |
|
|
AlecandOpenAI Codex
|
107210442d
|
[CI] Add NIXL EP import canary (#42567)
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-05-17 19:11:46 -07:00 |
|
|
|
36e74c9ea4
|
[KV Connector] Support disk offloading in MooncakeStoreConnector (#42689)
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-16 13:34:15 -07:00 |
|
|
Jiangyun Zhu
|
8a56da3845
|
[Experimental] Breakable CUDA graph (#42304)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-05-16 22:04:12 +08:00 |
|
|
Yifan Qiao
|
4b364f810e
|
[Core][DSV4] Skip caching SWA blocks that can never serve a prefix-cache hit (#42258)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-05-15 15:59:18 +08:00 |
|
|
|
bf610c2f56
|
[Bugfix] Fix inverted condition causing thinking_token_budget to be silently ignored (#41674)
Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-15 12:48:49 +08:00 |
|
|
Matthew Bonanni
|
9898f94abe
|
[Attention] Remove deprecated MLA prefill arguments (#42555)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-05-14 10:34:06 -07:00 |
|