696 Commits
Author SHA1 Message Date
7879f3687b [NIXL] Drop unreachable producer-side notif counting in push mode (#62210)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-17 21:24:35 +00:00
c3de0b37e2 [Core][Spec Decode] Replace EAGLE group heuristics with explicit draft-layer tags (#61455)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 08:22:33 +00:00
41b8c15809 [Feature][MoRIIO] Support hybrid WRITE transfers with DSpark (#60544)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 04:57:58 +00:00
Sherif Walyandvllm-agent ce8853d27a [Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts (#61674)
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-17 02:11:12 +00:00
Hexiang Wang 9a16d43ef0 [Bugfix][Mooncake] Fix MultiConnector stats and proxy SSE responses (#61218)
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
2026-10-16 17:54:48 +08:00
33212db2c2 [KV Connector][NIXL] Reject NVFP4 KV cache with heterogeneous TP head split (#61865)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-15 13:57:37 -07:00
49f944be3e [Perf][HiSparse] Reuse pinned host pool after graph profiling (#61668)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:33:03 -07:00
CharlesWuandYichao Zhu a97d19d792 [UMBP][1/10] Add shared connector, runtime contracts, and request lifecycle (#61789)
Signed-off-by: CharlesWu <Charles.Wu@amd.com>
Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com>
2026-10-15 15:12:39 +00:00
7020600b1d [KV Connector][NIXL] Register packed MLA KV rows as a single region (#61074)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 09:18:15 +00:00
Misha GoinandClaude Opus 5.5 1c4c319792 [Frontend] Advertise --enable-x/--disable-x for boolean CLI flags (#61479)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 01:08:02 -07:00
liranschourandKevin H. Luu c5f8adc0b1 [KV Offload] Expose OffloadingManager.lock()/unlock() and service the tiering control plane off the scheduler thread (#58168)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-15 08:50:56 +03:00
6605802904 [Perf][KV Connector][Mooncake] Send the KV layout once per peer (#59825)
Signed-off-by: staryxchen <staryxchen@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
2026-10-15 05:36:31 +00:00
429a3da665 [Bugfix][ROCm][Disagg] Allow MoRIIO HMA per-layer block_len (#58585)
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com>
Signed-off-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-10-15 09:35:31 +08:00
Nick HillandIlia Yastrebov eaf25f675a [NIXL] Prepare KV transfer descriptors as NIXL 1.5 strided lists (#61032)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Ilia Yastrebov <iyastrebov@nvidia.com>
2026-10-14 18:07:24 +02:00
775fefa804 [Metrics][KV Offload] Add vllm:kv_offload_config_info - generic info metrics for offloading the offloading connector (#56867)
Signed-off-by: Amir Friedman <Amir.Friedman1@ibm.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
2026-10-14 12:34:37 +00:00
linshi wangandOr Ozeri c67c69e657 [Bugfix][KV Offload] Track allocated block IDs when num_external_tokens is 0 (#47046)
Signed-off-by: linshi-w <linshiw.edu@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
2026-10-14 13:34:10 +03:00
3ccc0a90b6 [KV Connector] Support sleep mode with MooncakeConnector over RDMA (#59625)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com>
2026-10-14 04:18:47 +00:00
8082b62f32 [Mooncake] Allow Mamba cache mode none for capacity-only workers (#60712)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:13:58 +00:00
Hexiang Wang 7342dc0317 [MoRIIO] Require pending remote prefill for READ cache hits (#60638)
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
2026-10-13 20:19:53 +08:00
Akshay ViswanathanAkshay ViswanathanCursorSimon Danielssonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Hongxia Yang
802feb6770 [Bugfix][KVConnector] Fix MoRI-IO WRITE-mode requests stranded in WAITING_FOR_REMOTE_KVS (#57705)
Signed-off-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com>
Co-authored-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
2026-10-12 22:29:31 +00:00
5b0b8f251e [Bugfix][HiSparse] Fix TP group hang on steps without scheduled tokens (#60941)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 17:38:12 +00:00
2339bd39d6 [Bugfix][Mooncake] Use the EAGLE replay boundary when retaining sliding-window tails (#61012)
Signed-off-by: Stu Cao <stu@primeintellect.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-10-11 09:15:18 +00:00
Matthew BonanniandClaude Opus 5.5 d5f356dae0 [HiSparse] Own GPU residency instead of editing core block-table rows (#60647)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:25:44 -07:00
f8e6e365ae [Bugfix][Mooncake] Save the EAGLE attention proof with hybrid Mamba partial tails (#60695)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Signed-off-by: Dao007forever <daole@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
2026-10-10 00:08:08 +00:00
Matthew BonanniandClaude Opus 5.5 4c43254d52 [HiSparse] Report host-pool block residency as HiSparse metrics (#59485)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 16:45:34 -06:00
02e55d2af3 [KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-10-09 15:48:37 -04:00
Luca MotzandClaude Opus 5.5 83ed0c7c09 [Bugfix][NIXL] Fix receive post-process for different P/D block sizes (#58860)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:30:11 +08:00
Chao Lei bc713cf8cc [Bugfix]MooncakeConnect add query timeout and retry (#55923) 2026-10-09 09:19:33 +00:00
The Anh Nguyen bacbbe1878 [Bugfix][NIXL] Guard record_transfer against None telemetry (#57734)
Signed-off-by: The Anh Nguyen <ntheanh201@gmail.com>
2026-10-09 08:22:18 +00:00
72d59adc2c [Bugfix][KV Connector][NIXL] Transfer the whole PLE short-conv page in disaggregated P/D (#59997)
Signed-off-by: Alberto Massidda <amassidda@nvidia.com>
Co-authored-by: Alberto Massidda <amassidda@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-08 23:58:13 +04:00
zzaebok 2f7f05c256 [Bugfix][MoRIIO] Prevent remote KV reload after decoder preemption (#53250)
Signed-off-by: Jaebok Lee <jaebok9541@naver.com>
2026-10-08 22:28:16 +08:00
daa9085143 [KV Connector][NIXL] Register HiSparse host pool under single TP Rank (#60161)
Signed-off-by: Pravein Govindan Kannan <pravein.govindan.kannan@ibm.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 07:25:02 -07:00
Matthew BonanniandClaude Opus 5.5 4ed4c4433b [Bugfix][HiSparse] Drop the unaccounted, write-only prefill mirror staging buffer (#60429)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:00:49 +00:00
ef63c23d35 [Bugfix][DSv4.1] Skip SWA bounded replay for KV loads that carry the window (#59197)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 15:28:42 -07:00
MohitAMDandCursor 240ee8de9e [Bugfix][KV Connector][ROCm] Fix MoRI-IO KV block-offset for speculative decoding (MTP) (#55053)
Signed-off-by: Mohit Deopujari <Mohit.Deopujari@amd.com>
Signed-off-by: MohitAMD <mohit.deopujari@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-07 11:33:30 -05:00
df417f780a [Bugfix][MoRIIO] Exclude synchronous READ destinations from KV zeroing (#59164)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: Hexiang Wang <56632993+whx-sjtu@users.noreply.github.com>
Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
2026-10-07 16:35:56 +08:00
e37e51dd24 [Metrics] Expose cached prompt tokens by cache tier (#56318)
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
2026-10-06 14:52:05 -04:00
ahmed xijiaatOpenAI CodexNick HillYannik Hintereggermergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
9465e1c783 [Bugfix][NIXL] Recover locally invalidated pull peer metadata (#55471)
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 18:46:53 -07:00
2537629eb2 [Bugfix][KVConnector][NIXL] Size per-region replicate flags by each region's block count (#60107)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-10-06 01:27:50 +00:00
0e8f8ecba8 [Bugfix][KVConnector][NIXL] Return NumPy descriptors from mixed-memory local registration (#60108)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-10-06 01:23:20 +00:00
2e3154aa18 [HiSparse] Log steady-state max concurrency and expose host-tier utilization gauges (#58949)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-10-05 14:20:32 -07:00
Maroon Ayoub b1401e0aa7 [Bugfix][NIXL] Count heartbeats as remote engine activity (#59873)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
2026-10-05 09:36:13 +00:00
Hexiang WangandCodex f03026a548 [Bugfix][MoRIIO] Keep discovery heartbeats running while workers hold the GIL (#59441)
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-03 14:55:18 +00:00
Yicong Wang 5f30fc7031 [Bugfix][KV Connector][Mooncake] Suppress completion for empty pulls (#59347)
Signed-off-by: wangyicong <wangyicong@bytedance.com>
2026-10-03 15:28:00 +08:00
ca0df1f122 [Bugfix][Core] Exempt exactly the blocks an async KV load writes from zeroing (#59504)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-03 06:26:12 +00:00
4056c8ac1f [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-10-01 23:04:21 -07:00
ca65eb67d9 [Perf][HiSparse] Avoid repeated prefix scans and residency updates (#57930)
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-10-01 21:48:18 +00:00
Zijing Liu ea84051819 [KVConnector][NIXL] Count completion notifications that arrive after KV expiry (#58875)
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
2026-10-01 21:30:58 +00:00
Chanbin LimandChendi.Xue 47de9d4d04 [KV Connector][NIXL] Coalesce host-buffer KV copies across cache groups (#54483)
Signed-off-by: beenpow <57864041+beenpow@users.noreply.github.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
2026-10-01 13:10:54 -05:00
cb6458c5ce [Core] Remove deprecated mamba_cache_mode "all" (#58997)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
2026-10-01 02:59:37 -07:00