e5e38ba9b7
[Model Runner V2] Support randomized dummy inputs ( #58411 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-01 00:52:26 -07:00
Varun Sundar Rabindranath
5463fe4962
[KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA ( #57652 )
2026-09-30 09:11:03 -04:00
3d5625cc3c
[Security] Authenticate shared-memory multimodal cache handles ( #59357 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-30 05:16:26 -07:00
765872e7ed
[Core][BugFix] Tag prefix-cache extra keys by source ( #51899 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: Tai An <antai12232931@outlook.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 18:24:59 -07:00
Tyler Michael Smith and Codex
50239dfc4d
[WideEP] Change DeepEPv2 to auto select hybrid mode by default ( #57991 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-29 18:36:30 -04:00
faacc13565
[Perf][KV Connector][Mooncake] Pack hybrid/MLA KV into coalesced transfer regions ( #57952 )
...
Signed-off-by: staryxchen <staryxchen@tencent.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-09-29 13:29:04 -07:00
YukioZzz and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
30ae8248b6
[KVConnector][MoRIIO] Support K3 DSpark hybrid READ ( #57700 )
...
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 19:14:24 +00:00
Andreas Karatzas and OpenAI Codex
3acddf01e7
[Bugfix][ROCm] Keep Mooncake bootstrap ports bound during startup ( #58967 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-29 04:14:26 -05:00
Tyler Michael Smith and Codex
4406874e7c
[Bugfix][FT] Pass stateless process-group timeouts explicitly ( #58611 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-29 03:14:02 +00:00
Almog Tavor
171d1b9ef6
[Elastic EP] Support Model Runner V2 ( #53934 )
...
Signed-off-by: almogtavor <almogtavor@gmail.com >
Signed-off-by: Almog Tavor <70065337+almogtavor@users.noreply.github.com >
2026-09-29 00:20:40 +00:00
764413559a
[Bugfix] Don't sync-police or retry FlashInfer all-reduce workspace creation in eager mode ( #58498 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-28 12:14:21 -07:00
qizixi
6d3ea3c2c9
[KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill ( #50499 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
2026-09-28 18:50:46 +02:00
Chao Lei
386ac2573e
[Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout ( reopens #55763 ) ( #58919 )
2026-09-27 19:47:36 +00:00
YannikHinteregger and xijiade.aihemaiti
fff03267da
[Bugfix][NIXL] Release a dead peer's NIXL state without waiting for TTL ( #50047 )
...
Signed-off-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com >
Co-authored-by: xijiade.aihemaiti <3146335281@qq.com >
2026-09-26 20:30:09 -07:00
eb0f2ca37f
[mooncake] support CUSTOM_MEM_POOL in vllm ( #49300 )
...
Signed-off-by: bruce.xu <bruce.xb@alibaba-inc.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: bruce.xu <bruce.xb@alibaba-inc.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-27 00:26:05 +00:00
mevince
cd1daf58a9
[Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector ( #57251 )
...
Signed-off-by: Vincent <vincexxchan@gmail.com >
2026-09-26 17:17:11 -07:00
927c87b347
[PCP][DCP] Support DCP target model with non-DCP Dspark ( #56723 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-26 18:12:35 -04:00
Itay Alroy and Tyler Michael Smith
2bb7605ae1
[Elastic EP] Fix EPLB load statistics during scaling ( #58473 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-09-26 12:05:37 +00:00
b6d2b4fc25
[RL] Add sharding-aware NCCL M2N weight transfer ( #51520 )
...
Signed-off-by: Ke Wen <kwen@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-26 01:38:53 +00:00
GokayAI and GokayAI
c89b5b4176
[Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head ( #58292 )
...
Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com >
Co-authored-by: GokayAI <gokay-ai@users.noreply.github.com >
2026-09-25 17:23:44 +00:00
Mark McLoughlin and AI Assistant
45d56c0baf
[Core] Model console logging as CLI configuration ( #57205 )
...
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: AI Assistant <noreply@openai.com >
2026-09-25 11:36:59 +01:00
Yongye Zhu and Summer Yang
442bd00d9f
[Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 ( #55072 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
2026-09-25 03:27:21 +00:00
b3a87b2d04
[Bugfix][KVConnector] Finalize saves on steps without a forward ( #57775 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
2026-09-25 00:06:36 +00:00
7a9406690e
[KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode ( #58472 )
...
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-24 23:55:47 +00:00
stefankoncarevic and Andreas Karatzas
9ac3b7c69f
[ROCm][CI] Mirror the DSv4-Flash disaggregated DP EP group on MI355 ( #58558 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-24 18:38:25 -05:00
snadampal and Nicolò Lucchesi
6a86bf626e
[PD][PushConnector] Record last activity of remotes on the D side ( #52245 )
...
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
2026-09-24 23:15:32 +00:00
92e9c6bf13
[ROCm] Fix misrouting race-condition in multi-decode P/D disagg with mori-io ( #51681 )
...
Signed-off-by: Vincent Cave <vincent.cave@amd.com >
Signed-off-by: Shiksha Patel <shikpate@amd.com >
Co-authored-by: Shiksha Patel <shikpate@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-24 19:52:43 +00:00
Kapil Arya and Or Ozeri
70fc359d25
[Bugfix][KV Offload] Retain offload event metadata through batch translation ( #57453 )
...
Signed-off-by: Kapil Arya <kapila@nvidia.com >
Signed-off-by: Kapil Arya <kapil.arya.17@gmail.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-09-24 07:09:12 +03:00
e6dc16cebd
[PCP] Support prefill context parallelism with data parallelism ( #57075 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
Dao007forever and Codex
c4f6ce4874
[Bugfix][NIXL] Restore successful push completion reporting ( #58188 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-23 09:39:04 -07:00
Nicolò Lucchesi and Yifan Qiao
9488318c6a
[Mooncake] Address review nits from #56855 ( #57174 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Yifan Qiao <17067717+ivanium@users.noreply.github.com >
2026-09-23 10:15:45 +02:00
Tianyu Guo and Cyrus Leung
3ee2906f47
[EPD] Support metadata-only audio inputs ( #57887 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-23 09:03:50 +08:00
Tony Lin and Flora Feng
e4340e41c9
[Feat][XPU] VLLM_BATCH_INVARIANT support for Dense/MoE models ( #55881 )
...
Signed-off-by: Tony Lin <tony.lin@intel.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-22 11:06:54 -04:00
Karen Chung
986e217158
[Feature] Add first-class KV hints request envelope for programmatic KV management ( #53423 )
...
Signed-off-by: Karen Chung <karenc@nvidia.com >
2026-09-21 09:56:11 -07:00
Lucas Wilkinson and OpenAI Codex
8c0825090d
[Bugfix][NIXL] Fix DCP pulls across MLA cache regions ( #57389 )
...
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 11:57:49 -04:00
0b7f11a1ee
[AuxOutput] Add block-keyed storage for routed-expert outputs ( #45635 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-21 10:25:05 +00:00
YiSheng5 and Kunshang Ji
17e50b9b76
[XPU][UT]Bugfix when the process can't see all the world_size meet accuracy issue. ( #57779 )
...
Signed-off-by: yisheng <yi.sheng@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-21 06:50:25 +08:00
YukioZzz
9679173788
[KVConnector][MoRIIO] Transfer hybrid mamba/KDA recurrent state in READ mode ( #51052 )
...
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com >
2026-09-20 17:30:34 -04:00
90531daf6d
[Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints ( #55844 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-18 18:11:05 -07:00
ys2025-AI
6aab78f477
[Bugfix][KVConnector] Make ExampleHiddenStatesConnector abort-safe ( #56841 )
...
Signed-off-by: ys2025-AI <yuanshuaien@163.com >
2026-09-18 21:53:02 +00:00
Nicolò Lucchesi and OpenAI Codex
49e0427ca5
[Bugfix][NIXL] Avoid receive reports for notification-only requests ( #57570 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-18 13:33:02 -07:00
Nicolò Lucchesi
164dca8d1e
[Nixl] Separate transport-failure metrics from KV expiry ( #55854 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-09-18 18:08:59 +02:00
Kevin Li and kevinli
911548247a
[Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend ( #52101 )
...
Signed-off-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com >
Co-authored-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com >
2026-09-18 10:34:01 -04:00
67e5b0acc9
[Bugfix][HiSparse][NIXL] Import full blocks without tail prefill on D ( #57049 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-09-17 15:26:10 -04:00
9e1cd262cb
[Bugfix] Use DP index for dense DP weight updates and EC CPU region ( #56950 )
...
Signed-off-by: lxy-alexander <lxy_alexander@outlook.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-17 22:44:36 +08:00
0d72627565
[Perf][EPD] Batch image requests per encoder ( #57095 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com >
Co-authored-by: Zhou ziheng <jiaranran2@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-17 03:45:52 -07:00
Itay Etelis and Or Ozeri
b6e7c1f1f0
[kv_offload] Skip scratch groups ( #57145 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-09-17 13:22:49 +03:00
40b40d1d39
[Bugfix][KV Offload] Register the offload region in chunks ( #51081 )
...
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Colton Ottley <coltonottley@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-09-17 13:19:56 +03:00
Qiming Zhang
e6b1a5e5e3
[XPU] Enable XPU eplb ( #44987 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com >
2026-09-17 13:19:13 +08:00
2743dc4f92
[Feature][Multimodal] Add cross-encoder caching to Mooncake P2P ( #56242 )
...
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com >
Co-authored-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-17 13:16:41 +08:00
