iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,521 Commits 3 Branches 0 Tags
5145e35e50ca7fa2bf381eaacdb30e234d774d1c
Commit Graph
1207 Commits
Author SHA1 Message Date
ahmed xijiaatOpenAI CodexNick HillYannik Hintereggermergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		9465e1c783 [Bugfix][NIXL] Recover locally invalidated pull peer metadata (#55471) 
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 18:46:53 -07:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		2537629eb2 [Bugfix][KVConnector][NIXL] Size per-region replicate flags by each region's block count (#60107) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-06 01:27:50 +00:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		0e8f8ecba8 [Bugfix][KVConnector][NIXL] Return NumPy descriptors from mixed-memory local registration (#60108) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-06 01:23:20 +00:00
Robert ShawClaude Opus 5.5Matthew Bonanni
 		2e3154aa18 [HiSparse] Log steady-state max concurrency and expose host-tier utilization gauges (#58949) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-05 14:20:32 -07:00
Maroon Ayoub b1401e0aa7 [Bugfix][NIXL] Count heartbeats as remote engine activity (#59873) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
 2026-10-05 09:36:13 +00:00
Elvir Crnčević ce49174247 [Bugfix] Fix FlashInfer all_reduce backend selection (#56891) 2026-10-03 22:18:58 -04:00
aoshen02andClaude Opus 5.5 5e56e9af2a [Feature] Release the CUDA graph pool on sleep (#59160) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 01:13:10 +00:00
Hexiang WangandCodex f03026a548 [Bugfix][MoRIIO] Keep discovery heartbeats running while workers hold the GIL (#59441) 
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-03 14:55:18 +00:00
Yicong Wang 5f30fc7031 [Bugfix][KV Connector][Mooncake] Suppress completion for empty pulls (#59347) 
Signed-off-by: wangyicong <wangyicong@bytedance.com>
 2026-10-03 15:28:00 +08:00
Yifan QiaoClaude Opus 5.5Codex
 		ca0df1f122 [Bugfix][Core] Exempt exactly the blocks an async KV load writes from zeroing (#59504) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-03 06:26:12 +00:00
DOCCA0andRobert Shaw 7dfe3338d5 [EPLB] Add contention-aware expert migration batching (#52641) 
Signed-off-by: DOCCA0 <2947323341@qq.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-03 00:13:50 +00:00
Ayaan Nadamal 84593955fa [Perf][MoE] Support fp8 combine in FlashInfer one-sided MoE all2all (#57995) 
Signed-off-by: Ayaan Nadamal <anadamal@nvidia.com>
 2026-10-02 16:29:16 -04:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
tenotekandClaude Fable 5.1 443dccd3c9 [Bugfix][Multimodal] Fix reference counting of the SHM processor cache (#59695) 
Signed-off-by: tenotek <50898317+hdq66666@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 22:14:13 +08:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		4056c8ac1f [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-01 23:04:21 -07:00
Guanxin Liaoshen524Claude Opus 5.5aoshen02
 		a810316c08 [Bugfix] Keep batch-invariance NCCL pins out of the weight-transfer group (#59500) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-02 11:19:28 +08:00
Hyunjae Wooandaoshen02 5a661487e2 [Feature] Add native ModelExpress weight transfer backend (#58399) 
Signed-off-by: Hyunjae Woo <hwoo@nvidia.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-02 02:29:35 +00:00
Flora Feng 785b990116 [Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT (#58623) 
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-02 02:25:48 +00:00
Matej SirovatkaLucas WilkinsonClaude Fable 5.1Matthew Bonanni
 		ca65eb67d9 [Perf][HiSparse] Avoid repeated prefix scans and residency updates (#57930) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-01 21:48:18 +00:00
Zijing Liu ea84051819 [KVConnector][NIXL] Count completion notifications that arrive after KV expiry (#58875) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
 2026-10-01 21:30:58 +00:00
Chanbin LimandChendi.Xue 47de9d4d04 [KV Connector][NIXL] Coalesce host-buffer KV copies across cache groups (#54483) 
Signed-off-by: beenpow <57864041+beenpow@users.noreply.github.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
 2026-10-01 13:10:54 -05:00
Jiangyun ZhuClaude Opus 5.5Thomas Parnell
 		cb6458c5ce [Core] Remove deprecated mamba_cache_mode "all" (#58997) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2026-10-01 02:59:37 -07:00
Robert ShawRobert ShawClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick Hill
 		e5e38ba9b7 [Model Runner V2] Support randomized dummy inputs (#58411) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-10-01 00:52:26 -07:00
Varun Sundar Rabindranath 5463fe4962 [KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA (#57652) 2026-09-30 09:11:03 -04:00
Clinton ThomasLucas BourtouleCyrus Leung
 		3d5625cc3c [Security] Authenticate shared-memory multimodal cache handles (#59357) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-30 05:16:26 -07:00
Clinton ThomasLucas BourtouleTai AnNick HillClaude Opus 5.5
 		765872e7ed [Core][BugFix] Tag prefix-cache extra keys by source (#51899) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Tai An <antai12232931@outlook.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 18:24:59 -07:00
Tyler Michael SmithandCodex 50239dfc4d [WideEP] Change DeepEPv2 to auto select hybrid mode by default (#57991) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-29 18:36:30 -04:00
StaryCursorYifan Qiao
 		faacc13565 [Perf][KV Connector][Mooncake] Pack hybrid/MLA KV into coalesced transfer regions (#57952) 
Signed-off-by: staryxchen <staryxchen@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-09-29 13:29:04 -07:00
YukioZzzandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 30ae8248b6 [KVConnector][MoRIIO] Support K3 DSpark hybrid READ (#57700) 
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-29 19:14:24 +00:00
Andreas KaratzasandOpenAI Codex 3acddf01e7 [Bugfix][ROCm] Keep Mooncake bootstrap ports bound during startup (#58967) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-29 04:14:26 -05:00
Tyler Michael SmithandCodex 4406874e7c [Bugfix][FT] Pass stateless process-group timeouts explicitly (#58611) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-29 03:14:02 +00:00
Almog Tavor 171d1b9ef6 [Elastic EP] Support Model Runner V2 (#53934) 
Signed-off-by: almogtavor <almogtavor@gmail.com>
Signed-off-by: Almog Tavor <70065337+almogtavor@users.noreply.github.com>
 2026-09-29 00:20:40 +00:00
Kevin H. LuuClaude Opus 5.5Lucas WilkinsonNick Hill
 		764413559a [Bugfix] Don't sync-police or retry FlashInfer all-reduce workspace creation in eager mode (#58498) 
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-28 12:14:21 -07:00
qizixi 6d3ea3c2c9 [KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill (#50499) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
 2026-09-28 18:50:46 +02:00
Chao Lei 386ac2573e [Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout (reopens #55763) (#58919) 2026-09-27 19:47:36 +00:00
YannikHintereggerandxijiade.aihemaiti fff03267da [Bugfix][NIXL] Release a dead peer's NIXL state without waiting for TTL (#50047) 
Signed-off-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com>
Co-authored-by: xijiade.aihemaiti <3146335281@qq.com>
 2026-09-26 20:30:09 -07:00
voidbruce.xuYifan QiaoOpenAI Codex
 		eb0f2ca37f [mooncake] support CUSTOM_MEM_POOL in vllm (#49300) 
Signed-off-by: bruce.xu <bruce.xb@alibaba-inc.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: bruce.xu <bruce.xb@alibaba-inc.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-27 00:26:05 +00:00
mevince cd1daf58a9 [Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector (#57251) 
Signed-off-by: Vincent <vincexxchan@gmail.com>
 2026-09-26 17:17:11 -07:00
Lucas WilkinsonClaude Opus 5Codex
 		927c87b347 [PCP][DCP] Support DCP target model with non-DCP Dspark (#56723) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-26 18:12:35 -04:00
Itay AlroyandTyler Michael Smith 2bb7605ae1 [Elastic EP] Fix EPLB load statistics during scaling (#58473) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-09-26 12:05:37 +00:00
Ke Wenmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>aoshen02
 		b6d2b4fc25 [RL] Add sharding-aware NCCL M2N weight transfer (#51520) 
Signed-off-by: Ke Wen <kwen@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-26 01:38:53 +00:00
GokayAIandGokayAI c89b5b4176 [Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head (#58292) 
Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com>
Co-authored-by: GokayAI <gokay-ai@users.noreply.github.com>
 2026-09-25 17:23:44 +00:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Yongye ZhuandSummer Yang 442bd00d9f [Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-09-25 03:27:21 +00:00
Yifan QiaoOpenAI CodexKimi Code
 		b3a87b2d04 [Bugfix][KVConnector] Finalize saves on steps without a forward (#57775) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-09-25 00:06:36 +00:00
Zijing LiuYifan QiaoCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7a9406690e [KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode (#58472) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 23:55:47 +00:00
stefankoncarevicandAndreas Karatzas 9ac3b7c69f [ROCm][CI] Mirror the DSv4-Flash disaggregated DP EP group on MI355 (#58558) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 18:38:25 -05:00
snadampalandNicolò Lucchesi 6a86bf626e [PD][PushConnector] Record last activity of remotes on the D side (#52245) 
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-09-24 23:15:32 +00:00
Vincent CavéShiksha PatelDouglas LehrAndreas Karatzas
 		92e9c6bf13 [ROCm] Fix misrouting race-condition in multi-decode P/D disagg with mori-io (#51681) 
Signed-off-by: Vincent Cave <vincent.cave@amd.com>
Signed-off-by: Shiksha Patel <shikpate@amd.com>
Co-authored-by: Shiksha Patel <shikpate@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 19:52:43 +00:00
Kapil AryaandOr Ozeri 70fc359d25 [Bugfix][KV Offload] Retain offload event metadata through batch translation (#57453) 
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapil.arya.17@gmail.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-09-24 07:09:12 +03:00