iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
16,996 Commits 3 Branches 0 Tags
5963c194787d30ed4a49c1e2e01010d8dffe1e79
Commit Graph
788 Commits
Author SHA1 Message Date
jatseng-ai 05c50c721e [ROCm] mori: add InterNodeV1LL inter-node kernel selection via VLLM_MORI_INTERNODE_KERNEL (#41751) 
Signed-off-by: jatseng-ai <jatseng@amd.com>
 2026-05-28 00:33:32 +08:00
aoshen02aoshen524Claude Opus 4.7
 		1fc2cee50a [KVConnector][Mooncake] Wire reset_cache cascade end-to-end (#42694) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 20:52:35 -07:00
Lucas WilkinsonMatthew BonanniNicolò Lucchesi
 		7e33081cee [Attention] Make FlexAttention and FlashAttention use num-blocks first layouts (#42095) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-26 19:55:56 -07:00
Zhewen LiZhewen LiClaude Opus 4.7
 		d98cbf472b [KV Connector] MooncakeStore: drop dead discard_partial_chunks parameter (#43627) 
Signed-off-by: Zhewen Li <zhewen@inferact.ai>
Co-authored-by: Zhewen Li <zhewen@inferact.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 13:40:21 -07:00
Simon DanielssonandTJian 681d7dd38b [Misc][Refactor][ROCm] Convert MoRI-related envvars to extra config args (#43303) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-05-26 03:33:35 -07:00
Ethan Feng 755043cf3c [KV Transfer] Enable HMA by default for connectors that support it (#41847) 
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
 2026-05-26 12:28:51 +02:00
Dao007forever c2a4005c70 [KV Connector] Propagate MooncakeStore load failures (#42788) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
 2026-05-25 22:12:15 -07:00
Dao007foreverandClaude 7966fc7233 [KV Connector][Bugfix] MooncakeStore: don't double-apply Eagle prune in load_mask (#43516) 
Signed-off-by: Dao Le <daole@inferact.ai>
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-25 22:11:57 -07:00
Zhewen LiandZhewen Li 873758c13a [KV Connector] Handle Mooncake finish after preemption (#43281) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
 2026-05-25 01:58:38 -07:00
Yifan Qiao 81252d4e24 [Feat][KVConnector] Support DSV4 in SimpleCPUOffloadBackend (#42296) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-05-25 14:04:30 +08:00
Or Ozeri 357fddf614 [kv_offload]: Add DSv4 support (#43142) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-05-24 11:10:12 +03:00
Dao007foreverandClaude 0902d8e62f [KV Connector] Keep MooncakeStore full hits block-aligned (#43494) 
Signed-off-by: Dao Le <daole@inferact.ai>
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-23 23:15:03 -07:00
Dao007forever 819c610f9b [Mooncake] Add metrics for MooncakeStoreConnector operations (#43392) 2026-05-23 13:34:40 -07:00
Sumanth R Hegdeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 3cb83c9592 Add model to WeightTransferEngine.__init__ (#42922) 
Signed-off-by: SumanthRH <sumanthrh99@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-22 17:52:15 -07:00
Itay Alroy 6d30655b13 elastic_ep: stage/commit MoE quant method on reconfigure (#40881) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-05-22 18:57:26 -04:00
Benjamin Chislettandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 4e2eba28be [Perf] Optimize hidden state extraction logic (#37374) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-22 18:23:08 -04:00
SandishKumarHNandSandishKumarHN 977703aa94 [RFC][EPLB][#32028] Remove dead torch.accelerator.synchronize() from sync path (#40733) 
Signed-off-by: SandishKumarHN <3078999+SandishKumarHN@users.noreply.github.com>
Co-authored-by: SandishKumarHN <3078999+SandishKumarHN@users.noreply.github.com>
 2026-05-22 15:19:24 -04:00
Dao007foreverandClaude Opus 4.7 b21f3d56d4 [KV Connector] MooncakeStore: don't co-queue save with load to avoid double delayed-free (#43371) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-22 16:14:11 +00:00
Ilya MarkovandMarkov Ilya d3a563501b [EPLB] Change default EPLB communicator (#43110) 
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
 2026-05-22 09:43:27 -04:00
Simon Danielsson 86ccef7d44 [ROCm] Add XGMI backend for MoRI Connector (#41753) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2026-05-22 03:06:40 +00:00
fangyuchuzWaNg3mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e26e1f0928 [Feature] Add --cpu-distributed-timeout-seconds CLI Option for CPU Process Group Timeout (#42968) 
Signed-off-by: fangyuchu <fangyuchu@qq.com>
Signed-off-by: zWaNg3 <389750525@qq.com>
Co-authored-by: zWaNg3 <389750525@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-21 15:42:07 -07:00
Jee Jee Li ee05e8137e [Minor] Bigger overlap for FI AR (#43103) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-20 21:20:57 -07:00
akii96 bde560ed6e [ROCm] Add QuickReduce min-size override and codec threshold (#41675) 
Signed-off-by: <>
 2026-05-20 17:46:51 -05:00
Vadim Gimpelson 9c78c99995 [MISC] Fix symm_mem cap-equal gate; log AR backend selection (#42993) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-05-20 08:50:24 -07:00
tomeras91 f54721bcc3 [Bugfix][MoE] FlashInfer one-sided: workspace union across heterogeneous layers (#42976) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
 2026-05-19 14:43:04 -04:00
Xinyu Chen d740e2c029 [XPU] update xpu graph usage (#43043) 
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
 2026-05-19 23:09:07 +08:00
Yifan Qiao 056bc2e166 [KVConnector][DSV4] HMA support for Mooncake store connector (#42828) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-05-19 01:07:46 -07:00
Revital Surandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> afd7b1dce9 [Bugfix] Use platform-agnostic device in example_connector load (#42926) 
Signed-off-by: Revital Sur <eres@il.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-19 03:12:04 +00:00
liranschourOr Ozerimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e5417657e5 [KV Connector][Offloading] Flush all pending jobs on last step (#42611) 
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: liranschour <liranschour@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-18 12:59:42 +00:00
Kfir Toledo e414e1f1c0 [Bugfix][KV Offload] count appended GPU blocks in store group_sizes (#42945) 
Signed-off-by: Kfir Toledo <kfir.toledo@ibm.com>
 2026-05-18 11:36:02 +00:00
Zhewen LiZhewen LiClaude Opus 4.7
 		36e74c9ea4 [KV Connector] Support disk offloading in MooncakeStoreConnector (#42689) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-16 13:34:15 -07:00
chunxiaozheng f45c210885 [LMCacheMPConnector] Prioritize importing the lmcache_mp_connector from lmcache (#42596) 
Signed-off-by: idellzheng <idellzheng@tencent.com>
 2026-05-15 17:46:31 +00:00
Aaron Hao e0a45f1455 [Feat][RL] IPC weight sync optimizations: multigpu support and chunked packed tensors (#37476) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: hao-aaron <ahao@anyscale.com>
 2026-05-15 22:53:06 +08:00
zhanqiuhuandNicolò Lucchesi 24337fb860 PD disagg with NIXL Connector: GDN support (Qwen3.5) (#41869) 
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-14 16:33:01 +02:00
Martin Hickeygemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Or Ozeri
 		5bd8c71e79 [kv_offload] Implement reset_cache() for the offloading connector (#41956) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-05-14 16:00:10 +03:00
Michael GoinandClaude Opus 4.6 2f821faeae [Spec Decode] Support hybrid attention models in extract_hidden_states (#39949) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-13 10:45:53 -07:00
Ronen Schaffer 79fd1bc7ed [kv_offload] Add req_id to ReqContext for per-request tracking (#42507) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-05-13 11:11:10 +00:00
Yifan Qiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 13bf242100 [Feat][KVConnector] Add bind_gpu_block_pool() to KVConnectorBase_V1 (#39654) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-13 02:10:29 -07:00
Nicolò Lucchesi 71bcd02ef3 [Bugfix][PD] Fix multi-node TP (TP>8) (#39907) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-12 22:20:57 -07:00
bnellnm 18f6bf5a21 [MoE Refactor] Add sequence parallel tests to test_moe_layer.py (#41299) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-05-12 21:52:19 -04:00
+3
Chao Leileichao.lcivaniumaoshen524Dao007foreverTeng MaPz1116foraxeSkywalker-EPfems14jianzsbaxingpiaochongmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ebeb09d822 [KV Transfer] Add MooncakeStoreConnector for KV cache offloading via Mooncake distributed store (#40900) 
Signed-off-by: leichao.lc <leichao.lc@antgroup.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: leichao.lc <leichao.lc@antgroup.com>
Co-authored-by: ivanium <yifanqiao@inferact.ai>
Co-authored-by: aoshen524 <aoshen@inferact.ai>
Co-authored-by: Dao007forever <daole@inferact.ai>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: Pz1116 <zpbzpb123123@gmail.com>
Co-authored-by: foraxe <1055696449@qq.com>
Co-authored-by: Skywalker-EP <173423846@qq.com>
Co-authored-by: fems14 <1804143737@qq.com>
Co-authored-by: jianzs <zheng.shoujian@outlook.com>
Co-authored-by: baxingpiaochong <771405853@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-12 16:09:10 -07:00
bnellnmandRobert Shaw d9b4990783 [MoE Refactor] EPLB refactoring for FusedMoE (#41055) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-05-12 14:16:31 -04:00
Yan Ru Pei bcb9c133ba feat(kv-events): emit KV cache metadata (#40984) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-05-12 15:58:48 +00:00
Martin Hickey 418ba8ef14 [kv_offload][BugFix] Fix store deferral (#41945) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-05-12 18:04:44 +03:00
zhanqiuhuandNickLucche 593d5a4033 [Bugfix] Fix mismatched kernel-per-logical blocks in NIXL HMA transfer (#42097) 
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
 2026-05-12 15:53:30 +02:00
pschlan-amd 39dff5ff39 Add VLLM_USE_SPINLOOP_EXT to use more efficient busy polling (#36517) 
Signed-off-by: Patrick Schlangen <pschlan@amd.com>
 2026-05-11 16:11:49 -07:00
Jeffrey WangandTyler Michael Smith 56e5810ff1 [BugFix] Prevent orphaned process on NCCL destroy (#39846) 
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2026-05-11 15:25:26 -04:00
Martin Hickey 8415bf2cdb [kv_offload] Set offloading connector to prefer HND layout (#41928) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-05-11 15:05:41 +03:00
Nicolò Lucchesi 5672d100ed [KV Connector][NIXL][Bugfix] Fix NIXL handshake failures not honoring kv_load_failure_policy (#40364) 
When NIXL handshake fails (e.g., due to compatibility hash mismatch
between prefill and decode instances), requests fail with "engine dead"
error instead of gracefully falling back to local recomputation as configured
by kv_load_failure_policy='recompute'.

Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-11 09:37:21 +00:00
Nicolò Lucchesi 770e9bd6b3 [Nixl][PD] Lease renewal TTL KV blocks on P (#41383) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-11 09:27:30 +00:00