iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[KVConnector] Guard lmcache_mp_connector state transition with num_external_tokens (#47505)

Browse Source 
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: Alex-ai-future <Alex-ai-future@users.noreply.github.com>
This commit is contained in:
AlexHuang
2026-09-07 08:19:07 +00:00
committed by GitHub
parent 5893426b88
commit 49eb2accf0
1 changed files with 6 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vllm/distributed/kv_transfer/kv_connector/v1/lmcache_mp_connector.py
+6 -2
View File
@@ -835,8 +835,12 @@ class LMCacheMPConnectorUpstream(KVConnectorBase_V1):
if new_block_ids:
tracker.append_block_ids(new_block_ids)
# Update the state of the tracker
condition = tracker.needs_retrieve()
# num_external_tokens > 0 means this connector is chosen for loading.
# When non-chosen (num_external_tokens == 0, e.g. in MultiConnector),
# force condition=False so the state goes to READY and all lookup locks
# are released — end_session does NOT release lookup locks, so without
# this they would leak until TTL expiry (10 minutes).
condition = tracker.needs_retrieve() and num_external_tokens > 0
if tracker.state == LMCacheMPRequestState.PREFETCHING:
# If need to retrieve, change to WAITING_FOR_LOAD
# Otherwise, change to READY