diff --git a/docs/features/kv_offloading_usage.md b/docs/features/kv_offloading_usage.md index 83c98c3d01..e2450c6625 100644 --- a/docs/features/kv_offloading_usage.md +++ b/docs/features/kv_offloading_usage.md @@ -152,10 +152,12 @@ Inside that subdirectory, blocks are sharded across hash-prefix subdirectories t #### Cross-Process Sharing -To enable KV cache sharing between multiple vLLM instances using the same `root_dir` (e.g., via a shared PVC), the `PYTHONHASHSEED` environment variable must be set to the same fixed value (e.g., `"0"`) on every instance. Without this, each process initializes `NONE_HASH` (the chain-hash seed for block content hashes) with random bytes, producing different block filenames for identical token content. +KV cache sharing between multiple vLLM instances using the same `root_dir` (e.g., via a shared PVC) works by default: `NONE_HASH` (the chain-hash seed for block content hashes) is derived from a fixed default seed, so identical token content produces identical block filenames across instances. To use a custom shared seed instead, set the `PYTHONHASHSEED` environment variable to the same value on every instance. + +The exception is the non-cryptographic `xxhash` and `xxhash_cbor` values of `--prefix-caching-hash-algo`, which seed `NONE_HASH` randomly per process so the seed stays unpredictable. Sharing a cache across instances with those algorithms requires setting the same `PYTHONHASHSEED` on every instance. ```bash -PYTHONHASHSEED=0 vllm serve ... +PYTHONHASHSEED= vllm serve ... ``` ### Object Store (OBJ) @@ -182,13 +184,13 @@ The object-store tier (`type: "obj"`) offloads blocks to an S3-compatible object | `region` | no | `""` | Bucket region, if the endpoint requires one. | | `ca_bundle` | no | `""` | CA bundle path for TLS verification. | -Object keys follow the same run-configuration digest scheme as the filesystem tier (see [On-Disk Layout](#on-disk-layout)) and are stored under the optional `prefix`. The [Cross-Process Sharing](#cross-process-sharing) requirement (`PYTHONHASHSEED`) applies to shared buckets as well, so instances sharing a bucket produce identical keys for identical content. At startup the tier probes object store connectivity and fails fast with a configuration error if the bucket is unreachable. +Object keys follow the same run-configuration digest scheme as the filesystem tier (see [On-Disk Layout](#on-disk-layout)) and are stored under the optional `prefix`. The [Cross-Process Sharing](#cross-process-sharing) behavior applies to shared buckets as well, so instances sharing a bucket produce identical keys for identical content; set a shared `PYTHONHASHSEED` if you want a custom seed. At startup the tier probes object store connectivity and fails fast with a configuration error if the bucket is unreachable. ### P2P (Including P/D) The P2P tier (`type: "p2p"`) shares completed KV blocks between vLLM instances over RDMA via NIXL. Each instance binds a control socket on `host:port` and exchanges blocks directly with peers — no shared filesystem required. -The `PYTHONHASHSEED` environment variable must be set to the same fixed value (e.g. `"0"`) on all nodes so that block content hashes match across instances (see [Cross-Process Sharing](#cross-process-sharing)). This is enforced: a P2P instance started without `PYTHONHASHSEED` set fails at startup, and each peer's value is verified during the connect handshake — a peer advertising a different `PYTHONHASHSEED` is rejected. +Block content hashes must match across instances for peers to exchange blocks (see [Cross-Process Sharing](#cross-process-sharing)). This works by default via the deterministic `NONE_HASH` seed, so setting `PYTHONHASHSEED` is optional. If you do set it, it must be the same value on all nodes. Each peer's effective seed is verified during the connect handshake — a peer advertising a different seed is rejected. With the `xxhash`/`xxhash_cbor` algorithms the seed is random per process, so `PYTHONHASHSEED` must be set on every peer or the handshake rejects them. | Key | Required | Default | Notes | | --- | --- | --- | --- | diff --git a/docs/features/mooncake_store_connector_usage.md b/docs/features/mooncake_store_connector_usage.md index e2470bf8fc..a07716dea6 100644 --- a/docs/features/mooncake_store_connector_usage.md +++ b/docs/features/mooncake_store_connector_usage.md @@ -235,10 +235,12 @@ Strict isolation requires a Mooncake master started with `--enable_multi_tenants ### Reproducible Block Hashes Across Processes -The `MooncakeStoreConnector` relies on consistent block hashes across all vLLM processes sharing the distributed store. Because Python randomizes its hash seed per process by default, identical prompts can produce different block hashes on different processes — preventing cross-process prefix cache hits. +The `MooncakeStoreConnector` relies on consistent block hashes across all vLLM processes sharing the distributed store. Block hashes chain from `NONE_HASH`, which is derived from a fixed default seed, so identical prompts produce identical block hashes across processes by default — enabling cross-process prefix cache hits without extra configuration. -Set a fixed `PYTHONHASHSEED` on every instance that shares the store (DP ranks, separate prefiller/decoder nodes, and any other vLLM process pointed at the same Mooncake store): +The exception is the non-cryptographic `xxhash`/`xxhash_cbor` values of `--prefix-caching-hash-algo`, which seed `NONE_HASH` randomly per process; sharing a store with those requires `PYTHONHASHSEED`. + +To use a custom shared seed, set the same `PYTHONHASHSEED` on every instance that shares the store (DP ranks, separate prefiller/decoder nodes, and any other vLLM process pointed at the same Mooncake store): ```bash -PYTHONHASHSEED=0 vllm serve ... +PYTHONHASHSEED= vllm serve ... ``` diff --git a/tests/v1/core/test_kv_cache_utils.py b/tests/v1/core/test_kv_cache_utils.py index bfe4996b1a..829f45608a 100644 --- a/tests/v1/core/test_kv_cache_utils.py +++ b/tests/v1/core/test_kv_cache_utils.py @@ -20,7 +20,7 @@ from vllm.multimodal.inputs import ( PlaceholderRange, ) from vllm.sampling_params import SamplingParams -from vllm.utils.hashing import sha256, sha256_cbor +from vllm.utils.hashing import sha256, sha256_cbor, xxhash, xxhash_cbor from vllm.utils.mem_constants import GiB_bytes from vllm.v1.core.kv_cache_manager import KVCacheManager from vllm.v1.core.kv_cache_utils import ( @@ -199,14 +199,20 @@ def new_mamba_spec( def test_none_hash(monkeypatch, hash_fn): import vllm.v1.core.kv_cache_utils - # case 1: PYTHONHASHSEED is not set, use random + # case 1: PYTHONHASHSEED is not set -> deterministic default seed so that + # independent processes compute identical block hashes for identical + # content (e.g. for KV cache reuse across nodes). with monkeypatch.context() as m: m.delenv("PYTHONHASHSEED", raising=False) reloaded_kv_cache_utils = importlib.reload(vllm.v1.core.kv_cache_utils) reloaded_kv_cache_utils.init_none_hash(hash_fn) - assert reloaded_kv_cache_utils.NONE_HASH is not None - assert isinstance(reloaded_kv_cache_utils.NONE_HASH, bytes) - assert reloaded_kv_cache_utils.NONE_HASH != b"" + none_hash = reloaded_kv_cache_utils.NONE_HASH + assert isinstance(none_hash, bytes) + assert none_hash != b"" + assert none_hash == hash_fn(reloaded_kv_cache_utils.DEFAULT_NONE_HASH_SEED) + # deterministic across re-initialization within the same environment + reloaded_kv_cache_utils.init_none_hash(hash_fn) + assert none_hash == reloaded_kv_cache_utils.NONE_HASH # case 2: PYTHONHASHSEED is set, use the seed and hash_fn with monkeypatch.context() as m: @@ -218,6 +224,58 @@ def test_none_hash(monkeypatch, hash_fn): assert hash_fn("python hash seed") == reloaded_kv_cache_utils.NONE_HASH +@pytest.mark.parametrize("non_crypto_fn", [xxhash, xxhash_cbor]) +def test_none_hash_seed_random_for_non_crypto(monkeypatch, non_crypto_fn): + """Non-cryptographic algorithms keep the per-process random seed. + + A deterministic seed is safe for SHA-256, whose collision resistance does + not depend on a secret, but xxHash is not collision resistant: a known seed + would let an attacker precompute colliding blocks offline. Keep the + unpredictable seed there unless the operator opts into a shared one. + """ + # PYTHONHASHSEED unset -> unpredictable, differs per resolution. + with monkeypatch.context() as m: + m.delenv("PYTHONHASHSEED", raising=False) + seeds = {kv_cache_utils.resolve_none_hash_seed(non_crypto_fn) for _ in range(5)} + assert len(seeds) == 5 + assert kv_cache_utils.DEFAULT_NONE_HASH_SEED not in seeds + + # PYTHONHASHSEED set -> operator opt-in wins, so peers can share a cache. + with monkeypatch.context() as m: + m.setenv("PYTHONHASHSEED", "12345") + assert kv_cache_utils.resolve_none_hash_seed(non_crypto_fn) == "12345" + + +@pytest.mark.parametrize("hash_fn", [sha256, sha256_cbor]) +def test_none_hash_seed_deterministic_for_crypto(monkeypatch, hash_fn): + with monkeypatch.context() as m: + m.delenv("PYTHONHASHSEED", raising=False) + seed = kv_cache_utils.resolve_none_hash_seed(hash_fn) + assert seed == kv_cache_utils.DEFAULT_NONE_HASH_SEED + assert seed == kv_cache_utils.resolve_none_hash_seed(hash_fn) + + +def test_get_none_hash_seed_reports_effective_seed(monkeypatch): + """P2P advertises the seed NONE_HASH was actually derived from. + + The P2P tier is constructed before init_none_hash runs, so it must read the + resolved seed lazily rather than re-deriving it. + """ + import vllm.v1.core.kv_cache_utils + + with monkeypatch.context() as m: + m.delenv("PYTHONHASHSEED", raising=False) + reloaded = importlib.reload(vllm.v1.core.kv_cache_utils) + reloaded.init_none_hash(sha256) + assert reloaded.get_none_hash_seed() == reloaded.DEFAULT_NONE_HASH_SEED + + with monkeypatch.context() as m: + m.setenv("PYTHONHASHSEED", "12345") + reloaded = importlib.reload(vllm.v1.core.kv_cache_utils) + reloaded.init_none_hash(sha256) + assert reloaded.get_none_hash_seed() == "12345" + + def test_kv_cache_block(): # Test KVCacheBlock initialization block = KVCacheBlock(block_id=0) diff --git a/tests/v1/kv_offload/tiering/p2p/test_manager.py b/tests/v1/kv_offload/tiering/p2p/test_manager.py index df7a352192..2cc9618303 100644 --- a/tests/v1/kv_offload/tiering/p2p/test_manager.py +++ b/tests/v1/kv_offload/tiering/p2p/test_manager.py @@ -15,6 +15,8 @@ from types import SimpleNamespace import numpy as np import pytest +from vllm.utils.hashing import sha256 +from vllm.v1.core.kv_cache_utils import DEFAULT_NONE_HASH_SEED, init_none_hash from vllm.v1.kv_offload.base import LookupResult, ReqContext, ScheduleEndContext from vllm.v1.kv_offload.tiering.base import JobResult, TransferJob from vllm.v1.kv_offload.tiering.p2p import manager as manager_module @@ -123,23 +125,12 @@ def _init_offloading_spec() -> SimpleNamespace: # --------------------------------------------------------------------------- -# Tests for __init__ PYTHONHASHSEED assertion +# Tests for __init__ hash seed resolution # --------------------------------------------------------------------------- -class TestInitHashSeedAssertion: - def test_missing_pythonhashseed_raises(self, monkeypatch): - """P2P instance refuses to start when PYTHONHASHSEED is unset.""" - monkeypatch.delenv("PYTHONHASHSEED", raising=False) - with pytest.raises(ValueError, match="PYTHONHASHSEED"): - P2PSecondaryTierManager( - offloading_spec=_init_offloading_spec(), - primary_kv_view=memoryview(bytearray(16)), - ) - - def test_pythonhashseed_set_succeeds(self, monkeypatch): - """With PYTHONHASHSEED set, __init__ records it for the handshake.""" - monkeypatch.setenv("PYTHONHASHSEED", "12345") +class TestInitHashSeed: + def _build(self, monkeypatch) -> P2PSecondaryTierManager: monkeypatch.setattr(manager_module, "NixlTransport", lambda *a, **k: object()) monkeypatch.setattr(manager_module, "ZmqTransport", lambda *a, **k: object()) monkeypatch.setattr( @@ -147,11 +138,39 @@ class TestInitHashSeedAssertion: "from_offloading_spec", lambda **k: SimpleNamespace(get_run_config=lambda: {}), ) - mgr = P2PSecondaryTierManager( + return P2PSecondaryTierManager( offloading_spec=_init_offloading_spec(), primary_kv_view=memoryview(bytearray(16)), ) - assert mgr._hash_seed == "12345" + + def test_missing_pythonhashseed_uses_default(self, monkeypatch): + """P2P falls back to the deterministic default seed when unset.""" + monkeypatch.delenv("PYTHONHASHSEED", raising=False) + mgr = self._build(monkeypatch) + init_none_hash(sha256) + assert mgr._get_hash_seed() == DEFAULT_NONE_HASH_SEED + + def test_pythonhashseed_set_succeeds(self, monkeypatch): + """With PYTHONHASHSEED set, the handshake advertises it.""" + monkeypatch.setenv("PYTHONHASHSEED", "12345") + mgr = self._build(monkeypatch) + init_none_hash(sha256) + assert mgr._get_hash_seed() == "12345" + + def test_seed_resolved_after_init_none_hash(self, monkeypatch): + """The seed is read lazily, not at construction time. + + This tier is built before init_none_hash runs, and a non-cryptographic + hash algorithm seeds NONE_HASH randomly, so resolving in __init__ would + advertise a value that does not match the NONE_HASH actually in use. + """ + monkeypatch.delenv("PYTHONHASHSEED", raising=False) + mgr = self._build(monkeypatch) + assert mgr._hash_seed is None + monkeypatch.setattr( + manager_module, "get_none_hash_seed", lambda: "random-seed-abc" + ) + assert mgr._get_hash_seed() == "random-seed-abc" # --------------------------------------------------------------------------- diff --git a/vllm/v1/core/kv_cache_utils.py b/vllm/v1/core/kv_cache_utils.py index d6e401d118..4ac55dbbf7 100644 --- a/vllm/v1/core/kv_cache_utils.py +++ b/vllm/v1/core/kv_cache_utils.py @@ -15,7 +15,7 @@ from typing import Any, NamedTuple, NewType, TypeAlias, cast, overload from vllm import envs from vllm.config import VllmConfig from vllm.logger import init_logger -from vllm.utils.hashing import sha256_cbor, xxhash_cbor +from vllm.utils.hashing import xxhash, xxhash_cbor from vllm.utils.math_utils import cdiv, round_up from vllm.utils.mem_utils import format_gib from vllm.utils.torch_utils import get_dtype_size @@ -87,32 +87,73 @@ logger = init_logger(__name__) # The hash seed for the first block of any prefix block sequence. # -# We use a random value to avoid hash collisions or PYTHONHASHSEED environment -# variable if set such that processes can share the seed if needed. This aligns -# with the behavior of Python's hash() function, which also uses a random seed -# if PYTHONHASHSEED is not set. +# For cryptographic hash algorithms it is derived deterministically from a fixed +# default seed, so independent vLLM processes compute identical block hashes for +# identical content and can share a prefix cache (e.g. KV cache reuse across +# nodes) without extra configuration. This does not weaken collision resistance, +# which for SHA-256 does not depend on keeping the seed secret; ``cache_salt`` +# remains the mechanism for intentional cache isolation. +# +# Non-cryptographic algorithms keep a per-process random seed, because a +# predictable seed would let an attacker precompute colliding blocks offline +# (see #12621). Setting PYTHONHASHSEED overrides the seed in both cases. # # The function `init_none_hash` initializes this variable globally. NONE_HASH: BlockHash -_CBOR_HASH_FUNCTIONS = frozenset({sha256_cbor, xxhash_cbor}) + +# Fixed seed used when the PYTHONHASHSEED environment variable is not set and +# the hash algorithm is cryptographic. +DEFAULT_NONE_HASH_SEED = "vllm-none-hash" + +# Algorithms that are not collision resistant, so the seed must stay secret. +_NON_CRYPTO_HASH_FUNCTIONS = frozenset({xxhash, xxhash_cbor}) + +# The seed NONE_HASH was derived from, set by init_none_hash. +_NONE_HASH_SEED: str | None = None + + +def resolve_none_hash_seed(hash_fn: Callable[[Any], bytes]) -> str: + """Resolve the seed to derive NONE_HASH from. + + PYTHONHASHSEED wins if set. Otherwise cryptographic algorithms get the + fixed default (shareable across processes) and non-cryptographic ones get + fresh random bytes, keeping the seed unpredictable where collision + resistance depends on it. + """ + hash_seed = os.getenv("PYTHONHASHSEED") + if hash_seed is not None: + return hash_seed + if hash_fn in _NON_CRYPTO_HASH_FUNCTIONS: + return os.urandom(32).hex() + return DEFAULT_NONE_HASH_SEED + + +def get_none_hash_seed() -> str: + """Return the seed NONE_HASH was derived from. + + Components that must agree on NONE_HASH across processes (the P2P tier + advertises this during its connect handshake) read the resolved seed here + instead of re-deriving it, so they observe the random seed too. Falls back + to the deterministic seed before ``init_none_hash`` has run. + """ + if _NONE_HASH_SEED is None: + return DEFAULT_NONE_HASH_SEED + return _NONE_HASH_SEED def init_none_hash(hash_fn: Callable[[Any], bytes]): - global NONE_HASH + global NONE_HASH, _NONE_HASH_SEED - hash_seed = os.getenv("PYTHONHASHSEED") - if hash_seed is None and hash_fn in _CBOR_HASH_FUNCTIONS: + _NONE_HASH_SEED = resolve_none_hash_seed(hash_fn) + if hash_fn in _NON_CRYPTO_HASH_FUNCTIONS and os.getenv("PYTHONHASHSEED") is None: logger.warning( - "PYTHONHASHSEED is not set. This will lead to non-reproducible " - "block-hashes when using CBOR-based hash functions such as " - "sha256_cbor or xxhash_cbor. Consider setting PYTHONHASHSEED to a " - "fixed value for reproducibility." + "Using a random per-process NONE_HASH seed because %s is not " + "collision resistant. Block hashes are therefore not reproducible " + "across processes; set PYTHONHASHSEED to a shared value to reuse " + "the prefix cache across instances, or use sha256.", + hash_fn.__name__, ) - - if hash_seed is None: - NONE_HASH = BlockHash(os.urandom(32)) - else: - NONE_HASH = BlockHash(hash_fn(hash_seed)) + NONE_HASH = BlockHash(hash_fn(_NONE_HASH_SEED)) @dataclass(slots=True) diff --git a/vllm/v1/kv_offload/tiering/fs/manager.py b/vllm/v1/kv_offload/tiering/fs/manager.py index dae0d8d500..6de2d07460 100644 --- a/vllm/v1/kv_offload/tiering/fs/manager.py +++ b/vllm/v1/kv_offload/tiering/fs/manager.py @@ -95,13 +95,14 @@ class FileSystemTierManager(SecondaryTierManager): get_finished_jobs() polls job completion and returns completed JobResults. Cross-process sharing: - In order to enable KV cache sharing between multiple vLLM instances - using the same ``root_dir`` (e.g., via a shared PVC) the environment - variable ``PYTHONHASHSEED`` must be set to the same fixed value - (e.g., "0") on all instances. Without this, each process initializes - ``NONE_HASH`` (the chain-hash seed for block content hashes) with - random bytes, producing different block filenames for identical token - content. + KV cache sharing between multiple vLLM instances using the same + ``root_dir`` (e.g., via a shared PVC) works by default: ``NONE_HASH`` + (the chain-hash seed for block content hashes) is derived from a fixed + default seed, so identical token content produces identical block + filenames across instances. Setting the ``PYTHONHASHSEED`` environment + variable to the same value on all instances overrides the default seed, + and is required to share a cache when using a non-cryptographic + prefix-caching hash algorithm, which seeds ``NONE_HASH`` randomly. """ medium: ClassVar[Medium] = Medium.STORAGE diff --git a/vllm/v1/kv_offload/tiering/p2p/manager.py b/vllm/v1/kv_offload/tiering/p2p/manager.py index f76ed59b5c..8050be0f4f 100644 --- a/vllm/v1/kv_offload/tiering/p2p/manager.py +++ b/vllm/v1/kv_offload/tiering/p2p/manager.py @@ -8,7 +8,6 @@ Owns transports and a single bidirectional P2PSession per remote peer. from __future__ import annotations -import os import time import uuid from collections.abc import Iterable, Sequence @@ -19,6 +18,7 @@ from typing_extensions import override import vllm.envs as envs from vllm.logger import init_logger +from vllm.v1.core.kv_cache_utils import get_none_hash_seed from vllm.v1.kv_offload.base import ( LookupResult, OffloadKey, @@ -247,20 +247,15 @@ class P2PSecondaryTierManager(SecondaryTierManager): **kwargs: Reserved for future tier-specific options. """ super().__init__(offloading_spec, primary_kv_view, tier_type) - # Block hashes chain from NONE_HASH, seeded from PYTHONHASHSEED - # (see init_none_hash in v1/core/kv_cache_utils.py). Peers with - # different seeds compute different hashes for identical content, so - # lookups silently miss and no KV crosses the wire. Require it here so - # a misconfigured P2P instance fails at startup rather than degrading - # silently; the value is also verified against each peer on handshake. - hash_seed = os.getenv("PYTHONHASHSEED") - if hash_seed is None: - raise ValueError( - "PYTHONHASHSEED must be set for P2P KV offload so that block " - "hashes match across instances. Set it to a fixed value (e.g. " - "PYTHONHASHSEED=0) on every P2P peer." - ) - self._hash_seed = hash_seed + # Block hashes chain from NONE_HASH (see v1/core/kv_cache_utils.py). + # Peers whose seeds differ compute different hashes for identical + # content, so lookups silently miss and no KV crosses the wire. The + # seed is advertised and verified against each peer during the + # handshake, so a mismatch is rejected loudly instead of degrading + # silently. Resolved lazily in _get_hash_seed: this tier is built + # before init_none_hash runs, and a non-cryptographic hash algorithm + # seeds NONE_HASH randomly, so the value is only known afterwards. + self._hash_seed: str | None = None if host is None: host = envs.VLLM_P2P_SIDE_CHANNEL_HOST if port is None: @@ -602,6 +597,12 @@ class P2PSecondaryTierManager(SecondaryTierManager): # Internal # ------------------------------------------------------------------ + def _get_hash_seed(self) -> str: + """The seed NONE_HASH was derived from, resolved on first session.""" + if self._hash_seed is None: + self._hash_seed = get_none_hash_seed() + return self._hash_seed + def _get_or_create_session(self, peer_id: str) -> P2PSession: """Return the existing session for peer_id, or open one outbound. @@ -621,7 +622,7 @@ class P2PSecondaryTierManager(SecondaryTierManager): local_id=self._local_id, transport=self._data, local_block_len=self._data.block_len, - local_hash_seed=self._hash_seed, + local_hash_seed=self._get_hash_seed(), conn=conn, ) self._sessions[peer_id] = session @@ -643,7 +644,7 @@ class P2PSecondaryTierManager(SecondaryTierManager): local_id=self._local_id, transport=self._data, local_block_len=self._data.block_len, - local_hash_seed=self._hash_seed, + local_hash_seed=self._get_hash_seed(), conn=conn, ) logger.info( diff --git a/vllm/v1/kv_offload/tiering/p2p/session/client.py b/vllm/v1/kv_offload/tiering/p2p/session/client.py index 07a4620780..66b838047b 100644 --- a/vllm/v1/kv_offload/tiering/p2p/session/client.py +++ b/vllm/v1/kv_offload/tiering/p2p/session/client.py @@ -312,8 +312,9 @@ class ClientRole: if st is not None and load is not None: logger.warning( "P2PSession %s: load request %s (job_id=%d) timed out; " - "load job completed with failure. If this recurs, ensure " - "PYTHONHASHSEED is set to the same value on all nodes.", + "load job completed with failure. If this recurs, ensure all " + "nodes use the same prefix-cache hash seed (matching " + "PYTHONHASHSEED, if set) and hash algorithm.", self._peer_id, kv_request_id, load.job_id, diff --git a/vllm/v1/kv_offload/tiering/p2p/session/protocol.py b/vllm/v1/kv_offload/tiering/p2p/session/protocol.py index 33ae5bf2f7..ec4153e25b 100644 --- a/vllm/v1/kv_offload/tiering/p2p/session/protocol.py +++ b/vllm/v1/kv_offload/tiering/p2p/session/protocol.py @@ -125,9 +125,10 @@ class ConnectMsg: BLOCK_LEN: Size in bytes of each block (must match between peers). CONFIG_FINGERPRINT: SHA-256 prefix of the model configuration. Peers with different fingerprints are incompatible. - HASH_SEED: The peer's PYTHONHASHSEED. Block hashes chain from a seed - derived from it, so peers with different values compute different - hashes for identical content and must not exchange blocks. + HASH_SEED: The peer's effective prefix-cache hash seed (PYTHONHASHSEED + if set, otherwise the built-in default). Block hashes chain from a + seed derived from it, so peers with different values compute + different hashes for identical content and must not exchange blocks. """ TYPE = "connect" diff --git a/vllm/v1/kv_offload/tiering/p2p/session/session.py b/vllm/v1/kv_offload/tiering/p2p/session/session.py index 4bd780159a..ba2aca6438 100644 --- a/vllm/v1/kv_offload/tiering/p2p/session/session.py +++ b/vllm/v1/kv_offload/tiering/p2p/session/session.py @@ -458,9 +458,10 @@ class P2PSession: ) if msg[ConnectMsg.HASH_SEED] != self._local_hash_seed: raise ValueError( - f"PYTHONHASHSEED mismatch from {self.peer_id}: " + f"hash seed mismatch from {self.peer_id}: " f"remote={msg[ConnectMsg.HASH_SEED]!r}, " - f"local={self._local_hash_seed!r}" + f"local={self._local_hash_seed!r}. Ensure PYTHONHASHSEED " + "(if set) matches on all P2P peers." ) self._transport.add_remote_peer( self.peer_id,