[Bugfix] Don't reuse engine core payload buffer while zmq is sending it (#50053)
Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com>
This commit is contained in:
co-authored by
Andreas Karatzas
parent
4fb483ca86
commit
6453fc0b8c
+1
-1
@@ -1547,7 +1547,7 @@ environment_variables: dict[str, Callable[[], Any]] = {
|
||||
# tensors above will instead be sent via a separate message.
|
||||
# While the sending side still actually copies the tensor
|
||||
# in all cases, on the receiving side, tensors above this
|
||||
# limit will actually be zero-copy decoded.
|
||||
# limit will actually be zero-copy decoded. The unit is bytes.
|
||||
"VLLM_MSGPACK_ZERO_COPY_THRESHOLD": lambda: int(
|
||||
os.getenv("VLLM_MSGPACK_ZERO_COPY_THRESHOLD", "256")
|
||||
),
|
||||
|
||||
Reference in New Issue
Block a user