[V1][DP][LB] Publish request counts at the start of each engine step (#41626)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com> Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
co-authored by
Nick Hill
parent
24337fb860
commit
f60c6b33a5
@@ -1817,6 +1817,8 @@ class DPEngineCoreProc(EngineCoreProc):
|
||||
while self._handle_shutdown():
|
||||
# 1) Poll the input queue until there is work to do.
|
||||
self._process_input_queue()
|
||||
# Publish request counts before and after GPU step to ensure freshness.
|
||||
self._maybe_publish_request_counts()
|
||||
|
||||
if self.eep_scaling_state is not None:
|
||||
_ = self.eep_scaling_state.progress()
|
||||
|
||||
Reference in New Issue
Block a user