[Elastic EP] Async preparation (#47288)

Signed-off-by: Itay Alroy <ialroy@nvidia.com>
This commit is contained in:
Itay Alroy
2026-07-28 05:19:13 -07:00
committed by GitHub
parent d2bfc6fe20
commit 35efdf6b34
21 changed files with 701 additions and 677 deletions
+3
View File
@@ -376,6 +376,9 @@ class DPCoordinatorProc:
eng_index = outputs.engine_index
scheduler_stats = outputs.scheduler_stats
if scheduler_stats:
# Elastic EP stats may arrive while the engine list changes.
if eng_index >= len(self.engines):
continue
# 1. Updated request load stats - update our local
# state with these.
stats = self.engines[eng_index].request_counts