[Bugfix][Elastic EP] Route only to surviving engines during scale-down (#55772)

Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
This commit is contained in:
djramic
2026-09-08 21:28:28 -05:00
committed by GitHub
co-authored by Andreas Karatzas
parent 21a117ad87
commit c6ba5e89d6
2 changed files with 62 additions and 0 deletions
+2
View File
@@ -1914,6 +1914,8 @@ class DPLBAsyncMPClient(DPAsyncMPClient):
# NOTE(yongji): Immediately stop sending requests to the removing engines.
self.core_engines = old_core_engines[:new_data_parallel_size]
self.lb_engines = self.lb_engines[:new_data_parallel_size]
# Pending coordinator snapshots must use the surviving ranks too.
self.engine_ranks_managed = self.engine_ranks_managed[:new_data_parallel_size]
removed_dp_size = cur_data_parallel_size - new_data_parallel_size
pause_modes = ["keep"] * new_data_parallel_size + ["abort"] * removed_dp_size
pause_futures = [