[Frontend][Core] Re-add shutdown timeout - allowing in-flight requests to finish (#36666)

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
Mark McLoughlin
2026-03-13 12:10:06 -07:00
committed by GitHub
co-authored by Claude Sonnet 4.5 Nick Hill
parent 5a3f1eb62f
commit 7afe0faab1
14 changed files with 762 additions and 96 deletions
+5
View File
@@ -200,6 +200,11 @@ class EngineClient(ABC):
"""Return whether the engine is currently paused."""
...
@abstractmethod
def shutdown(self, timeout: float | None = None) -> None:
"""Shutdown the engine with optional timeout."""
...
async def scale_elastic_ep(
self, new_data_parallel_size: int, drain_timeout: int = 300
) -> None: