[ASR] Optimize CPU preproc to get 2.5x RTFx via multi-threading (#44612)

Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Ekagra Ranjan
2026-06-11 21:05:11 -07:00
committed by GitHub
co-authored by mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent 42ae5e7ac6
commit e0b9fb1290
4 changed files with 105 additions and 34 deletions
@@ -474,6 +474,13 @@ async def lifespan(app: FastAPI):
finally:
if task is not None:
task.cancel()
for attr_name in (
"openai_serving_transcription",
"openai_serving_translation",
):
serving = getattr(app.state, attr_name, None)
if serving is not None and hasattr(serving, "shutdown"):
serving.shutdown()
finally:
# Ensure app state including engine ref is gc'd
del app.state