Fix an endless loop issue when engine_step throws a RuntimeError (#339)

This commit is contained in:
coolcloudcol
2023-07-03 15:22:28 -07:00
committed by GitHub
parent 42e0c1df78
commit 7717d0838b
+5 -1
View File
@@ -144,7 +144,11 @@ class AsyncLLMEngine:
# Kick the engine if the engine is not running.
if not self.is_engine_running:
await self.engine_step(request_id)
try:
await self.engine_step(request_id)
except RuntimeError as e:
await self.abort(request_id)
raise e
# Wait for new output. The group_event will be set in engine_step
# when there is new output available for the sequence group.