[V1] [P/D] Add Support for KV Load Failure Recovery (#19330)

Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
This commit is contained in:
David Ben-David
2025-09-30 14:57:08 -07:00
committed by GitHub
co-authored by David Ben-David
parent 67f3fb0844
commit 9a9f48dff7
24 changed files with 1039 additions and 86 deletions
+1
View File
@@ -250,6 +250,7 @@ def test_update_states_request_resumed(model_runner, dist_init):
new_token_ids=[[]],
new_block_ids=([[0]], ),
num_computed_tokens=[0],
num_output_tokens=[0],
)
scheduler_output = SchedulerOutput(