attach_deferred_state raised 'deferred model state for a step without
sampled outputs' when a non-last PP rank produced a PLE deferred state but
receive() left the step slot None (no row samples). With num_running_reqs
> 1 a prefill-only step appears (a 2048-token chunk for one request,
num_output_tokens=0, scheduled_spec_decode_tokens={}), and the raise
killed the engine every time. Confirmed in the container log: the same
PLE fatal at sample_tokens on sequence 101, 1766, 3150, 3264.
The deferred state has no consumer for a None slot (get_prev_sampled_outputs
returns None) and references the KV cache it was built for, so drop it to
release the reference. This is a prefill-only step under concurrent load,
not a fault.
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
MOVZXandHinata
50dabc9989