[Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
Matthew Bonanni
parent
0cbac6cd13
commit
4056c8ac1f
@@ -133,7 +133,10 @@ duplicate mapping.
|
||||
Speculative decoding resolves all verification rows of a request in one pass:
|
||||
one block resolves the union of the rows' top-k against the request's hot-cache
|
||||
state, so rows that select the same host row share its hot row and no row
|
||||
evicts a hot row another row of the step still reads.
|
||||
evicts a hot row another row of the step still reads. Draft layers write their
|
||||
rows after the target forward, so their host mirror runs at the start of the
|
||||
next step, after the drafter, and the step's page transfers are submitted
|
||||
behind it.
|
||||
|
||||
## P/D import target
|
||||
|
||||
|
||||
Reference in New Issue
Block a user