[Spec Decode] Rename AR speculators to StandaloneAR / TargetDependentAR (#60335)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
This commit is contained in:
@@ -820,7 +820,7 @@ class GPUModelRunner(LoRAModelRunnerMixin):
|
||||
"skip_attn must only be True for initial memory profiling."
|
||||
)
|
||||
|
||||
# Create a dummy scheduler output. Plain draft-model speculation adds
|
||||
# Create a dummy scheduler output. Standalone AR speculation adds
|
||||
# one correction slot per request during prefill. The scheduler
|
||||
# accounts for these slots, while dummy runs bypass the scheduler.
|
||||
# Adjust the dummy token count only when the expanded draft batch
|
||||
|
||||
Reference in New Issue
Block a user