[PCP][Spec Decode] Adds PCP support for single-module MTP and replicated DSpark. (#56107)
Signed-off-by: Summer Yang <girasoleyang@gmail.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Summer Yang <girasoleyang@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
co-authored by
Summer Yang
OpenAI Codex
OpenAI Codex
parent
e7edf17cea
commit
980c16c8e4
@@ -48,6 +48,7 @@ def _make_fake_speculator(
|
||||
)
|
||||
return SimpleNamespace(
|
||||
arange=torch.arange(max_num_reqs + 1, dtype=torch.int32, device="cpu"),
|
||||
draft_is_prefilling=torch.zeros(max_num_reqs, dtype=torch.bool),
|
||||
block_tables=fake_block_tables,
|
||||
input_buffers=fake_input_buffers,
|
||||
attn_groups=[],
|
||||
|
||||
Reference in New Issue
Block a user