[Elastic EP] Support Model Runner V2 (#53934)
Signed-off-by: almogtavor <almogtavor@gmail.com> Signed-off-by: Almog Tavor <70065337+almogtavor@users.noreply.github.com>
This commit is contained in:
@@ -224,7 +224,9 @@ def test_speculator_capture_preserves_decode_query_bounds(
|
||||
)
|
||||
manager = SpeculatorCudaGraphManager.__new__(SpeculatorCudaGraphManager)
|
||||
manager.max_num_reqs = num_reqs
|
||||
manager.dp_size = 1
|
||||
manager.vllm_config = SimpleNamespace(
|
||||
parallel_config=SimpleNamespace(data_parallel_size=1)
|
||||
)
|
||||
buffers = InputBuffers(num_reqs, num_tokens, torch.device("cpu"))
|
||||
block_tables = MagicMock()
|
||||
block_tables.cp_size = 1
|
||||
|
||||
Reference in New Issue
Block a user