[Perf][PCP] Shard decode requests across PCP ranks (#52162)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
This commit is contained in:
co-authored by
Lucas Wilkinson
parent
a810316c08
commit
8486f1ebfe
@@ -205,6 +205,7 @@ def test_short_prefill_updates_k_cache_before_scoring_decision(
|
||||
topk_indices,
|
||||
False,
|
||||
False,
|
||||
False,
|
||||
MLA_LAYER,
|
||||
)
|
||||
|
||||
@@ -259,6 +260,7 @@ def test_skipped_k_cache_insert_accepts_no_k(
|
||||
topk_indices,
|
||||
True,
|
||||
False,
|
||||
False,
|
||||
"",
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user