[Perf][PCP] Shard decode requests across PCP ranks (#52162)

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
This commit is contained in:
Qiu Chunshuo
2026-10-01 23:50:13 -04:00
committed by GitHub
co-authored by Lucas Wilkinson
parent a810316c08
commit 8486f1ebfe
13 changed files with 406 additions and 33 deletions
@@ -205,6 +205,7 @@ def test_short_prefill_updates_k_cache_before_scoring_decision(
topk_indices,
False,
False,
False,
MLA_LAYER,
)
@@ -259,6 +260,7 @@ def test_skipped_k_cache_insert_accepts_no_k(
topk_indices,
True,
False,
False,
"",
)