[ROCm][Bugfix] Sanitize AITER paged-MQA logits before sparse top-k for DeepSeek-V4 (#49714)
Signed-off-by: shen-shanshan <467638484@qq.com>
This commit is contained in:
@@ -442,6 +442,7 @@ def rocm_fp8_paged_mqa_logits(
|
||||
KVBlockSize=block_size,
|
||||
WavePerEU=2,
|
||||
)
|
||||
out_logits.nan_to_num_(float("-inf"))
|
||||
return out_logits
|
||||
deepgemm_fp8_paged_mqa_logits_stage1 = (
|
||||
aiter_paged_mqa_logits_module.deepgemm_fp8_paged_mqa_logits_stage1
|
||||
|
||||
Reference in New Issue
Block a user