[Kernel][Perf] Support Qwen head ratios in fused GDN MTP (#52539)
Signed-off-by: kiroxu <148877251+BabyDrangoner@users.noreply.github.com>
This commit is contained in:
@@ -1805,7 +1805,8 @@ class QwenGatedDeltaNetAttention(GatedDeltaNetAttention):
|
||||
and attn_metadata.num_spec_decodes > 0
|
||||
and self.kv_cache[1].dtype in FUSED_GDN_STATE_DTYPES
|
||||
and self.gdn_decode_kernel == "cuda"
|
||||
and self.num_v_heads == 8 * self.num_k_heads
|
||||
and self.num_v_heads % self.num_k_heads == 0
|
||||
and self.num_v_heads // self.num_k_heads in (1, 2, 3, 4, 8)
|
||||
and state_indices is not None
|
||||
and state_indices.size(1) <= MAX_FUSED_GDN_MTP_TOKENS
|
||||
and hasattr(torch.ops._C, "fused_gdn_decode_post_conv_mtp")
|
||||
|
||||
Reference in New Issue
Block a user