[Feat][DSV4] Fuse q pad into deepseek v4 fused kernel (#43162)
This commit is contained in:
@@ -8,7 +8,7 @@ the existing separate operations (inverse RoPE via rotate_neox + FP8 quant
|
||||
via per_token_group_quant_fp8).
|
||||
|
||||
The reference faithfully reproduces the exact flow in
|
||||
deepseek_v4/nvidia/ops/attention.py:295-310:
|
||||
deepseek_v4/attention.py:295-310:
|
||||
1. Apply inverse RoPE (NeoX style, last rope_dim=64 dims of each head)
|
||||
2. Reshape [T, H, head_dim] -> [T, G, D]
|
||||
3. Transpose+flatten to [G*T, D], quantize, reshape back
|
||||
@@ -668,7 +668,7 @@ def _unfused_inv_rope_fp8_quant(
|
||||
nope_dim: int = NOPE_DIM,
|
||||
rope_dim: int = ROPE_DIM,
|
||||
) -> tuple[torch.Tensor, torch.Tensor]:
|
||||
"""Unfused path matching deepseek_v4/nvidia/ops/attention.py:295-310.
|
||||
"""Unfused path matching deepseek_v4/attention.py:295-310.
|
||||
|
||||
Uses the production CUDA RoPE kernel + per_token_group_quant_fp8.
|
||||
"""
|
||||
|
||||
Reference in New Issue
Block a user