[Doc] Fix group numbering in Case 3 of hybrid_kv_cache_manager.md (#52160)
Signed-off-by: qwerqwerqwe8688-jpg <xuuuuu2021@163.com>
This commit is contained in:
@@ -115,9 +115,10 @@ Unfortunately, not all models have such a beautiful ratio, and approach in Case
|
||||
- Group 0: 10 full attention layers (full.0 - full.9)
|
||||
- Group 1: 10 sliding window attention layers (sw.0 - sw.9)
|
||||
- Group 2: 10 sliding window attention layers (sw.10 - sw.19)
|
||||
- ...
|
||||
- Group 6: 10 sliding window attention layers (sw.40 - sw.49)
|
||||
- Group 7: 2 sliding window attention layers (sw.50 - sw.51) and 8 padding layers
|
||||
- Group 3: 10 sliding window attention layers (sw.20 - sw.29)
|
||||
- Group 4: 10 sliding window attention layers (sw.30 - sw.39)
|
||||
- Group 5: 10 sliding window attention layers (sw.40 - sw.49)
|
||||
- Group 6: 2 sliding window attention layers (sw.50 - sw.51) and 8 padding layers
|
||||
|
||||
We will update this algorithm if this heuristic leads to a bad result when a new model comes out (e.g., 20 full + 30 sw, the group size should be 10 instead of 20).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user