[Bugfix] GLM-5.3-Flash: launch the kpool paged MQA logits in the varlen mode its schedule was built with (#55270)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
064bc747ab
commit
39724758e7
@@ -558,6 +558,7 @@ def sparse_attn_indexer_kpool(
|
||||
decode_metadata.schedule_metadata,
|
||||
max_model_len=max_pool_len,
|
||||
clean_logits=False,
|
||||
indices=decode_metadata.indices,
|
||||
)
|
||||
num_rows = logits.shape[0]
|
||||
# kpool: logits are pool-granular -> select topk_tokens//kpool pools,
|
||||
|
||||
Reference in New Issue
Block a user