[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: LopezCastroRoberto <rocastro@redhat.com> Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
co-authored by
Lucas Wilkinson
OpenAI Codex
parent
e19a3e172e
commit
aed894c190
@@ -175,6 +175,15 @@ class Glm5NextLinearAttention(GatedDeltaNetAttention):
|
||||
projection_size = self.head_dim * self.num_heads
|
||||
self.local_projection_size = divide(projection_size, self.tp_size)
|
||||
|
||||
_, recurrent_state_shape = self.get_state_shape()
|
||||
_, recurrent_state_dtype = self.get_state_dtype()
|
||||
scatter_states.register_warmup(
|
||||
state_shape=recurrent_state_shape,
|
||||
state_dtype=recurrent_state_dtype,
|
||||
indices_dtype=torch.int32,
|
||||
max_num_tokens=vllm_config.scheduler_config.max_num_seqs,
|
||||
)
|
||||
|
||||
# Merge q, k, v, b, f_a, g_a projections into one GEMM (6→1 launches).
|
||||
# Order matches checkpoint's fused_qkvbfg_a_proj convention.
|
||||
# Shards 4 (f_a) and 5 (g_a) are replicated across TP ranks.
|
||||
|
||||
Reference in New Issue
Block a user