iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Bugfix][Docs] Package glm5next nvidia subtree and fix its docstrings (#55214)

Browse Source 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Harry Mellor
2026-09-03 21:25:26 -07:00
committed by GitHub
co-authored by Claude Opus 5
parent e35298628f
commit 78300cdabf
4 changed files with 12 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vllm/models/glm5next/amd/ops/kpool_compress.py
+4 -2
View File
@@ -631,9 +631,11 @@ def kpool_decode_update_and_maybe_write_cache_batched(
tail_kv_cache: paged tail cache ``[num_blocks, 2, pool_size, head_dim]``
bf16 (K at half 0, gate score at half 1).
tail_slot_mapping: ``[num_requests, next_n]`` int32.
key / slot_score: ``[num_requests, next_n, head_dim]`` bf16.
key: ``[num_requests, next_n, head_dim]`` bf16.
slot_score: ``[num_requests, next_n, head_dim]`` bf16.
ape: ``[pool_size, head_dim]`` fp32.
slot_mapping / positions: ``[num_requests, next_n]`` int32.
slot_mapping: ``[num_requests, next_n]`` int32.
positions: ``[num_requests, next_n]`` int32.
"""
num_requests, next_n = key.shape[0], key.shape[1]
if num_requests == 0 or next_n == 0:
vllm/models/glm5next/nvidia/__init__.py
+2
View File
@@ -0,0 +1,2 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
vllm/models/glm5next/nvidia/ops/__init__.py
+2
View File
@@ -0,0 +1,2 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
vllm/models/glm5next/nvidia/ops/kpool_compress.py
+4 -2
View File
@@ -637,9 +637,11 @@ def kpool_decode_update_and_maybe_write_cache_batched(
tail_kv_cache: paged tail cache ``[num_blocks, 2, pool_size, head_dim]``
bf16 (K at half 0, gate score at half 1).
tail_slot_mapping: ``[num_requests, next_n]`` int32.
key / slot_score: ``[num_requests, next_n, head_dim]`` bf16.
key: ``[num_requests, next_n, head_dim]`` bf16.
slot_score: ``[num_requests, next_n, head_dim]`` bf16.
ape: ``[pool_size, head_dim]`` fp32.
slot_mapping / positions: ``[num_requests, next_n]`` int32.
slot_mapping: ``[num_requests, next_n]`` int32.
positions: ``[num_requests, next_n]`` int32.
"""
num_requests, next_n = key.shape[0], key.shape[1]
if num_requests == 0 or next_n == 0: