[Core/Bugfix] Add query dtype as per FlashInfer API requirements. (#8173)

This commit is contained in:
Elfie Guo
2024-09-05 05:12:26 +00:00
committed by GitHub
parent ba262c4e5a
commit e39ebf5cf5
2 changed files with 10 additions and 2 deletions
+2 -1
View File
@@ -445,7 +445,8 @@ def test_flashinfer_decode_with_paged_fp8_kv(
head_size,
block_size,
"NONE",
data_type=dtype)
data_type=dtype,
q_data_type=dtype)
output = wrapper.forward(query,
kv_cache_fp8,
logits_soft_cap=soft_cap,