Update Flashinfer to 0.2.14.post1 (#23537)
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com> Signed-off-by: siyuanf <siyuanf@nvidia.com> Signed-off-by: Weiliang Liu <weiliangl@nvidia.com> Signed-off-by: Michael Goin <mgoin64@gmail.com> Co-authored-by: Siyuan Fu <siyuanf@nvidia.com> Co-authored-by: Michael Goin <mgoin64@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Siyuan Fu
Michael Goin
gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
parent
906e461ed6
commit
ae067888d6
@@ -465,7 +465,8 @@ if flashinfer_comm is not None:
|
||||
quant_out=quant_out,
|
||||
scale_out=scale_out,
|
||||
# in vllm we only support swizzled layout
|
||||
layout_code=flashinfer_comm.FP4QuantizationSFLayout.SWIZZLED,
|
||||
layout_code=flashinfer_comm.QuantizationSFLayout.
|
||||
SWIZZLED_128x4,
|
||||
scale_factor=scale_factor,
|
||||
)
|
||||
else:
|
||||
|
||||
Reference in New Issue
Block a user