iHore/vLLM - vLLM - iHore Software Repository

2 Commits

Author SHA1 Message Date

Cheng Li c5e8330997 [Bugfix] Fix null modules_to_not_convert in FBGEMM Fp8 quantization (#6665 ) 2024-07-22 19:25:05 -07:00

Author	SHA1	Message	Date
Cheng Li	c5e8330997	[Bugfix] Fix null `modules_to_not_convert` in FBGEMM Fp8 quantization (#6665 )	2024-07-22 19:25:05 -07:00
Cheng Li	e69ded7d1c	[Bug Fix] Fix the support check for FP8 CUTLASS (#5352 ) Bug description: With torch 2.4.0.dev20240603+cu121, cutlass_fp8_supported outputs False, and the (capability, version) before the comparison is (90, 11111111112) This PR fixes the support check for FP8 CUTLASS ( cutlass_fp8_supported) which was introduced in https://github.com/vllm-project/vllm/pull/5183.	2024-06-08 00:42:05 +00:00

Cheng Li

e69ded7d1c

[Bug Fix] Fix the support check for FP8 CUTLASS (#5352 )

Bug description:
With torch 2.4.0.dev20240603+cu121,
cutlass_fp8_supported outputs False, and the (capability, version) before the comparison is (90, 11111111112)

This PR fixes the support check for FP8 CUTLASS ( cutlass_fp8_supported) which was introduced in https://github.com/vllm-project/vllm/pull/5183.

2024-06-08 00:42:05 +00:00