From d3fe85713508a69144ec610aa13253de34896338 Mon Sep 17 00:00:00 2001 From: Yan Ma Date: Mon, 23 Mar 2026 13:19:03 +0800 Subject: [PATCH] update doc for online fp8 quantization (#37851) Signed-off-by: Yan Ma --- docs/features/quantization/fp8.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/features/quantization/fp8.md b/docs/features/quantization/fp8.md index fd57e2063a..2165161ce8 100644 --- a/docs/features/quantization/fp8.md +++ b/docs/features/quantization/fp8.md @@ -137,6 +137,3 @@ llm = LLM("facebook/opt-125m", quantization="fp8") result = llm.generate("Hello, my name is") print(result[0].outputs[0].text) ``` - -!!! warning - Currently, we load the model at original precision before quantizing down to 8-bits, so you need enough memory to load the whole model.