[Migration] Migrate bitsandbytes support to OOT plugin (#43529)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> Signed-off-by: Isotr0py <Isotr0py@outlook.com>
This commit is contained in:
@@ -5,6 +5,7 @@ This example shows how to use LoRA with different quantization techniques
|
||||
for offline inference.
|
||||
|
||||
Requires HuggingFace credentials for access.
|
||||
The bitsandbytes example also requires installing vllm-bnb-plugin.
|
||||
"""
|
||||
|
||||
import gc
|
||||
|
||||
Reference in New Issue
Block a user