[Hardware][Power] Enable W4A16 (AWQ & GPTQ) quantization on POWER10 using VSX (#59149)

Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
This commit is contained in:
Rukhaiya2004
2026-09-30 19:44:48 +08:00
committed by GitHub
co-authored by Li, Jiang <jiang1.li@intel.com>
parent 863475eb9c
commit 72e7874fa6
7 changed files with 99 additions and 9 deletions
+2 -2
View File
@@ -312,8 +312,8 @@ vLLM CPU supports data parallel (DP), tensor parallel (TP) and pipeline parallel
### Which quantization configs does vLLM CPU support?
- vLLM CPU supports quantizations:
- AWQ (x86, s390x)
- GPTQ (x86, s390x)
- AWQ (x86, s390x, Power)
- GPTQ (x86, s390x, Power)
- compressed-tensor INT8 W8A8 (x86, s390x only)
### Why do I see `get_mempolicy: Operation not permitted` when running in Docker?