[CPU][s390x] Optimize inference perf and add oneDNN INT8 GEMM for s390x (#50219)
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com>
This commit is contained in:
co-authored by
Li, Jiang <jiang1.li@intel.com>
parent
482cfc20f0
commit
88bc8fb56f
@@ -315,7 +315,7 @@ vLLM CPU supports data parallel (DP), tensor parallel (TP) and pipeline parallel
|
||||
- vLLM CPU supports quantizations:
|
||||
- AWQ (x86 only)
|
||||
- GPTQ (x86 only)
|
||||
- compressed-tensor INT8 W8A8 (x86 only)
|
||||
- compressed-tensor INT8 W8A8 (x86, s390x only)
|
||||
|
||||
### Why do I see `get_mempolicy: Operation not permitted` when running in Docker?
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ Currently, the CPU implementation for s390x architecture supports FP32, BF16 and
|
||||
|
||||
- OS: `Linux`
|
||||
- SDK: `gcc/g++ >= 14.0.0` or later with Command Line Tools
|
||||
- Instruction Set Architecture (ISA): VXE support is required. Works with Z14 and above.
|
||||
- Instruction Set Architecture (ISA): VXE support is required. Works with Z15 and above.
|
||||
- Build from source python packages (no pre-built s390x wheels): `torchvision`, `llvmlite`, `numba`, `opencv-python-headless`, `hf-xet`
|
||||
|
||||
--8<-- [end:requirements]
|
||||
|
||||
Reference in New Issue
Block a user