7 Commits

Include renames

Author	SHA1	Message	Date
Kunshang JiandJiang Li	96b6f475dd	Remove hardcoded device="cuda" to support more devices (#2503) ... Co-authored-by: Jiang Li <jiang1.li@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>	2024-02-01 15:46:39 -08:00
Casper	beb89f68b4	AWQ: Up to 2.66x higher throughput (#2566)	2024-01-26 23:53:17 -08:00
CHU Tianxiang	0fbfc4b81b	Add GPTQ support (#916)	2023-12-15 03:04:22 -08:00
Yanming W	e0c6f556e8	[Build] Avoid building too many extensions (#1624)	2023-11-23 16:31:19 -08:00
Simon Mo	5ffc0d13a2	Migrate linter from pylint to ruff (#1665)	2023-11-20 11:58:01 -08:00
Woosuk Kwon	8d17774f92	Add AWQ support for all models (#1714)	2023-11-18 17:56:47 -08:00
Zhuohan Li	7076fa1c9f	TP/quantization/weight loading refactor part 2 - Refactor quantized linear logic and extend quantization support to all models (#1622) ... Refactor the tensor parallelism, quantization, and weight-loading codes. Summary of the new features enabled by this PR: - **All models** are able to be quantized with AWQ and SqueezeLLM, and [soon GPTQ](https://github.com/vllm-project/vllm/pull/1580). - Model loading code became much simpler. - Support model parallelism for all MQA/GQA models when the number of key/value heads is smaller than the tensor parallel size.	2023-11-15 22:50:41 -08:00