Isotr0pyandMichael Goin 360bd67cf0 [Core] Support loading GGUF model (#5191) ...

Co-authored-by: Michael Goin <michael@neuralmagic.com>

2024-08-05 17:54:23 -06:00

..

compressed_tensors

[Misc] Disambiguate quantized types via a new ScalarType (#6396)

2024-08-02 13:51:58 -07:00

utils

[Misc] Disambiguate quantized types via a new ScalarType (#6396)

2024-08-02 13:51:58 -07:00

__init__.py

[Core] Support loading GGUF model (#5191)

2024-08-05 17:54:23 -06:00

aqlm.py

[ Misc ] fbgemm checkpoints (#6559)

2024-07-20 09:36:57 -07:00

awq_marlin.py

[Misc] Disambiguate quantized types via a new ScalarType (#6396)

2024-08-02 13:51:58 -07:00

awq.py

[ Misc ] fbgemm checkpoints (#6559)

2024-07-20 09:36:57 -07:00

base_config.py

[Core] Support loading GGUF model (#5191)

2024-08-05 17:54:23 -06:00

bitsandbytes.py

[bitsandbytes]: support read bnb pre-quantized model (#5753)

2024-07-23 23:45:09 +00:00

deepspeedfp.py

[ Misc ] fbgemm checkpoints (#6559)

2024-07-20 09:36:57 -07:00

fbgemm_fp8.py

[Misc] Pass cutlass_fp8_supported correctly in fbgemm_fp8 (#6871)

2024-07-28 11:13:49 -04:00

fp8.py

[Bugfix] Allow vllm to still work if triton is not installed. (#6786)

2024-07-29 14:51:27 -07:00

gguf.py

[Core] Support loading GGUF model (#5191)

2024-08-05 17:54:23 -06:00

gptq_marlin_24.py

[Misc] Disambiguate quantized types via a new ScalarType (#6396)

2024-08-02 13:51:58 -07:00

gptq_marlin.py

[Misc] Disambiguate quantized types via a new ScalarType (#6396)

2024-08-02 13:51:58 -07:00

gptq.py

[ Misc ] fbgemm checkpoints (#6559)

2024-07-20 09:36:57 -07:00

kv_cache.py

[Bugfix] Fix kv_cache_dtype=fp8 without scales for FP8 checkpoints (#6761)

2024-07-25 09:46:15 -07:00

marlin.py

[ Misc ] fbgemm checkpoints (#6559)

2024-07-20 09:36:57 -07:00

qqq.py

Support W4A8 quantization for vllm (#5218)

2024-07-31 07:55:21 -06:00

schema.py

Enable scaled FP8 (e4m3fn) KV cache on ROCm (AMD GPU) (#3290)

2024-04-03 14:15:55 -07:00

squeezellm.py

[ Misc ] fbgemm checkpoints (#6559)

2024-07-20 09:36:57 -07:00