iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
3,273 Commits 3 Branches 0 Tags
ffc0f2b47add6e0f70e2b5d4b4aaac64ee97f8ad
Commit Graph
2 Commits
Author SHA1 Message Date
Michael Goin 65b1f121c8 [Bugfix] Fix kv_cache_dtype=fp8 without scales for FP8 checkpoints (#6761) 2024-07-25 09:46:15 -07:00
Michael Goin 9e0b558a09 [Misc] Support FP8 kv cache scales from compressed-tensors (#6528) 2024-07-23 04:11:50 +00:00