4 ms·
Quantization can take it under 30GB (with quality degradation). For example, take a look at the GGUF file sizes here: https://huggingface.co/TheBloke/Llama-2-7
by rgbrgb 3y ago
Quantization can take it under 30GB (with quality degradation).
For example, take a look at the GGUF file sizes here: https://huggingface.co/TheBloke/Llama-2-70B-GGUF https://huggingface.co/TheBloke/Llama-2-70B-GGUF