4 ms·
I was thinking the same. On HF, I see 4bit gguf of this 2:4 model, and I'm like...that works?
by SubiculumCode 2y ago
I was thinking the same. On HF, I see 4bit gguf of this 2:4 model, and I'm like...that works?
- dcreater 2y agoLink?
- SubiculumCode 2y agohttps://huggingface.co/QuantFactory/Sparse-Llama-3.1-8B-2of4-GGUF https://huggingface.co/QuantFactory/Sparse-Llama-3.1-8B-2of4...