3 ms·
I thought the wisdom is more so don’t bother going below 4bit and you won’t see a difference above 8bit.
by teaearlgraycold 18d ago
I thought the wisdom is more so don’t bother going below 4bit and you won’t see a difference above 8bit.
- anyfoo 18d agoDepends on the actual audience, I guess. My stated “wisdom” comes in part from /r/LocalLLaMa, and my impression is that the tasks that users there give their models to try them out lean towards rather simplistic, on the reasoning side. But there I literally did read “you don’t need anything better than 4 bpw” a bunch of times.
- neonstatic 18d ago> my impression is that the tasks that users there give their models to try them out lean towards rather simplistic, on the reasoning side. That sub is games, porn, and complaints about not having money. A waste of time.