4 ms·
There is a user called The Bloke on hugging face- they release pre quantized models pretty soon after the full size drop. Just watch their page and pray you can
by bevekspldnw 3y ago
There is a user called The Bloke on hugging face- they release pre quantized models pretty soon after the full size drop. Just watch their page and pray you can fit the 4 bit in your GPU.
I’m sure they are already working on it.
- nathanasmith 3y agoTheBloke stopped uploading in January. There are others that have stepped up though.
- bevekspldnw 3y agoOh really? Who else should I be looking at? That person is a hero, super bummed!
- fzzzy 3y agoTheBloke's grant ran out.
- MPSimmons 3y agoI think 4b for this is support to be over 70GB, so definitely still heavy hardware.
- bevekspldnw 3y agoFucking hell, my A6000 is shy of that and I can’t reasonably justify picking up a second.