2 ms·
Its a 7B model with GGUFs already on Hugging face, with down to a 2-bit quantization. It’ll probably run on an RPi; you definitely don’t need an RTX 4090.
by dragonwriter 3y ago
Its a 7B model with GGUFs already on Hugging face, with down to a 2-bit quantization. It’ll probably run on an RPi; you definitely don’t need an RTX 4090.