3 ms·
It fits a 4090. Nvidia lists the models and therefore i assume 24gig is min
by Bumblonono 2y ago
It fits a 4090. Nvidia lists the models and therefore i assume 24gig is min
- michaelt 2y agoA 4090 will just narrowly fit a 34B parameter model at 4-bit quantisation. A 12B model will run on a 4090 with plenty room to spare, even with 8-bit quantisation.