6 ms·
Does this mean that we can also run the 60B model on a 16GB ram computer now? I have the M2 air and can't wait until further optimisation with the Neural Engin
by kossTKR 4y ago
Does this mean that we can also run the 60B model on a 16GB ram computer now?
I have the M2 air and can't wait until further optimisation with the Neural Engine / multicore gpu + shared ram etc.
I find it absolutely mind boggling that GPT-3.5(4?) level quality may be within reach locally on my $1500 laptop / $800 m2 mini.
- thomastjeffery 4y agoI doubt it: text size and text pattern size don't scale linearly.
- kossTKR 4y agoInteresting, i wonder how it scales.