2 ms·
You can run some of the (right combination of smaller and/or quantized) models on consumer laptop/desktop GPUs and even more can be run (if slowly) on CPU with
by dragonwriter 3y ago
You can run some of the (right combination of smaller and/or quantized) models on consumer laptop/desktop GPUs and even more can be run (if slowly) on CPU with plenty of RAM, but, yeah, beyond a certain point in model capability/performance, you are going to own, or rent, datacenter GPUs.