2 ms·
My laptop from multiple years ago (6GB VRAM) can run local models with decent performance. It's my understanding that the major energy cost is from training the
by insanitybit 3y ago
My laptop from multiple years ago (6GB VRAM) can run local models with decent performance. It's my understanding that the major energy cost is from training the models, which you do somewhat infrequently, and that generation isn't nearly as costly.
Someone who knows more, please correct me if I'm wrong!