3 ms·
Don't forget that you're also spending much more electricity because it takes so long to run inference.
by iamsaitam 7mo ago
Don't forget that you're also spending much more electricity because it takes so long to run inference.
- Wowfunhappy 7mo agoGiven current hardware prices I wouldn't expect this to tip the scales. Mind you, for me local models are a fun experiment, for anything serious I would use a frontier model. So I'm not using this for hours a day.