4 ms·
I’m not that interested in models that I can’t run on my desktop for ~0€, which is my AI budget.
by Hamuko 4mo ago
I’m not that interested in models that I can’t run on my desktop for ~0€, which is my AI budget.
- igravious 4mo agoCool beans. You're not the target audience then.
- Hamuko 4mo agoDid I claim I was? I just said why I and people like me are not talking about it.
- simianwords 4mo agoand he said its cool
- andai 4mo agoElectricity cost seems to be about $30/month for a 32B model on a GPU. It's probably better on Apple hardware. https://github.com/QuantiusBenignus/Zshelf/discussions/2 https://github.com/QuantiusBenignus/Zshelf/discussions/2 Not accounting for hardware, of course :)
- Hamuko 4mo agoMy Mac Studio uses about 60–80 watts whenever I’m running a model (as measured by the system metrics), so it’s less than 2 kWh/day at full blast. Electricity is like 0.125 €/kWh, so that 24-hour period would be <0.25 €. Not accounting hardware in my costs, since I didn’t buy my hardware for running models. Running models is just something it can do in addition to what I got it for.
- NorwegianDude 4mo agoThe price, processed tokens, and output can be anything, it just depends on what GPU it is. Nvidia GPUs are much more efficient than Apple hardware for inference(and training).