3 ms·
That's 6 YEARS of their cloud inference! And, you have to include power. Where I am, the power for a rig with 2x 3090, running for four hours a day, would be a
by nomel 2mo ago
That's 6 YEARS of their cloud inference!
And, you have to include power. Where I am, the power for a rig with 2x 3090, running for four hours a day, would be around $70/month!
It's kinda like datacenter only exist to optimize compute cost through oversubscription of hardware/time share, cheaper business energy rates, and bulk discounts, compared to self hosting. ;)
- ActorNightly 2mo agoRunning local models isn't about the cost, its about owning your own data and being able to run uncensored models. If you just want to code with an assistant, local inference is by far not worth it. You also don't need to run the models 24/7, nor is your computer gonna be doing inference all the time when you are coding.
- nomel 1mo agoNo, you're mixing unrelated concepts here. Cloud does not mean you don't own the data, and can't do what you want. It means it's running on commoditized server hardware, powered with cheap commercial energy costs, oversubscribed so nothing sits idle.