3 ms·
You can buy a good GPU and quickly come below cost for GPT APIs depending on your workload. I’m doing millions of tasks, so eating $8k on a custom build for ML
by sjwhevvvvvsj 3y ago
You can buy a good GPU and quickly come below cost for GPT APIs depending on your workload. I’m doing millions of tasks, so eating $8k on a custom build for ML up front is a long term cost savings. Not least of all that every other month there’s an even better model.
- rmbyrro 3y agoYou'd need high usage density to justify investing on a GPU just to run a 70b model locally.
- suslik 3y agoI don't see myself doing millions of tasks on 70b+ models. Work pays for more gpt4 that I can chew; I can run up to 30b models on my mac with a decent speed, and I can use mistral APIs when I travel without powering my own home server. I can see how that buying nvidia GPUs would make sense for a heavy user... Also, I had a gaming addiction for years and try to stay away from anything that can be used for gaming.
- sjwhevvvvvsj 3y agoI’m bootstrapping a startup, so it ends up being cost effective. I also think the long-term outcome is open models will win, so I’m betting on that future while also maintaining OpenAI compatibility in the present.