3 ms·
I have heard estimates [1] that OpenAI's $20/month ChatGPT charge isn't enough to cover the cost of compute. For example, if their model needs ~100GB of VRAM, a
by michaelt 3y ago
I have heard estimates [1] that OpenAI's $20/month ChatGPT charge isn't enough to cover the cost of compute. For example, if their model needs ~100GB of VRAM, at cloud prices of $0.08 per minute for a g5.12xlarge. you make a loss if the average customer uses more than 9 minutes per day. And chatgpt's responses are pretty slow.
And that's without counting the huge free tier they have to support, the employees they have to pay for, and any overheads.
No rush to go multi-cloud if you're making a loss - you'd want to put your efforts into reducing costs.
[1] AI chatbots lose money every time you use them, https://www.washingtonpost.com/technology/2023/06/05/chatgpt-hidden-cost-gpu-compute/ https://www.washingtonpost.com/technology/2023/06/05/chatgpt...
- hesdeadjim 3y agoThey are in “survive until GPUs have enough RAM for break even costs” mode right now. It’s why I don’t think we’ll see any big significant changes in parameter count for a while because it’s just too damned expensive. I would however be happy to pay waaaaay more than $20/mo for a “gpt-5” evolved model.
- janalsncm 3y agoI don’t think that’s a fair comparison. AWS GPU pricing is kind of ridiculous. $30-40 per hour for A100 clusters is several times what other companies offer. I only use them because I have free credits but I would never pay out of pocket for their GPUs.