4 ms·
Most of the loss comes from hefty cost of inference, right? OAI runs on Azure, so everything is expensive at their scale: the data, the compute, and GPU instanc
by hintymad 2y ago
Most of the loss comes from hefty cost of inference, right? OAI runs on Azure, so everything is expensive at their scale: the data, the compute, and GPU instances, and storage, and etc.
I'd venture to guess that they will start building their own data centers with their own inference infra to cut the cost by potentially 75% -- i.e., the gross markup of a public cloud service. Given their cost structure, building their own infra seems cheap.
- slashdave 2y agoI cannot believe they don't already get steep discounts via custom contracts with providers. Someone has to pay for the hardware and electricity.
- azinman2 2y agoGiven their scale and direct investments from Microsoft, what makes you think this is any cheaper? They’ll be getting stuff at or near cost from azure, and Azure already exists and with huge scale to amortize all of the investments across more than just OpenAI, including volume discounts (and preference) from nvidia.
- hintymad 2y agoI assumed that Microsoft gave them discount and credits as investment, but the cost by itself is like any other big customer can get.
- azinman2 2y agoOf course the cost isn’t near zero, but is the pricing “at Microsoft’s cost”? If it is, then them building a datacenter wouldn’t save anything, plus would have enormous expenses related to, well, building, maintaining, and continuously upgrading data centers.
- hintymad 2y agoGood point. I updated the assumption accordingly. I assume the cost of using Azure cloud after discount but before the credit will be on par with Azure's big customers. And given the scale of OAI, I suspect that the only way to be profitable is to have their own infra.
- adastra22 2y agoThe cost of vertically integrated inference isn’t zero either.
- lesuorac 2y agoWouldn't it be foolish for microsoft to give them a discount? Like you could discount say a 10B investment down to 6B or w/e the actual costs are but now your share of OpenAI is based on 6B. Seems better to me to give OpenAI a "10B" investment that internally only costs 6B so your share of OpenAI is based on 10B. Plus OpenAI probably likes the 10B investment more as it raises their market cap.
- the_real_cher 2y agoAre there enough gpus available at the scale that they need them?
- gm3dmo 2y agoBuilding datacenters will take a significant amount of time. if they don’t have locations secured then even more so.
- paxys 2y agoAs part of Microsoft's last investment in 2023 OpenAI agreed to exclusively use Azure for all their computing needs. I'm not sure what the time limit on that agreement is, but at least for now OpenAI cannot build their own datacenters even if they wanted to.
- neomoto 2y agoMicrosoft restricted their computing needs provider, but they could have allowed them to build their own datacenters.
- tracerbulletx 2y agoIf they only have enough for a year or two of runway at current costs how the heck would they have enough capital to build an AI data center?