5 ms·
I’d be curious if $20/mo is or is not enough to cover their costs. It’s opaque what their inference setup is, or what the final economics of LLMs will be. With
by lumost 3y ago
I’d be curious if $20/mo is or is not enough to cover their costs. It’s opaque what their inference setup is, or what the final economics of LLMs will be. With 30-100 Billion param models becoming available which are in spitting distance of gpt-4, and dedicated ASIC implementations… openAI might be charging too much.
- scarface_74 3y agoAre you taking into account they are also paying far below market costs for compute or even what a large customer could negotiate?
- Grimblewald 3y agoSo because their costs are lower than that of competitors, customers should pay more? Ok.
- scarface_74 3y agoThere costs are lower…for now. Do you think Microsoft is going to subsidize them forever?
- lumost 3y agoGPUs will be cheaper later. GPUs have still been tracking Moore's law, enterprise GPUs went from a niche product to being the hottest enterprise hardware overnight. Margins on H200's are massive from what I understand. I would bet that we'll see massive competition in the hardware space from AMD, NVDA, MSFT, AMZN, and GOOG + startups. We're likely to see an 8-16x reduction in hardware costs over the next decade. That's about enough to shrink the current GPT-4 inference endpoints to a laptop/desktop sized hardware footprint.
- PeterStuer 3y agoCan you provide a source for this? Since it is MS providing the compute for equity, they would not have an incentive to discount.