2 ms·
I think OpenAI, Anthropic and SpaceX are going to envy the dinosaurs because there's not asteroid coming for them, there's three: 1. There will be no moat arou
by jmyeet 3mo ago
I think OpenAI, Anthropic and SpaceX are going to envy the dinosaurs because there's not asteroid coming for them, there's three:
1. There will be no moat around frontier AI models in the future. China is going to make sure that happens. It's a national security interest for them. DeepSeek was the first shot across the bow for that but it won't end with them. There are other labs and there are non-Chinese actors too. The stratospheric valuations depend on there being that moat; and
2. Nobody seems to be considering what the next generation of AI hardware is going to do with current hyperscalar investments. We're about to go through this with the B100/200 move to R100/200 but a lot of the investments are probably slated for that next-gen. But what about 3 years from now when the hypothetical X100/200 comes out and doubles FLOPS and halves performance-per-watt. What will that do to existing investments? Some people are delusional and think that they'll get 10 years out of GPUs when 10 year old GPUs (eg V100) are sold for scrap and 5 year old GPUs (A100) cannot run DeepSeek v4 Pro. And people think the A100 is going to get another 5 years of use? No; and
3. Local LLMs are coming for remote usage. You can buy a 5090 PC for less than $5000 currently but you're limited to 32GB of VRAM, which will comfortably run 31B models but nothing really larger. Go to $12-13k to upgrade to an RTX 8000 Pro and you have 96GB of VRAM, which will run larger models (but certainly not, say, DS v4 Pro or even Flash). You have shared video memory products rapidly coming from NVidia's aggressive market segmentation. Things like Strix Halo and DGX Spark have severe limits on memory bandwidth (<300GB/s compared to 1.8TB/s for a 5090/6000 Pro and 3TB/s+ for server grade HBM3e/4 based GPUs). Macs could be real interesting in this space butr they lack the raw FLOPS with the M5 generation.
But what will this local hardware look like in 2-3 years? I think people will be shocked at how much better it will be with the Apple M7 Pro/Max generation (2028 expected) and the RTX 6000 cards at that time although I fully expect NVidia consumer GPUs to still top out at 32GB of VRAM to maintain that segmentation. And I look forward to what the next generation of the AMD Ryzen AI Halo platform will look like if they really try.
All of this adds up to these three companies needing to cash out before the music stops (IMHO).
- regularfry 3mo agoOn 1 and 3, the obvious move is to shift the bulk of the harness behind a new API that's not based on raw LLM access. Then they get to hide secret sauce behind that API and all three go from commodity to premium while simultaneously being able to try out whatever tricks they can get away with to reduce their own inference costs. I'm almost surprised this hasn't happened already.
- port3000 3mo agoInteresting, so point 2 means that a lot of the hardware being installed now won't be able to run the frontier models of 2029? How does that change the demand for compute/models in the future, I can imagine that even if OpenAI/Anthropic have a moat in 2029 there will be so much older hardware and such a hangover from that investment boom that there will be very little installed capacity that can run it