4 ms·

I felt anxious about all the insane valuations and spending around AI lately, and I knew it couldn't last (I mean there's only so much money, land, energy, wate

I felt anxious about all the insane valuations and spending around AI lately, and I knew it couldn't last (I mean there's only so much money, land, energy, water, business value, etc). But I didn't really know when it was going to collapse, or why. But recently I've been diving into using local models, and now it's way more clear. There seems to be a specific path for the implosion of AI: - Nvidia is the most valuable company. Why? It makes GPUs. Why does that matter? Because AI is faster on them than CPUs, ASICs are too narrowly useful, and because first-mover advantage. AMD makes GPUs that work great for AI, but they're a fraction of the value of Nvidia, despite the fact that they make more useful products than Nvidia. Why? Nvidia just got there first, people started building on them, and haven't stopped, because it's the path of least resistance. But if Nvidia went away tomorrow, investors would just pour money into AMD. So Nvidia doesn't have any significant value compared to AMD other than people are lazy and are just buying the hot thing. Nvidia was less valuable than AMD before, they'll return there eventually; all AMD needs is more adoption and investment. - Every frontier model provider out there has invested billions to get models to the advanced state they're in today. But every single time they advance the state of the art, open weights soon match them. Very soon, there won't be any significant improvement, and open weights will be the same as frontier, meaning there's no advantage to paying for frontier models. So within a few years, there will be no point to paying OpenAI, Anthropic, etc. Again, these were just first-movers in a commodity market. The value just isn't there. They can still provide unique services, tailored polished apps, etc (Anthropic is already doing this by banning users who have the audacity to use their fixed-price plans with non-Anthropic tools). But with AI code tools, anyone can do this. They are making themselves obsolete. - The final form of AI coding is orchestrated agent-driven vibe-coding with safeguards. Think an insane asylum with a bowling league: you still want 100 people to autonomously (and in parallel) knock the pins knocked over, but you have to prevent the inmates from killing anyone. That's where the future of coding is. It's just too productive to avoid. But with open models and open source interfaces, anyone can do this, whether with hosted models (on any of 50 different providers), or a Beowulf cluster of cobbled together cheap hardware in a garage. - Eventually, in like 5-10 years (a lifetime away), after AI Beowulfs have been a fad for a while, people will tire of it and move back to the cloud, where they can run any model they want on a K8s cluster full of GPUs, basically the same as today. Difference between now and then is, right now everyone is chasing Anthropic because their tools and models are slightly better. But by then, they won't be. Maybe people will use their tools anyway? But they won't be paying for their models. And it's not just price: one of the things you learn quickly by running models, is they're all good for different things. Not only that, you can tweak them, fine-tune them, and make them faster, cheaper, better than what's served up by frontier models. So if you don't care about the results or cost, you could use frontier, but otherwise you'll be digging deep into them, the same way some companies invest in writing their own software vs paying for it. - Finally, there's the icing on the cake: LLMs will be cooked in 10 years. I keep reading from AI research experts that "LLMs are a dead end" - and it turns out it's true. LLMs are basically only good because we invest an unsustainable amount of money in the brute-forcing of a relatively dumb form of iteration: download all knowledge, do some mind-bogglingly expensive computational math on it, tweak the reasults, repeat. There's only so many of that loop you can do, because fundamentally, all you're doing is trying to guess your way to an answer from a picture of the past. It doesn't actually learn, the way a living organism learns, from experience, in real-time, going forward; LLMs only look backward. Like taking a snapshot of all the books a 6 year old has read, then doing tweaks to try to optimize the knowledge from those books, then doing it again. There's only so much knowledge, only so many tweaks. The sensory data of the lived experience of a single year of life of a 6 year old is many times more information than everything ever recorded by man. Reinforcement Learning actually gives you progressive, continuously improved knowledge. But it's slow, which is why we aren't doing it much. We do LLMs instead because we can speed-run them. But the game has an end, and it's the total sum of our recorded knowledge and our tweaks. So LLMs will plateau, frontier models will make no sense, all lines of code will be hands-off, and Nvidia will return to making hardware for video games. All within about 10 years. With the caveat that there might be a shift in global power and economic stability that interrupts the whole game.... but that's where we stand if things keep on course. Personally, I am happy to keep using AI and reap the benefits of all these moronic companies dumping their money into it, because the open weights continue being useful after those companies are dead. But I'm not gonna be buying Nvidia stock anytime soon, and I'm definitely not gonna use just one frontier model company.

Turfie 9mo ago
I've thought about this too. I do agree that open source models look good and enticing, especially from a privacy standpoint. But these solutions are always going to remain niche solutions for power users. I'm not one of them. I can't be hassled/bothered to setup that whole thing (local or cloud) to gain some privacy and end up with an inferior model and tool. Let's not forget about the cost as well! Right now I'm paying for Claude and Gemini. I run out of Claude tokens real fast, but I can just keep on going using Gemini/GeminiCLI for absolutely no cost it seems like. The closed LLMs with the biggest amount of users will eventually outperform the open ones too, I believe. They have a lot of closed data that they can train their next generation on. Especially the LLMs that the scientific community uses will be a lot more valuable (for everyone). So in terms of quality, the closed LLMs should eventually outperform the open ones, I believe, which is indeed worrisome. I also felt anxious early december about the valuations, but, one thing remains certain. Compute is in heavy demand, regardless of which LLM people use. I can't go back to pre-AI. I want more and more and faster and faster AI. The whole world is moving that way it seems like. I'm invested into phsyical AI atm (chips, ram, ...) whose evaluations look decently cheap.
- 0xbadcafebee 9mo ago
  I think you should reconsider the idea that frontier models will be superior, for a couple reasons: - LLMs have fixed limitations. The first one is training, the dataset you use. There's only so much information in the world and we've largely downloaded it all, so it can't get better there. Next you can do training on specific things to make it better at specific things, but that is by definition niche; and you can actually do that for free today with Google's Tensors in free Cloud products. Later people will pay for this, but the point is, it's ridiculously easy for anyone to fine-tune training, we don't need frontier companies for that. And finally, LLM improvements come by small tweaks to models that already come to open weights within a matter of months, often surpassing the frontier! All you have to do is sit on your ass for a couple months and you have a better open model. Why would anyone do this? Because once all models are extremely good (about 1 year from now) you won't need them to be better, they'll already do everything you need in 1-shot, so you can afford to sit and wait for open models. Then the only reason left to use frontier cloud is that they host a model; but other people do cloud-hosted models! Because it's a commodity! (And by the way, people like me are already pissed off at Anthropic because we're not allowed to use OAuth with 3rd party tools, which is complete bullshit. I won't use them on general principle now, they're a lock-in moat, and I don't need them) There will also be better, faster, more optimized open models, which everyone is going to use. For doing math you'll use one model, for intelligence you'll use a different model, for coding a different model, for health a different model, etc, and the reason is simple: it's faster, lower memory, and more accurate. Why do things 2x slower if you don't have to? Frontier model providers just don't provide this kind of flexibility, but the community does. Smart users will do more with less, and that means open. On the hardware: - Def it will continue to be investment-worthy, but be cautious. The growth simply isn't going to continue at pace, and the simple reason is we've already got enough hardware. They want more hardware so they can continue trying to "scale LLMs" the way they have with brute force. But soon the LLMs will plateau and the brute force method isn't going to net the kind of improvements that justify the cost. Demand for hardware is going to drop like a stone in 1-2 years; if they don't cease building/buying then, they risk devaluing it (supply/demand), but either way Nvidia won't be selling as much product so there goes their valuation. And RAM is eventually going to get cheaper, so even if demand goes up, spending is less. The other reason demand won't continue at pace is investors are already scared, so the taps are being tightened (I'm sure the "Megadeal" being put on-hold is the secret investment groups tightening their belts or trying to secure more favorable terms). I honestly can't say what the economic picture is going to look like, but I guarantee you Nvidia will fall from its storied heights back to normal earth, and other providers will fill the gap. I don't know who for certain, but AMD just makes sense, because they're already supported by most AI software the way Nvidia is (try to run open-source inference today, it's one of those two). Frontier and cloud providers have Tensors and other exotic hardware, which is great for them, but everyone else is gonna buy commodity chips. Watch for architectures with lower price and higher parts availability.

Turfie 9mo ago

I've thought about this too. I do agree that open source models look good and enticing, especially from a privacy standpoint. But these solutions are always going to remain niche solutions for power users. I'm not one of them. I can't be hassled/bothered to setup that whole thing (local or cloud) to gain some privacy and end up with an inferior model and tool. Let's not forget about the cost as well! Right now I'm paying for Claude and Gemini. I run out of Claude tokens real fast, but I can just keep on going using Gemini/GeminiCLI for absolutely no cost it seems like. The closed LLMs with the biggest amount of users will eventually outperform the open ones too, I believe. They have a lot of closed data that they can train their next generation on. Especially the LLMs that the scientific community uses will be a lot more valuable (for everyone). So in terms of quality, the closed LLMs should eventually outperform the open ones, I believe, which is indeed worrisome. I also felt anxious early december about the valuations, but, one thing remains certain. Compute is in heavy demand, regardless of which LLM people use. I can't go back to pre-AI. I want more and more and faster and faster AI. The whole world is moving that way it seems like. I'm invested into phsyical AI atm (chips, ram, ...) whose evaluations look decently cheap.

0xbadcafebee 9mo ago
I think you should reconsider the idea that frontier models will be superior, for a couple reasons: - LLMs have fixed limitations. The first one is training, the dataset you use. There's only so much information in the world and we've largely downloaded it all, so it can't get better there. Next you can do training on specific things to make it better at specific things, but that is by definition niche; and you can actually do that for free today with Google's Tensors in free Cloud products. Later people will pay for this, but the point is, it's ridiculously easy for anyone to fine-tune training, we don't need frontier companies for that. And finally, LLM improvements come by small tweaks to models that already come to open weights within a matter of months, often surpassing the frontier! All you have to do is sit on your ass for a couple months and you have a better open model. Why would anyone do this? Because once all models are extremely good (about 1 year from now) you won't need them to be better, they'll already do everything you need in 1-shot, so you can afford to sit and wait for open models. Then the only reason left to use frontier cloud is that they host a model; but other people do cloud-hosted models! Because it's a commodity! (And by the way, people like me are already pissed off at Anthropic because we're not allowed to use OAuth with 3rd party tools, which is complete bullshit. I won't use them on general principle now, they're a lock-in moat, and I don't need them) There will also be better, faster, more optimized open models, which everyone is going to use. For doing math you'll use one model, for intelligence you'll use a different model, for coding a different model, for health a different model, etc, and the reason is simple: it's faster, lower memory, and more accurate. Why do things 2x slower if you don't have to? Frontier model providers just don't provide this kind of flexibility, but the community does. Smart users will do more with less, and that means open. On the hardware: - Def it will continue to be investment-worthy, but be cautious. The growth simply isn't going to continue at pace, and the simple reason is we've already got enough hardware. They want more hardware so they can continue trying to "scale LLMs" the way they have with brute force. But soon the LLMs will plateau and the brute force method isn't going to net the kind of improvements that justify the cost. Demand for hardware is going to drop like a stone in 1-2 years; if they don't cease building/buying then, they risk devaluing it (supply/demand), but either way Nvidia won't be selling as much product so there goes their valuation. And RAM is eventually going to get cheaper, so even if demand goes up, spending is less. The other reason demand won't continue at pace is investors are already scared, so the taps are being tightened (I'm sure the "Megadeal" being put on-hold is the secret investment groups tightening their belts or trying to secure more favorable terms). I honestly can't say what the economic picture is going to look like, but I guarantee you Nvidia will fall from its storied heights back to normal earth, and other providers will fill the gap. I don't know who for certain, but AMD just makes sense, because they're already supported by most AI software the way Nvidia is (try to run open-source inference today, it's one of those two). Frontier and cloud providers have Tensors and other exotic hardware, which is great for them, but everyone else is gonna buy commodity chips. Watch for architectures with lower price and higher parts availability.