6 ms·
This is the result of the winner-take-all (most) economy. If the very best LLM is 1.5x as good as good as the next-best, then pretty much everyone in the world
by apparent 1y ago
This is the result of the winner-take-all (most) economy. If the very best LLM is 1.5x as good as good as the next-best, then pretty much everyone in the world will want to use the best one. That means billions of dollars of profit hang in the balance, so companies want to make sure they get the very best people (even if they have to pay hundreds of millions to get it).
It's the same reason that sports stars, musicians, and other entertainers that operate on a global scale make so much more money now than they did 100 years ago. They are serving a market that is thousands of times larger than their predecessors did, and the pay is commensurately larger.
- deleted 1y ago[deleted]
- mi_lk 1y ago> winner-take-all (most) > If the very best LLM is 1.5x as good as good as the next-best, then pretty much everyone in the world will want to use the best one Is it? Gemini is arguably better than OAI in most cases but I'm not sure it's as popular among general public
- apparent 1y agoI don't think there's a consensus on this. I have found Gemini to be so-so, and the UX is super annoying when you run out of your pro usage. IME, there's no way to have continuity to a lower-tier model, which makes is a huge hassle. I basically never use it anymore.
- decimalenough 1y agoIn other words, what matters is not just which one is "best"?
- apparent 1y agoIf the Google model was 50% better than OpenAI I would have bought a subscription, which would moot the UX issue. But IME it isn't discernibly better at all, let alone 50% better.
- ant6n 1y agoThe hack I found to “get around” the secret limits that pop out of nowhere is to you export the chat, then import it with another account. So you don’t need to pay 200, just 40 (and only when you actually need that much).
- esafak 1y agoIt's multivariate; better for what? None of them are best across the board. I think what we're seeing here is superstar economics, where the market believes the top players are disproportionately more valuable than average. Typically this is bad, because it leads to low median compensation but in this rare case it is working out.
- toddmorey 1y agoI don’t see a winner takes all moat forming. If anything, the model providers are almost hot-swappable. And it seems the lifespan for being the best SOTA model is now measured in weeks.
- apparent 1y agoIt's true they are currently quite interchangeable these days. But the point is that if one can pull far enough ahead, it will get a much bigger share of the market.
- jjmarr 1y agoThe actual OpenRouter data says otherwise.[1] Right now, Google leads with only 28.4% marketshare. Anthropic (24.7%), Deepseek (15.4%), and Qwen (10.8%) are the runners-up. If this were winner-take-all market with low switching costs, we'd be seeing instant majority market domination whenever a new SOTA model comes out every few weeks. But this isn't happening in practice, even though it's much easier to switch models on OpenRouter than many other inference providers. I get the perception of "winner-take-all" is why the salaries are shooting up, but it's at-odds with the reality. [1] https://openrouter.ai/rankings https://openrouter.ai/rankings
- nextworddev 1y agoOpenrouter data is skewed toward 1) startups, 2) cost sensitive workloads, and generally not useful as a gauge of enterprise adoption
- jjmarr 1y agoIs there good public data on enterprise adoption?
- ascorbic 1y agoBut those are just the sort of cases that are more likely to switch to the latest, greatest and cheapest when they come around. The fact that even these haven't become winner takes all is a strong signal that these models have sticking power.
- nextworddev 1y agoNo because more than half of open router is character chat and writing ad copies… so it’s not indicative of the important use cases
- jjmarr 1y agoThe top 3 apps are Kilo Code, Roo Code, and Cline. These are all coding assistants. While they can be turned into waifus they're not intended as such.
- TrackerFF 1y agoI don't think AI will be a winner-take-all scenario. If that is to happen, I think the following assumptions must hold: 1) The winner immediately becomes a monopoly 2) All investments are directed from competitors, to the winner 3) Research on AGI/ASI ceases I don't see how any of these would be viable. Right now there's an incremental model arms race, with no companies holding a secret sauce so powerful that they're miles above the rest. I think it will continue like it does today. Some company will break through with some sort of AGI model, and the competitors will follow. Then open source models will be released. Same with ASI. The things that will be important and guarded are: data and compute.
- apparent 1y agoYeah, this is why I said "(most)". But regardless, I think it's pretty uncontroversial that not all companies currently pursuing AI will ultimately succeed. Some will give up because they aren't in the top few contenders, who will be the only ones that survive in the long run. So maybe the issue is more about staying in the top N, and being willing to pay tons to make sure that happens.
- rcxdude 1y ago>I think it's pretty uncontroversial that not all companies currently pursuing AI will ultimately succeed. That's probably true, but at the moment the only thing that creates something resembling a moat is the fact that progress is rapid (i.e. the top players are ~6-12 months ahead of the already commoditized options, but the gap in capabilities is quite large): if progress plateaus at all, the barrier to be competitive with the top dogs is going to drop a lot, and anyone trying to extra value out of their position is going to attract a ton of competition even from new players.
- nsriv 1y agoI agree with you, and think we are in the heady days where moat building hasn't quite begun. Regarding 1) and 3), most models have API access to facilitate quick switching and agentic AI middleware reaps the benefits of new models being better at some specific use-case than a competitor. In the not-so-distant future, I can see the walls coming up, with some version of white-listed user-agent access only. At the moment, model improvement hype and priority access are the product, but at some point capability and general access will be the product. We are already seeing diminishing returns from compute and training costs going up, but as more and more AI is used in the wild and pollutes training data, having validated data becomes the moat.
- adammarples 1y agoWho the heck is making profit?
- gddgb 1y ago[dead]
- beAbU 1y agoIf the next best model is 0.5x as expensive, then many might opt for that in use cases where the results are already good enough. At work we are optimising cost by switching in different models for different agents based on use case, and where testing has demonstrated a particular model's output is sufficient.
- stefan_ 1y agoThat is exactly how all of this has played out so far (beep, not at fucking all!) You are not one random hyperparameter away from the SciFi singularity. You are making iterative improvements and throwing more compute at the problem, as are all your competitors, all of which are to some degree utterly exchangeable.
- polski-g 1y ago> If the very best LLM is 1.5x as good as good as the next-best, then pretty much everyone in the world will want to use the best one. Well only if the price is the same. Otherwise people will value price over quality, or quality over price. Like they do for literally every other product they select...
- fooker 1y agoYou’re right. It would be unfortunate if something like Grok takes the cake here.
- apparent 1y agoWhy?
- redox99 1y agoMost likely won't be a winner-take-all, but something like it is right now, a never ending treadmill where the same 3 or 4 players release a SOTA model every year or so.
- croes 1y agoThere is LLM 1.5x as good as the next best. I tried multiple and they all fail and some point so I let another LLM take over. As soon it’s not some boilerplate thing it becomes harder to get the correct result
- UmGuys 1y agoI don't think so. It's just a bubble. There's no AI, we have fancy chatbots. If someone were to achieve AGI, maybe they win, but it's unlikely to exist. Or if it does, we can't define it.
- nopinsight 1y agoWhat would you say if the IMO Gold Medal models from DeepMind and OpenAI turn out to be generalizable to other domains including those with hard-to-verify reward signals? Hint: Researchers from both companies said publicly they employ generalized reasoning techniques in these IMO models.
- UmGuys 1y agoNice. That's really great progress. Maybe a model will be able to infer what consciousness is.
- cesarb 1y ago> There's no AI, we have fancy chatbots. "Fancy chatbots" is a classic AI use case. ELIZA is a well-known example of early AI software.
- UmGuys 1y agoI'm not familiar with ELIZA. Technical terms mean things. AI has no technical meaning. It's difficult to distinguish what people mean when they say AI, but as technical people I assume we know AGI doesn't exist yet and we don't know if it ever will.