5 ms·
There is no moat in LLMs, I don't see this ending well. NVIDIA might buy them, but not for the price they want.
by 7e 2y ago
There is no moat in LLMs, I don't see this ending well. NVIDIA might buy them, but not for the price they want.
- dartos 2y agoThe moat is in the sheer cost of training models. At most you’ll have like 5 major players.
- turnsout 2y agoIsn’t their training subsidized by Microsoft? Sounds like Microsoft has the moat.
- dartos 2y agoSure, whatever. The moat is still the cost.
- turnsout 2y agoI hear ya. But with great open source models like Llama 3.2 dropping near-daily, I wonder if/when we’ll reach a point where free models are good enough for 99% of tasks.
- dartos 2y agoLlama is NOT open source. It’s freeware. We can’t reproduce it and its license has commercial restrictions. That being said, most models are based on llama, so it may seem like a lot of movement is happening, but it’s not, really. RWKV and deepseek are the only LLMs that’re fully open that I’m aware of. If meta decided to stop releasing llama, that’s that.
- n_ary 2y agoIf llama achieves valuable strides, it will definitely be put behind paywall. The play has always been the same, release something semi-polished until you gain mass adaptation, then build a paywall infront when people are reliant on it and attrition is expensive. Look at the twitter API for example, or reddit or what not.
- __loam 2y agoWhen there's like 5 essentially interchangeable products from your competitors, you don't have a moat.
- dartos 2y agoWe don’t know that they will be interchangeable forever. Just because they share a near identical interface does not mean they will always have identical capabilities
- pr337h4m 2y agoMicrosoft is extremely likely to get rugged: “Fifth, the board determines when we've attained AGI. Again, by AGI we mean a highly autonomous system that outperforms humans at most economically valuable work. Such a system is excluded from IP licenses and other commercial terms with Microsoft, which only apply to pre-AGI technology.” https://openai.com/our-structure/ https://openai.com/our-structure/
- throwaway314155 2y agoSorry but that quote just makes it seem more likely that Microsoft will be able to hold on to their current agreement for a fairly long time.
- dartos 2y ago“If we discover the holy grail in the unicorn meadow, you can’t have it” Is a fine clause for MS. Hardly a rug pull.
- nkozyra 2y ago> At most you’ll have like 5 major players. 5 companies doing a repeated leapfrog game where consumers can flip without much effort seems like the opposite of a moat, unless and until they consolidate into a monopoly. I'm starting to feel some diminishing returns on the new releases, too. The wow moments are still there but fundamentally we're reaching a point wherein even open models can come close enough on some tasks that we can eschew paid models for not-insignificant pieces here and there.
- dartos 2y agoWe don’t know if they will stay exactly interchangeable forever. Google has a deal for Reddit’s data, so its models will have that information. I can see this kind of thing being a moat as well. And yeah obviously you’re going to feel diminishing returns. Going from 0 to 1 always feels like a bigger step than from 1 to 2.
- nkozyra 2y ago> Google has a deal for Reddit’s data, so its models will have that information. I can see this kind of thing being a moat as well. I don't see this as a silver bullet but let's assume this is true and a differentiator. This kind of validates that building a robust model hits a data saturation wall at some point and the moat dries up.
- ethbr1 2y agoFresh human data feeds at scale are always going to be valuable. - Meta has its own - Reddit is monetizing its - Twitter was bought for its - MS is trying to create one - Amazon has a funhouse mirror version (reviews) Apple seems oddly without a dance partner, but I'm guessing they figure they can leverage pre-built / open models for the features they're targeting.
- jprete 2y agoEvery single one of those is flooded with bots and spam.
- gloryjulio 2y agoIf meta keeps giving the model away, wondering how that impacts their valuation
- n_ary 2y agoGiving away is sort of beta testing and free marketing for adaptation. Just like OpenAI(who last released GPT2 as ~open), Meta can do the same if llama becomes valuable. However, meta already has moat(user data), so I somewhat expect llama to be sort of ReactJS where it becomes industry de-facto rather than a product for sale.
- bronco21016 2y agoAs they make progress they slowly remove “Open” from “OpenAI”. The “o1” models give almost no details other than RL on chain of thought. They haven't demonstrated a clear moat yet but I wouldn’t be surprised to see this restructuring occur and then they show the product that justifies the valuation.
- bottlepalm 2y agoIdk OpenAI sure makes it seem like they have some secret sauce that others have been unable to catch up to.
- throwaway314155 2y agoIs that why they've been barely ahead/losing in benchmarks to anthropic lately?
- bottlepalm 2y agoLast I checked Anthropic has no image, video, or audio generation. No multi model. Does it have an answer to o1?
- ac29 2y agoAnthropic has had multimodal models since at least Claude 3.
- bottlepalm 2y agoWhere? I only see image input, nothing else. And nothing other than text available through the API.
- throwaway314155 2y agoMultimodal generally refers to _just_ two (or more) modes. Omni-modal was invented by OpenAI to describe their approach. I'm sure Anthropic is working on something similar to gpt-4o but I don't see how that invalidates my argument about anthropic doing well on benchmarks. In particular because their model was released several months _before_ gpt-4o and still holds its own fairly well.
- bottlepalm 2y agoPersonally I like Anthropic, they do one thing well, but there are a lot more applications for AI that OpenAI is offering solutions for. Anthropic, not so much.
- Yizahi 2y agoThe moat is a stolen copyrighted data, used for training LLMs. For now you can probably repeat all what OpenLLM did and others too, but the window of opportunity is closing fast. Some companies are making 1-1 deals with existing LLM provides regarding their data, some are closing down, laws are being written slowly but surely.
- 93po 2y agoCan we stop calling it stolen? It wasn't stolen. Copyright infringement is never stealing and it's not even well established infringement is happening.
- Yizahi 2y agoI once thought like that too, "we need balanced, moderate approach etc.". But seeing blatant copyright violations and abuse by megacorps I tend to skew to calling it theft. Purely due to the imbalance of power between individuals and megacorps. The problem we have is that hearing "copyright infringement" people think of different aspects of it. Most people think about how individuals infringe on corpos, like pirating software or a movie and that is indeed is morally grey zone. I was talking purely about reverse situation - when corpos infringe on the copyright of individuals. That is unquestionably abuse of power and in my opinion it is theft. This part at least.