7 ms·

Genuinely curious if anyone has ideas how an LLM provider could moat AI. They feel interchangeable like bandwidth providers and seem like it will be a never end

Genuinely curious if anyone has ideas how an LLM provider could moat AI. They feel interchangeable like bandwidth providers and seem like it will be a never ending game of leapfrog. I thought perhaps we’d end up with a small set of few top players just based on the scale of investment but now I’ve also seen impressive gains from much smaller companies & models, too.

wmf 2y ago
You'd have to have either a breakthrough algorithm (that you keep secret forever) or proprietary training data.
deleted 2y ago
[deleted]
scarmig 2y ago
1) Push for regulations around data provenance. If you train on anything, you have to prove that you had the rights to train on it. This would kill all but the largest providers in the USA, though China would still pose a problem. You could work around that bit by making businesses and consumers liable for usage of models that don't have proof of provenance. 2) If you had some secret algorithm that substantially outperformed everyone, you could win if you prevented leakage. This runs into the issue that two people can keep a secret, but three cannot. Eventually it'll leak. 3) Keep costs exceptionally low, sell at cost (or for free), and flood the market with that, which you use to enhance other revenue streams and make it unprofitable for other companies to compete and unappealing as a target for investors. To do this, you have to be a large company with existing revenue streams, highly efficient infrastructure, piles of money to burn while competitors burn through their smaller piles of money, and the ability to get something of value from giving something out for free.
- sdesol 2y ago
  > China would still pose a problem Not if regulation prohibits LLMs from China, which isn't that far fetched to be honest. I think LLM will turn into a commodity product and if you want to dominate with a commodity product, you need to provide twice the value at half the cost. Open AI will need a breakthrough in reliability and/or inference cost to really create a moat.
  - callc 2y ago
    If US regulation prohibits China from training LLMs? How? If you mean trying to stop GPUs getting to China, US already has tried that with specific GPU models, but China still gets them. Seems hard/impossible to do. Even if US and CCP were trying to stop Chinese citizens and companies doing LLM stuff
    - sdesol 2y ago
      Not prohibit training, but make it illegal for US companies to embed, use or distribute LLMs created by China. Basically the general consumer will need to go through the effort of use the LLM that they want, which we know means only a small fraction of people.
      - seanmcdirmid 2y ago
        That assumes that only the US is interested in using LLMs commercially, which isn't really true. Even if you can get America to sanction Chinese LLM use, you aren't even going to American allies to go along with that, let alone everyone else. China's biggest challenge ATM is that they do not yet economically produce the GPUs and RAM needed to train and use big models. They are still behind in semiconductors, maybe 10 or 20 years (they can make fast chips, they can make cheap chips, they can't yet make fast cheap chips).
        sdesol 2y ago
        Canada would be reluctant. So would Europe, South Asia and so forth. The biggest hurdle for China is the CCP. It is one thing to use physical products from China but relying on the CCP for knowledge may be a step too far for many nations.
        seanmcdirmid 2y ago
        Most people won't care if the products are useful. Chinese EVs, Chinese HSR, Chinese industrial robots, Chinese power tech, they are already selling. LLM isn't just a chatbot, it could be a medical device to help in areas without sufficiently trained doctors, for example.
- manbart 2y ago
  I'm pretty sure 3) is Meta's strategy currently
lolinder 2y ago
Regulatory capture. Persuade governments that AI is so dangerous that it must be regulated, then help write those regulations to ensure no one else can follow you to the ladder. That's half the point of OpenAI's game of pretending each new thing they make is too dangerous to release. It's half directed at investors to build hype, half at government officials to build fear.
- esafak 2y ago
  If Elon Musk's pals regulate away OpenAI because they declared their technology to be too dangerous that would be an ironic turn.
marcyb5st 2y ago
An algorithmic breakthrough IMHO. If someone finds out either how to get 10x performance per parameter or how to have a model actually model real causality (to some degree) they will have a moat. Also, I suspect that the next breakthrough will be kept under wraps and no papers will be published explaining it.
- andy_ppp 2y ago
  The people working on it will still be allowed to move companies and people talk to each other informally. These Chinese groups working on this with far fewer GPUs appear to be getting 10x results tbh. Maybe they have more GPUs than claimed but we’ll see.
Jyaif 2y ago
> curious if anyone has ideas how an LLM provider could moat AI By knowing a lot about me, like the details of my relationships, my interests, my work. The LLM would then be able to be better function than the other LLMs. OpenAI already made steps in that direction by learning facts about you. By offering services only possible by integrating with other industries, like restaurants, banks, etc... This take years to do, and other companies will take years to catch up, especially if you setup exclusivity clauses. There's lots of ways to slow down your competitors when you are the first to do something.
- chrisvalleybay 2y ago
  It is better to leave this up to the «LLM browser», than the LLM. Both because of privacy and portability.
mmaunder 2y ago
DeepSeek’s recent progress with V3 being a case in point which reportedly only cost $5.5M.
malux85 2y ago
Legislation, if you can’t compete on merit then regulate.
phodo 2y ago
One view is that it’s not first mover, but first arriver advantage. Whoever gets to AgI (the fabled city of gold or silver?, Ag pun intended) will achieve exponential gains from that point forward, and that serves as the moat in the limit. So you can think of it as buying a delayed moat, with an option price equivalent to the investment required until you get to that point in time. Either you believe in that view or you don’t. It’s more of an emotional / philosophical investment thesis, with a low probability of occurrence, but with a massive expected value.Meanwhile, consumers and the world benefit.
- nick3443 2y ago
  What if the AGI takes an entire data center to process a few tokens per second. Is the still a first-arriver advantage? Seems like the first to make it cheaper than an equivalent-cost employee (fully loaded incl hiring and training) will begin to see advantage.
  - scarmig 2y ago
    Depending on how powerful your model is, a few tokens per second per data center would still be extraordinarily valuable. It's not out of the realm of possibility that a next generation super intelligence could be trained with a couple hundred lines of pytorch. If that's the case, a couple tokens per second per data center is a steal.
  - usrusr 2y ago
    What if the next one to get there produces a similar service for 5% less? Race to the bottom. And would AI that is tied to some interface that provides lock-in even be qualified to be called general? I have trouble pointing my finger on it, but AGI and lock-in causes a strong dissonance in my brain. Would AGI perhaps strictly imply commodity? (assuming that more than one supplier exists)
  - phodo 2y ago
    Good point. It’s 2 conditions and both have to be true : - Arrive first - Use that first arrival to innovate with your new AGI pet / overlord to stay exponentially ahead
CamperBob2 2y ago
Genuinely curious if anyone has ideas how an LLM provider could moat AI. Patents. OpenAI already has a head start in the game of filing patents with obvious (to everybody except USPTO examiners), hard-to-avoid claims. E.g.: https://patents.google.com/patent/US12008341B2 https://patents.google.com/patent/US12008341B2
btilly 2y ago
Three ideas. All tried, none worked yet. First, get government regulation on your side. OpenAI has already looked for this, including Sam Altman testifying to Congress about the dangers of AI, but didn't get the regulations that they wanted. Second, put the cost of competing out of reach. Build a large enough and good enough model that nobody else can afford to build a competitor. Unfortunately a few big competitors keep on spending similar sums. And much cheaper sums are good enough for many purposes. Third, get a new idea that isn't public. For instance one on how to better handle complex goal directed behavior. OpenAI has been trying, but have failed to come up with the right bright idea.
minimaxir 2y ago
One way to build a technical moat is to build services which encourage lock-in, and therefore make it hard to switch to a competitor. Some of OpenAI's product releases help facilitate that: the Assistant API creates persistent threads that cannot be exported, and their File Search APIs build vector stores that cannot be exported.
brookst 2y ago
Create value at a higher layer and depend on tight integration to generate revenue and stickiness at both layers. See: Windows + MS Office. They don’t need to make a moat for AI, they need to make a moat for the OpenAI business, which they have a lot of flexibility to refactor and shape.
seanmcdirmid 2y ago
Create closed source models that are much better than the other ones, don't publish your tricks to obtaining your results, and then lock them down in a box that can't be tampered with? I hope it doesn't go that way. Alternatively, a model that takes a year and the output of a nuclear power plant to train (and then you can tell them about your tricks, since they aren't very reproducible).
shalmanese 2y ago
In the early days of Google, people believed there could be absolutely no moat in search because competition was just "one click away" and even Google believed this and deeply internalized this into their culture of technological dominance as the only path to survival. At the beginning of ride sharing, people believed there was absolutely no geographical moat and all riders were just one cheaper ride from switching so better capitalized incumbents could just win a new area by showering the city with discounts. It took Uber billions of dollars to figure out the moats were actually nigh insurmountable as a challenger brand in many countries. Honestly, with AI, I just instinctively reach for ChatGPT and haven't even bothered trying with any of the others because the results I get from OAI are "good enough". If enough other people are like me, OAI gets order of magnitudes more query volume than the other general purpose LLMs and they can use that data to tweak their algorithms better than anyone else. Also, current LLMs, the long term user experience is pretty similar to the first time user experience but that seems set to change in the next few generations. I want my LLM over time to understand the style I prefer to be communicated in, learn what media I'm consuming so it knows which references I understand vs those I don't, etc. Getting a brand new LLM familiar enough to me to feel like a long established LLM might be an arduous enough task that people rarely switch.
- barnabyjones 2y ago
  It is really unbelievable how much money companies will spend to avoid talking to and thoroughly understanding their users. OpenAI could probably learn a lot from interviewing 200 random people every 6 months and seeing what they use and why, but my guess is they would consider that frivolous.
  - ChadNauseam 2y ago
    what makes you think that they don’t?
  - Jabrov 2y ago
    UXR is a thing that all large companies invest in
- chrisvalleybay 2y ago
adamnemecek 2y ago
Spend money developing proprietary approaches to ML.

wmf 2y ago

You'd have to have either a breakthrough algorithm (that you keep secret forever) or proprietary training data.

deleted 2y ago

[deleted]

scarmig 2y ago

1) Push for regulations around data provenance. If you train on anything, you have to prove that you had the rights to train on it. This would kill all but the largest providers in the USA, though China would still pose a problem. You could work around that bit by making businesses and consumers liable for usage of models that don't have proof of provenance. 2) If you had some secret algorithm that substantially outperformed everyone, you could win if you prevented leakage. This runs into the issue that two people can keep a secret, but three cannot. Eventually it'll leak. 3) Keep costs exceptionally low, sell at cost (or for free), and flood the market with that, which you use to enhance other revenue streams and make it unprofitable for other companies to compete and unappealing as a target for investors. To do this, you have to be a large company with existing revenue streams, highly efficient infrastructure, piles of money to burn while competitors burn through their smaller piles of money, and the ability to get something of value from giving something out for free.

sdesol 2y ago
> China would still pose a problem Not if regulation prohibits LLMs from China, which isn't that far fetched to be honest. I think LLM will turn into a commodity product and if you want to dominate with a commodity product, you need to provide twice the value at half the cost. Open AI will need a breakthrough in reliability and/or inference cost to really create a moat.
- callc 2y ago
  If US regulation prohibits China from training LLMs? How? If you mean trying to stop GPUs getting to China, US already has tried that with specific GPU models, but China still gets them. Seems hard/impossible to do. Even if US and CCP were trying to stop Chinese citizens and companies doing LLM stuff
  - sdesol 2y ago
    Not prohibit training, but make it illegal for US companies to embed, use or distribute LLMs created by China. Basically the general consumer will need to go through the effort of use the LLM that they want, which we know means only a small fraction of people.
    - seanmcdirmid 2y ago
      That assumes that only the US is interested in using LLMs commercially, which isn't really true. Even if you can get America to sanction Chinese LLM use, you aren't even going to American allies to go along with that, let alone everyone else. China's biggest challenge ATM is that they do not yet economically produce the GPUs and RAM needed to train and use big models. They are still behind in semiconductors, maybe 10 or 20 years (they can make fast chips, they can make cheap chips, they can't yet make fast cheap chips).
      - sdesol 2y ago
        Canada would be reluctant. So would Europe, South Asia and so forth. The biggest hurdle for China is the CCP. It is one thing to use physical products from China but relying on the CCP for knowledge may be a step too far for many nations.
        seanmcdirmid 2y ago
        Most people won't care if the products are useful. Chinese EVs, Chinese HSR, Chinese industrial robots, Chinese power tech, they are already selling. LLM isn't just a chatbot, it could be a medical device to help in areas without sufficiently trained doctors, for example.
manbart 2y ago
I'm pretty sure 3) is Meta's strategy currently

lolinder 2y ago

Regulatory capture. Persuade governments that AI is so dangerous that it must be regulated, then help write those regulations to ensure no one else can follow you to the ladder. That's half the point of OpenAI's game of pretending each new thing they make is too dangerous to release. It's half directed at investors to build hype, half at government officials to build fear.

esafak 2y ago
If Elon Musk's pals regulate away OpenAI because they declared their technology to be too dangerous that would be an ironic turn.

marcyb5st 2y ago

An algorithmic breakthrough IMHO. If someone finds out either how to get 10x performance per parameter or how to have a model actually model real causality (to some degree) they will have a moat. Also, I suspect that the next breakthrough will be kept under wraps and no papers will be published explaining it.

andy_ppp 2y ago
The people working on it will still be allowed to move companies and people talk to each other informally. These Chinese groups working on this with far fewer GPUs appear to be getting 10x results tbh. Maybe they have more GPUs than claimed but we’ll see.

Jyaif 2y ago

> curious if anyone has ideas how an LLM provider could moat AI By knowing a lot about me, like the details of my relationships, my interests, my work. The LLM would then be able to be better function than the other LLMs. OpenAI already made steps in that direction by learning facts about you. By offering services only possible by integrating with other industries, like restaurants, banks, etc... This take years to do, and other companies will take years to catch up, especially if you setup exclusivity clauses. There's lots of ways to slow down your competitors when you are the first to do something.

chrisvalleybay 2y ago
It is better to leave this up to the «LLM browser», than the LLM. Both because of privacy and portability.

mmaunder 2y ago

DeepSeek’s recent progress with V3 being a case in point which reportedly only cost $5.5M.

malux85 2y ago

Legislation, if you can’t compete on merit then regulate.

phodo 2y ago

One view is that it’s not first mover, but first arriver advantage. Whoever gets to AgI (the fabled city of gold or silver?, Ag pun intended) will achieve exponential gains from that point forward, and that serves as the moat in the limit. So you can think of it as buying a delayed moat, with an option price equivalent to the investment required until you get to that point in time. Either you believe in that view or you don’t. It’s more of an emotional / philosophical investment thesis, with a low probability of occurrence, but with a massive expected value.Meanwhile, consumers and the world benefit.

nick3443 2y ago
What if the AGI takes an entire data center to process a few tokens per second. Is the still a first-arriver advantage? Seems like the first to make it cheaper than an equivalent-cost employee (fully loaded incl hiring and training) will begin to see advantage.
- scarmig 2y ago
  Depending on how powerful your model is, a few tokens per second per data center would still be extraordinarily valuable. It's not out of the realm of possibility that a next generation super intelligence could be trained with a couple hundred lines of pytorch. If that's the case, a couple tokens per second per data center is a steal.
- usrusr 2y ago
  What if the next one to get there produces a similar service for 5% less? Race to the bottom. And would AI that is tied to some interface that provides lock-in even be qualified to be called general? I have trouble pointing my finger on it, but AGI and lock-in causes a strong dissonance in my brain. Would AGI perhaps strictly imply commodity? (assuming that more than one supplier exists)
- phodo 2y ago
  Good point. It’s 2 conditions and both have to be true : - Arrive first - Use that first arrival to innovate with your new AGI pet / overlord to stay exponentially ahead

CamperBob2 2y ago

Genuinely curious if anyone has ideas how an LLM provider could moat AI. Patents. OpenAI already has a head start in the game of filing patents with obvious (to everybody except USPTO examiners), hard-to-avoid claims. E.g.: https://patents.google.com/patent/US12008341B2 https://patents.google.com/patent/US12008341B2

btilly 2y ago

Three ideas. All tried, none worked yet. First, get government regulation on your side. OpenAI has already looked for this, including Sam Altman testifying to Congress about the dangers of AI, but didn't get the regulations that they wanted. Second, put the cost of competing out of reach. Build a large enough and good enough model that nobody else can afford to build a competitor. Unfortunately a few big competitors keep on spending similar sums. And much cheaper sums are good enough for many purposes. Third, get a new idea that isn't public. For instance one on how to better handle complex goal directed behavior. OpenAI has been trying, but have failed to come up with the right bright idea.

minimaxir 2y ago

One way to build a technical moat is to build services which encourage lock-in, and therefore make it hard to switch to a competitor. Some of OpenAI's product releases help facilitate that: the Assistant API creates persistent threads that cannot be exported, and their File Search APIs build vector stores that cannot be exported.

brookst 2y ago

Create value at a higher layer and depend on tight integration to generate revenue and stickiness at both layers. See: Windows + MS Office. They don’t need to make a moat for AI, they need to make a moat for the OpenAI business, which they have a lot of flexibility to refactor and shape.

seanmcdirmid 2y ago

Create closed source models that are much better than the other ones, don't publish your tricks to obtaining your results, and then lock them down in a box that can't be tampered with? I hope it doesn't go that way. Alternatively, a model that takes a year and the output of a nuclear power plant to train (and then you can tell them about your tricks, since they aren't very reproducible).

shalmanese 2y ago

In the early days of Google, people believed there could be absolutely no moat in search because competition was just "one click away" and even Google believed this and deeply internalized this into their culture of technological dominance as the only path to survival. At the beginning of ride sharing, people believed there was absolutely no geographical moat and all riders were just one cheaper ride from switching so better capitalized incumbents could just win a new area by showering the city with discounts. It took Uber billions of dollars to figure out the moats were actually nigh insurmountable as a challenger brand in many countries. Honestly, with AI, I just instinctively reach for ChatGPT and haven't even bothered trying with any of the others because the results I get from OAI are "good enough". If enough other people are like me, OAI gets order of magnitudes more query volume than the other general purpose LLMs and they can use that data to tweak their algorithms better than anyone else. Also, current LLMs, the long term user experience is pretty similar to the first time user experience but that seems set to change in the next few generations. I want my LLM over time to understand the style I prefer to be communicated in, learn what media I'm consuming so it knows which references I understand vs those I don't, etc. Getting a brand new LLM familiar enough to me to feel like a long established LLM might be an arduous enough task that people rarely switch.

barnabyjones 2y ago
It is really unbelievable how much money companies will spend to avoid talking to and thoroughly understanding their users. OpenAI could probably learn a lot from interviewing 200 random people every 6 months and seeing what they use and why, but my guess is they would consider that frivolous.
- ChadNauseam 2y ago
  what makes you think that they don’t?
- Jabrov 2y ago
  UXR is a thing that all large companies invest in
chrisvalleybay 2y ago

adamnemecek 2y ago

Spend money developing proprietary approaches to ML.