4 ms·

Seems that given enough compute everyone can build a near-SOTA LLM. So what is this craze about securing AI dominance?

littlestymaar 2y ago
> everyone Let's not disrespect the team working on Qwen, these folks have shown that they are able to ship models that are better than everybody else's in the open weight category. But fundamentally yes, OpenAI has no other moat than the ChatGPT trademark at this point.
- _1 2y ago
  It just shows that they're unimaginative and good at copying.
  - amazingamazing 2y ago
    What’s wrong with copying?
    - ralusek 2y ago
      If they can only copy, which I'm not saying is the case, then their progress would be bounded by whatever the leader in the field is producing. In much the same way with an LLM, if it can only copy from its training data, then it's bounded by the output of humans themselves.
- miohtama 2y ago
  They have the moat of being able to raise large funding rounds than everybody else: Access to capital.
  - littlestymaar 2y ago
    But access to capital is highly dependent on how interesting you look to investors. If you don't manage to create a technological gap when you are better funded than your competitors then your attractivity will start being questioned. They have dilapidated their “best team” asset with internal drama, and now that they see their technological advance being demolished by competitors, I'm not too convinced in their prospect for a new funding round unless they show that they can make money out of the consumer market which is where their branding is an unmatched asset (in which case it's not even clear that investing in being the state of the art model is a good business decision).
  - tempusalaria 2y ago
    many of these labs have more funding in theory than OpenAI. FAIR, GDM, Qwen all are subsidiaries of companies with $10s of billions in annual profits.
  - seccode 2y ago
    Maybe truth here, but also Microsoft didn't lead their latest round, which isn't a great sign for their moat
  - lmm 2y ago
    Do they have more access to capital than the CCP, if the latter decided to put its efforts behind Alibaba on this? Genuine question.
- nxobject 2y ago
  And perhaps exclusive archival content deals from publishers – but that probably works only in an American context.
- miki123211 2y ago
  > But fundamentally yes, OpenAI has no other moat than the ChatGPT trademark at this point. That's like saying that CocaCola has no other moat than the CocaCola trademark. That's an extremely powerful moat to have indeed.
  - littlestymaar 2y ago
    There's a big difference though Coca Cola makes its money from customers out its brands, OpenAI doesn't and it's not clear at all that there is monetization potential in that direction. Their business case was about being the provider of artificial intelligence to other businesses, not to monetize ChatGPT. There my be an opportunity for a pivot, that would include getting rid of the goal of having the most performant model, cutting training cost to the minimum, and be profitable from there, but I'm not sure it would be enough to justify their $157 Billion valuation.
  - anon373839 2y ago
    Actually, they don’t have the trademark (yet). USPTO rejected the application: > [Trademark] Registration is refused because the applied-for mark merely describes a feature, function, or characteristic of applicant’s goods and services. https://tsdr.uspto.gov/documentviewer?caseId=sn97733261&docId=FREF20240209093901&linkId=5#docIndex=4&page=1 https://tsdr.uspto.gov/documentviewer?caseId=sn97733261&docI...
  - sundarurfriend 2y ago
    It's not nothing, but it's very different from the CocaCola case. Coke is built on getting people used to the specific taste, especially when they're young, and taste is an almost subconscious/System1 thing where people seek comfort in the familiar. ChatGPT is more of a System2 thing, people don't really care whether the answer they seek comes from ChatGPT or from AskJeeves, they just want answers of reasonably similar quality. There's still value in branding and name recognition, but the switching costs are much lower here.
    - littlestymaar 2y ago
      People can't tell coke apart from Pepsi in blind tests though, the perceived difference when people are getting asked “Is Pepsi ok?” is 100% conditioned by the attachment to the brand.
deadbabe 2y ago
AI dominance is secured through legal and regulatory means, not technical methods. So for instance, a basic strategy is to rapidly develop AI and then say “Oh wow AI is very dangerous we need to regulate companies and define laws around scraping data” and then make it very difficult for new players to enter the market. When a moat can’t be created, you resort to ladder kicking.
- Onavo 2y ago
  I believe in china they have been trying to make all data training data https://www.forbes.com/councils/forbestechcouncil/2024/04/18/china-treats-data-as-an-asset-heres-why-your-business-should-too/ https://www.forbes.com/councils/forbestechcouncil/2024/04/18...
  - yazzku 2y ago
    Unlike in the US?
- greenavocado 2y ago
  Operation Chokepoint 2.0 Relevant https://x.com/benaverbook/status/1861511171951542552 https://x.com/benaverbook/status/1861511171951542552
nextworddev 2y ago
1) spreading AI dominance FUD is a good way to get government subsidies 2) not exactly everyone with compute can make LLMs, they need data. Conveniently, the U.S. has been supplying infinite tokens to China through Tiktok.
- nemothekid 2y ago
  >Conveniently, the U.S. has been supplying infinite tokens to China through Tiktok How is this not FUD? What competitive advantage is China seeing in LLM training through dancing videos on TikTok?
  - nextworddev 2y ago
    you get video tokens through those seemingly dumb tiktok shorts
    - nl 2y ago
      Of all the types of tokens in the world video is not the one that comes to mind as having a shortage. By setting a a few thousand security cameras in various high traffic places you can get almost infinite footage. Instagram, Youtube and Snapchat have no shortage of data too.
      - nextworddev 2y ago
        except 1) tiktok is video stream data many orders of magnitude larger than any security cam data, that's attached to real identity 2) china doesn't have direct access to Instagram reels and shorts, so yeah
        nl 2y ago
        Why does tying it to identity help LLM training? It's pretty unclear that having orders of magnitude more video data of dancing is useful. Diverse data is much useful!

> everyone Let's not disrespect the team working on Qwen, these folks have shown that they are able to ship models that are better than everybody else's in the open weight category. But fundamentally yes, OpenAI has no other moat than the ChatGPT trademark at this point.

_1 2y ago
It just shows that they're unimaginative and good at copying.
- amazingamazing 2y ago
  What’s wrong with copying?
  - ralusek 2y ago
    If they can only copy, which I'm not saying is the case, then their progress would be bounded by whatever the leader in the field is producing. In much the same way with an LLM, if it can only copy from its training data, then it's bounded by the output of humans themselves.
miohtama 2y ago
They have the moat of being able to raise large funding rounds than everybody else: Access to capital.
- littlestymaar 2y ago
  But access to capital is highly dependent on how interesting you look to investors. If you don't manage to create a technological gap when you are better funded than your competitors then your attractivity will start being questioned. They have dilapidated their “best team” asset with internal drama, and now that they see their technological advance being demolished by competitors, I'm not too convinced in their prospect for a new funding round unless they show that they can make money out of the consumer market which is where their branding is an unmatched asset (in which case it's not even clear that investing in being the state of the art model is a good business decision).
- tempusalaria 2y ago
  many of these labs have more funding in theory than OpenAI. FAIR, GDM, Qwen all are subsidiaries of companies with $10s of billions in annual profits.
- seccode 2y ago
  Maybe truth here, but also Microsoft didn't lead their latest round, which isn't a great sign for their moat
- lmm 2y ago
  Do they have more access to capital than the CCP, if the latter decided to put its efforts behind Alibaba on this? Genuine question.
nxobject 2y ago
And perhaps exclusive archival content deals from publishers – but that probably works only in an American context.
miki123211 2y ago
> But fundamentally yes, OpenAI has no other moat than the ChatGPT trademark at this point. That's like saying that CocaCola has no other moat than the CocaCola trademark. That's an extremely powerful moat to have indeed.
- littlestymaar 2y ago
  There's a big difference though Coca Cola makes its money from customers out its brands, OpenAI doesn't and it's not clear at all that there is monetization potential in that direction. Their business case was about being the provider of artificial intelligence to other businesses, not to monetize ChatGPT. There my be an opportunity for a pivot, that would include getting rid of the goal of having the most performant model, cutting training cost to the minimum, and be profitable from there, but I'm not sure it would be enough to justify their $157 Billion valuation.
- anon373839 2y ago
  Actually, they don’t have the trademark (yet). USPTO rejected the application: > [Trademark] Registration is refused because the applied-for mark merely describes a feature, function, or characteristic of applicant’s goods and services. https://tsdr.uspto.gov/documentviewer?caseId=sn97733261&docId=FREF20240209093901&linkId=5#docIndex=4&page=1 https://tsdr.uspto.gov/documentviewer?caseId=sn97733261&docI...
- sundarurfriend 2y ago
  It's not nothing, but it's very different from the CocaCola case. Coke is built on getting people used to the specific taste, especially when they're young, and taste is an almost subconscious/System1 thing where people seek comfort in the familiar. ChatGPT is more of a System2 thing, people don't really care whether the answer they seek comes from ChatGPT or from AskJeeves, they just want answers of reasonably similar quality. There's still value in branding and name recognition, but the switching costs are much lower here.
  - littlestymaar 2y ago
    People can't tell coke apart from Pepsi in blind tests though, the perceived difference when people are getting asked “Is Pepsi ok?” is 100% conditioned by the attachment to the brand.

deadbabe 2y ago

AI dominance is secured through legal and regulatory means, not technical methods. So for instance, a basic strategy is to rapidly develop AI and then say “Oh wow AI is very dangerous we need to regulate companies and define laws around scraping data” and then make it very difficult for new players to enter the market. When a moat can’t be created, you resort to ladder kicking.

Onavo 2y ago
I believe in china they have been trying to make all data training data https://www.forbes.com/councils/forbestechcouncil/2024/04/18/china-treats-data-as-an-asset-heres-why-your-business-should-too/ https://www.forbes.com/councils/forbestechcouncil/2024/04/18...
- yazzku 2y ago
  Unlike in the US?
greenavocado 2y ago
Operation Chokepoint 2.0 Relevant https://x.com/benaverbook/status/1861511171951542552 https://x.com/benaverbook/status/1861511171951542552

nextworddev 2y ago

1) spreading AI dominance FUD is a good way to get government subsidies 2) not exactly everyone with compute can make LLMs, they need data. Conveniently, the U.S. has been supplying infinite tokens to China through Tiktok.

nemothekid 2y ago
>Conveniently, the U.S. has been supplying infinite tokens to China through Tiktok How is this not FUD? What competitive advantage is China seeing in LLM training through dancing videos on TikTok?
- nextworddev 2y ago
  you get video tokens through those seemingly dumb tiktok shorts
  - nl 2y ago
    Of all the types of tokens in the world video is not the one that comes to mind as having a shortage. By setting a a few thousand security cameras in various high traffic places you can get almost infinite footage. Instagram, Youtube and Snapchat have no shortage of data too.
    - nextworddev 2y ago
      except 1) tiktok is video stream data many orders of magnitude larger than any security cam data, that's attached to real identity 2) china doesn't have direct access to Instagram reels and shorts, so yeah
      - nl 2y ago
        Why does tying it to identity help LLM training? It's pretty unclear that having orders of magnitude more video data of dancing is useful. Diverse data is much useful!