3 ms·

They have absolutely no moat. Trained on published architecture using publicly available data. By the end of 2023 all companies including meta, Google, and Amaz

They have absolutely no moat. Trained on published architecture using publicly available data. By the end of 2023 all companies including meta, Google, and Amazon will be have similar models. Only moat they currently have is tons of specially curated data for rlhf which other companies are catching up.

revelio 3y ago
GPT-4 is pretty clearly not just public data + public papers. It's a lot of optimizations and other things on top of what is itself a lot of optimizations and other things that led to GPT-3. We're going to have to wait and see if OpenAI has a moat. You could have argued in 2001 that Google had no real moat. All they were doing was indexing public data with an algorithm they'd already published in a research paper, after all. But it turned out to be a deep space with an endless potential for optimizations and improvements, combined with the need for great expertise in scalable computations and ad optimization that other companies just weren't able to catch up with even after trying hard for years.
- wodenokoto 3y ago
  We’ve seen “sweat-shops” in African countries where hordes of people are annotating video for training self driving cars. Id be surprised if the same thing isn’t true for chatgpt.
  - zamnos 3y ago
    The RLHF data has to come from somewhere.
    - typon 3y ago
      It came from Kenyan annotators for ChatGPT. The next dataset can come from ChatGPT users.
  - deleted 3y ago
    [deleted]
- hgsgm 3y ago
  A head start is not a moat. It's faster to follow than lead.
bcherny 3y ago
Effectively structuring (publicly available) data for training is a moat. So is the details of the model, and the UX of how the model is presented to users. Consider why Google won in search for so long, despite anyone being able to write their own web crawler in an afternoon.
tim333 3y ago
They presumably have smart people working there to get the results they have so far. By the end of 2023 they will have moved ahead while the competition try to catch up to where they are now.

revelio 3y ago

GPT-4 is pretty clearly not just public data + public papers. It's a lot of optimizations and other things on top of what is itself a lot of optimizations and other things that led to GPT-3. We're going to have to wait and see if OpenAI has a moat. You could have argued in 2001 that Google had no real moat. All they were doing was indexing public data with an algorithm they'd already published in a research paper, after all. But it turned out to be a deep space with an endless potential for optimizations and improvements, combined with the need for great expertise in scalable computations and ad optimization that other companies just weren't able to catch up with even after trying hard for years.

wodenokoto 3y ago
We’ve seen “sweat-shops” in African countries where hordes of people are annotating video for training self driving cars. Id be surprised if the same thing isn’t true for chatgpt.
- zamnos 3y ago
  The RLHF data has to come from somewhere.
  - typon 3y ago
    It came from Kenyan annotators for ChatGPT. The next dataset can come from ChatGPT users.
- deleted 3y ago
  [deleted]
hgsgm 3y ago
A head start is not a moat. It's faster to follow than lead.

bcherny 3y ago

Effectively structuring (publicly available) data for training is a moat. So is the details of the model, and the UX of how the model is presented to users. Consider why Google won in search for so long, despite anyone being able to write their own web crawler in an afternoon.

tim333 3y ago

They presumably have smart people working there to get the results they have so far. By the end of 2023 they will have moved ahead while the competition try to catch up to where they are now.