4 ms·

>AI similarly has powerful network effects. OpenAI's current business model is genius. By allowing free usage of chatGPT, they are getting millions of "free" em

>AI similarly has powerful network effects. OpenAI's current business model is genius. By allowing free usage of chatGPT, they are getting millions of "free" employees, all actively and eagerly providing more data and more training to the system. I never buy this. This story is repeated so often in so many places. You hear it about Tesla all the time "Their dataset is so big because they have all those cars out there no one could compete!". But what do we see every time an actual technical article about building something with AI comes up? Throwing a shit tonne of data at a model doesn't work. The number 1 most important thing about training neural nets is carefully grooming the training data so that the NN learns what you're actually trying to teach it, and doesn't just cheat your tests. So no, I don't believe the 5 millionth user typing "Write a monologue in the style of Benoit Blanc about the Johnny Depp/Amber Heard trial" is helping. I'm not saying there's no incumbency advantage - there clearly is, which is one of the reasons why Office is still dominant. But it's not about raw data, yes you need data, but at the scale that tech giants operate, multiple companies can have that scale of data, and certainly there's atleast a dozen companies that could scrape the entire web if they wanted to. There's also the simple fact that the team that built OpenAI is probably just... better at building AI products, so it's not really surprising that if they pull this off, they will continue to produce the best AI products, regardless of their data advantage.

code51 4y ago
We know that OpenAI was able to classify the prompts it received, at first privately - now publicly, so I'm thinking they are sorting out by use type, not prompt targets. They have probably learned a lot about new RLHF targets to focus on. It'll blind you to see the actual process of OpenAI if you focus on the noise in people's usage of ChatGPT.
impulser_ 4y ago
I doubt the data OpenAI receives from the usage of ChatGPT is even remotely close to the amount of data Google or Facebook handles that could be used to build a better language model. They have advantage over other AI startups, but I don't think they have much of an advantage over companies like Google and Facebook which both have massive amounts of data, money, and computing power. Which at the current state is the three most important things needed to build models that can operate at internet scale. OpenAI can never run ChatGPT at the scale needed for massive usage. They would drown in debt. Hence why ChatGPT is extremely slow, often crashes, and is protected by a rate limiter.
- techdemic 4y ago
  Agreed. Google processes about 8.5 billion search queries daily. Conversational voice assistants process about 120 million queries daily.
  - naasking 4y ago
    > Agreed. Google processes about 8.5 billion search queries daily. Misstyped queries of mostly the same stuff. Google's real data advantage here is gmail. Billions of real conversations between real people.
    - jen20 4y ago
      Your email account must look verify different to mine, which consists almost entirely of marketing spam and GitHub notifications… I don’t recall the last time I sent an email.
- TowerTall 4y ago
  > OpenAI can never run ChatGPT at the scale needed for massive usage. They would drown in debt. They have access to Microsoft's billions and the compute powers of Azure. The other day there was a HN post [1] about Microsoft has build a top 5 super computer. That computer is probably the computer OpenAI used to train their model on. [1] https://news.ycombinator.com/item?id=34307951 https://news.ycombinator.com/item?id=34307951
joxel 4y ago
This is an old narrative focused on the foibles of supervised learning. The data acquired by chatGPT and Tesla is extremely valuable even while unlabeled. It can be used for pretraining, both self and unsupervised. I guarantee you that having larger and more diverse datasets for pretraining provides a lot of advantages.
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]

code51 4y ago

We know that OpenAI was able to classify the prompts it received, at first privately - now publicly, so I'm thinking they are sorting out by use type, not prompt targets. They have probably learned a lot about new RLHF targets to focus on. It'll blind you to see the actual process of OpenAI if you focus on the noise in people's usage of ChatGPT.

impulser_ 4y ago

I doubt the data OpenAI receives from the usage of ChatGPT is even remotely close to the amount of data Google or Facebook handles that could be used to build a better language model. They have advantage over other AI startups, but I don't think they have much of an advantage over companies like Google and Facebook which both have massive amounts of data, money, and computing power. Which at the current state is the three most important things needed to build models that can operate at internet scale. OpenAI can never run ChatGPT at the scale needed for massive usage. They would drown in debt. Hence why ChatGPT is extremely slow, often crashes, and is protected by a rate limiter.

techdemic 4y ago
Agreed. Google processes about 8.5 billion search queries daily. Conversational voice assistants process about 120 million queries daily.
- naasking 4y ago
  > Agreed. Google processes about 8.5 billion search queries daily. Misstyped queries of mostly the same stuff. Google's real data advantage here is gmail. Billions of real conversations between real people.
  - jen20 4y ago
    Your email account must look verify different to mine, which consists almost entirely of marketing spam and GitHub notifications… I don’t recall the last time I sent an email.
TowerTall 4y ago
> OpenAI can never run ChatGPT at the scale needed for massive usage. They would drown in debt. They have access to Microsoft's billions and the compute powers of Azure. The other day there was a HN post [1] about Microsoft has build a top 5 super computer. That computer is probably the computer OpenAI used to train their model on. [1] https://news.ycombinator.com/item?id=34307951 https://news.ycombinator.com/item?id=34307951

joxel 4y ago

This is an old narrative focused on the foibles of supervised learning. The data acquired by chatGPT and Tesla is extremely valuable even while unlabeled. It can be used for pretraining, both self and unsupervised. I guarantee you that having larger and more diverse datasets for pretraining provides a lot of advantages.

deleted 4y ago
[deleted]
deleted 4y ago
[deleted]