4 ms·
>There are only a handful of companies with enough data to be able to train artificial intelligence algorithms. https://commoncrawl.org/ https://commoncrawl.or
by Imnimo 4y ago
>There are only a handful of companies with enough data to be able to train artificial intelligence algorithms.
https://commoncrawl.org/ https://commoncrawl.org/
- janalsncm 4y agoData isn’t the only bottleneck. Training models at scale is an expensive and difficult engineering task. It’s something that the “data is the new gold” people always seem to overlook. And then even if you have the data and the engineering talent, it takes competent leadership to deploy a product and shepherd it to success. (Looking at Google’s graveyard here)
- Imnimo 4y agoTotally agree. Data is a problem when you need millions of labels, but when your training corpus is just "all the text I can find on the internet" and the training objective is self-supervised, it's not what's stopping people. But as you say, actually training a model of this size is really, really tricky. You can't just grab a few ML engineers off the street, write a check to AWS for some compute and have your own GPT-3.