2 ms·
You can't use humans to manually refine a dataset on the scale of GPT-3 or DALL-E Clip was trained on 400,000,000 images, GPT is roughly 180B tokens, at 1-2 to
by FrenchDevRemote 4y ago
You can't use humans to manually refine a dataset on the scale of GPT-3 or DALL-E
Clip was trained on 400,000,000 images, GPT is roughly 180B tokens, at 1-2 tokens per word, that's 120,000,000,000 words.
- pshc 4y agoAt least cleaning it up is an embarrassingly parallel problem, so if you had the resources to throw incentives at millions of casual gamers, you might make a nice dent on Clip.
- zanderwohl 4y agoAlternatively, making a captcha where half the data is unlabeled, and half is labeled, forcing users to categorize data for you as they log into accounts.