2 ms·
I think we'll need about 10 times as many parameters and text on the web... Or so. How long would that take?
by sva_ 4y ago
I think we'll need about 10 times as many parameters and text on the web... Or so. How long would that take?
- visarga 4y agoA couple of months ago OpenAI created a model called Whisper that can transcribe speech efficiently and with high quality in 97 languages. It might be used to transcribe the whole YouTube, someone estimated about 12T tokens. GPT-3 used about 0.5T tokens for training. https://twitter.com/HenriLemoine13/status/1572846452895875073 https://twitter.com/HenriLemoine13/status/157284645289587507... If you add the visual data you get another order of magnitude increase in size. I think this training will teach the model procedural knowledge - how to do things step by step, and that will have a huge impact in robotics and software agents.