3 ms·
Why is there so much focus on just the language generation aspect of this model? The language generation examples are to show how well the model generalized the
by baron_harkonnen 8y ago
Why is there so much focus on just the language generation aspect of this model? The language generation examples are to show how well the model generalized the language it's trained on, but the whole point of releasing the pretrained model is that this model also:
>achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization — all without task-specific training
Every comment I see here and else where is strictly about automatically generating comments, but the real loss of having this model kept private is that it would allow anyone to create very, very good NLP projects right away.
The big problem in ML/"AI" now and going forward is that there is an extreme asymmetry emerging in tech. The start of the last major tech boom was because of "disruption". Any clever hacker with a laptop and developer skills could create a product that would topple a giant. Likewise anyone could learn to do anything that needed to be done on their own computer in their own time. Now that's changing, to even play with these new tools you need special access and corporate support.
Releasing the pretrained model to the public would allow a range of groups that cannot afford to build a model such as this to experiment on their own and achieve state of the art NLP results, not just in language generation but in classification, translation, summarization etc. Some kid in the middle of know where could download this and start messing with ideas about building a chat bot.
The threat is not that some rogue actor will create evil spam bots, but that some small scrappy startup would leverage the same, currently exclusive tools, of large technology companies.
The reason why democratizing AI is important is not because we can better fight off some vague threat, but so that, at least for the near term, individuals and small groups can still reasonably compete with major players in the ML space. And of course the fact that someone who claims openness and that they exist to "ensure AGI's benefits are as widely and evenly distributed as possible." Can arbitrarily choose not to do go the other direction just shows that tech is moving in a direction where "disruption" is going to become increasingly less possible, and the future of the industry is going to be controlled by a handful of major players.