4 ms·
Recently heard a resident machine learning expert describe GPT-3 as 'not revolutionary in the slightest' or something like that.
by Tycho 6y ago
Recently heard a resident machine learning expert describe GPT-3 as 'not revolutionary in the slightest' or something like that.
- minimaxir 6y agoIt's not revolutionary, just a typical-but-notable iterative step in NLP. Which is fine! I wrote a blog post on that a few months ago after playing a bit with GPT-3, and it holds up. https://news.ycombinator.com/item?id=23891226 https://news.ycombinator.com/item?id=23891226
- jokethrowaway 6y agoIt's not, but it showed that we can get a magnitude better results by adding a magnitude more data. To be honest, it's not where I'd like to see efforts in the field go. Not because I'm afraid of AI taking over, but because I'd rather have humans recreate something comparable to a human brain (functionality wise).
- visarga 6y agoWho knows, maybe in a few years you will be amazed at the new universal transformer chip that runs on 20W of power and can do almost any task. No need for retraining, just speak to it, show it what you need. Even prompt engineering has been automated (https://arxiv.org/abs/2101.00121 https://arxiv.org/abs/2101.00121) so no more mystery. So much for the new hot job of GPT prompt engineer that would replace software dev.
- asbund 6y agoI was skeptical before, but now i open to this idea
- jokethrowaway 6y agowe've been making these since the beginning of time, we call them humans
- commonturtle 6y agoHuman's want health insurance and 40 hours work-weeks. The super-smart AGI that will exist 20 years from now won't.
- FL33TW00D 6y agoI think that they're correct saying that GPT-3 isn't revolutionary, since it just demonstrates the power of scaling. However I would argue that the underlying architecture, the Transformer (GP(T)), is/was/will be revolutionary.
- visarga 6y agoIt's revolutionary in costs, and delivers for every dollar spent.