Well really, what people in the field felt and saw was that on metrics for measuring tasks, what had happened in computer vision with giant jumps in performance

Well really, what people in the field felt and saw was that on metrics for measuring tasks, what had happened in computer vision with giant jumps in performance for tasks previously considered hard to ever get to human level (like identifying cats in photos), then came to pass for NLP: Tasks and metrics we never thought we'd see even close to human level without some new theories or approaches to thinking and language, jumped up overnight. The metrics and applications of NLP before 2017 heavily relied on operating on words, tokens and adding in rules-based(GOFAI/expert system) modifications to do things like old-style chatbots. So many tasks now done by 355m tiny models locally had dismal performance, like repunctuation, translating faithfully, really - what turned out to be basically "easy" stuff for a network to learn. And then add on RLHF and instruct tuning which is bizzarely more capable than anybody reading the RL/T5 papers that lead to these largely expected at the time. It's the gap in researcher experiences vs assumptions as of 2017 that lead to this description imo. If you had projected the performance from 2012 to 2017 in NLP, you'd have about 3 major advancements, around word embeddings, and there wasn't much room for innovation there (as it turned out, although obviously semantic search and stuff followed to an extent, it kind of existed as "IR"). But from 2017 to 2023, we've had 30 of similar calibre, and they've stacked (GPT-2, then GPT-3, then instruct/RLHF basically). The real point, as I see it, is that if we have the same jump - we outperform expectations by an order of magnitude - we'd be in trouble. Now maybe the expectations have adjusted, but I can tell you even as of 1 year ago, many NLP labs still disliked and ignored GPT models in favour of clearly relatively obsolete/now academic-curiosity projects.