I tend to roll my eyes at a lot of the hype a fair bit, too, but the tricks that are getting media play aren't the only tricks this pony can do. They're just th

I tend to roll my eyes at a lot of the hype a fair bit, too, but the tricks that are getting media play aren't the only tricks this pony can do. They're just the one that are generating most the hype because that's what captures peoples imaginations. And understandably. The basic formula of futurism has always been and will always be, "Take X everyday thing and come up with a version of it that's fundamentally the same as how it works now, only with robots." But it's also pretty darned good at doing things that might not be as popularly relatable. There are some teams reporting that GPT4 can outperform Mechanical Turk labelers for many corpus labeling tasks, for example. That might then significantly reduce the cost of developing NLP products that may or may not use a GPT-series network as part of the production implementation. ChatGPT and GPT4 can also do a pretty darned good job at producing synthetic data. Other methods may be able to produce higher-quality data, but at typically at the cost of a much larger development effort. That could, for example, be useful for privacy-preserving machine learning where you want to develop a model that's tailored to a specific task, but you don't want to (or, if you're lucky enough to live and work in a sensible regulatory regime, maybe even can't) use actual sensitive data to train the model. The underlying embedding model may or may not outperform BERT for specific tasks. I haven't poked at that issue much, but it wouldn't surprise me.