They don't really outperform human experts on real tasks yet (no matter what some superGLUE or other benchmark shows); but in general, once a system can solve a

They don't really outperform human experts on real tasks yet (no matter what some superGLUE or other benchmark shows); but in general, once a system can solve a particualr task well, it would be plausible to outperform human experts simply by not making random errors. If we have multiple human experts annotate a NLP task and measure inter-annotator agreement, it will be far from 100%; part of that will be genuine disagreements or fuzzy gray area, but part of the identified differences will be simply obviously wrong answers given by the experts - everyone makes mistakes. The same applies for many other domains - business process automation, data entry, etc; no employee will produce error-free output in a manual process, no matter how simple and unambiguous the task is. And for simpler tasks the computer can easily make less mistakes than a human - especially if you measure the human reliability not for a few minutes of focus, but for a whole tedious working day.