Because while the model at baseline is a regression towards the mean, it is easily biased based on additional context - one of the big eye openers in research o

Because while the model at baseline is a regression towards the mean, it is easily biased based on additional context - one of the big eye openers in research over the past 18 months has been the impact and success of in context learning alone. So with an LLM you have a very broad basis that can be biased into a niche application in a single unit whereas with humans you have a fairly static set of capabilities that are widely and normally distributed across individual units. You can be lucky in finding the right unit for the task with humans that are extremely good at extending a given context. But you need to have been lucky with matching the unit's performance with the right context. A different unit will suck. But LLMs given enough coverage in the training set and enough additional context to bias towards that subset of the training set can be consistently good (and will likely continue to get better).