In Sutton's essay he's talking about more general techniques vs ones that build in expert knowledge. In the case of this paper it's the same technique in both t

In Sutton's essay he's talking about more general techniques vs ones that build in expert knowledge. In the case of this paper it's the same technique in both the fine-tuned and "general" cases. So Sutton's argument doesn't apply here. This is entirely about training data. The big flaw in this paper, to me, is that no one knows what GPT-4 has been trained on. It could include all the same training sources as the specialized models for all we know. If that's the case, it's a bit curious that it requires prompting to get the accuracy, but again, we'll never know. I'd go so far as to say this paper isn't even giving any useful information here, aside from the fact that GPT-4 can be made to work well in these specific domains with prompting.