>Gotta say in my opinion it didn't age well for a paper published in 2020 (but pre GPT-3). Agreed. The glaring issue with their argument falls out of their own

>Gotta say in my opinion it didn't age well for a paper published in 2020 (but pre GPT-3). Agreed. The glaring issue with their argument falls out of their own introduction. What is the "linguistic intent" of the sentence "When was Malala Yousafzai born"? Well, it falls right out of a natural language training corpus which will, with high likelihood, be followed up with her actual birthdate! So there is some non-trivial signal within the training corpus of linguistic intent. Capturing this intent is then just a computational problem. But the objective function of predicting the subsequent word has within the space of solutions the linguistic intent of the writers of the training corpus.