Seems to be entirely a conjecture ... not based on even limited empirical testing. Which then links to the elephant in the room, which is that the theoretical b

Seems to be entirely a conjecture ... not based on even limited empirical testing. Which then links to the elephant in the room, which is that the theoretical basis of why GPT's work so well is not well understood in the first place. One of the downsides of having a field be so empirically driven is that theoretical arguments suggesting improvements just don't have much value until someone actually tests it and shows that it works (and not just tests it, but tests it at scale).