Came here for this... there isn't anything clearly "inherent" about these limitations, except to the degree that they are part of how some folks define the labe

Came here for this... there isn't anything clearly "inherent" about these limitations, except to the degree that they are part of how some folks define the label "GPT-3". If one packed GPT-3's structural architecture into a reinforcement learning system to grant memory, or pushed it through a compression system that made it cheaper to train or run, would you say GPT-3 had transcended its limitations, or just that you created something new? The fact that this question is semantic and uninteresting is why "Method / Model X doesn't do everything" posts don't progress the scientific conversation.