Because —and this serves as a reply to the sibling comment too— GPT-3 is just a relatively simple construct pumped full of data and compute. After training it i

Because —and this serves as a reply to the sibling comment too— GPT-3 is just a relatively simple construct pumped full of data and compute. After training it is relatively static, and while output might appear novel in some cases I think it is far more likely that GPT-3 is not more imaginative than most people suspect, but that people are less imaginative than most people expect. I’m not knocking your work, it certainly may be the first (or near enough to) instance that techniques are applied in the way that they are; but in my understanding and opinion, the very fact that Copilot can output code that seems novel and groundbreaking suggests the opposite. Occam’s razor.