I’ve repeatedly seen the following sequence, from acquaintances and randos on twitter: 1) GPT-3 is actually really dumb. The people posting impressive results

I’ve repeatedly seen the following sequence, from acquaintances and randos on twitter: 1) GPT-3 is actually really dumb. The people posting impressive results are either cherry picking top 1% best results or just lying. 2) (time passes, they figure it out) Okay, I get it, you have to prompt it right, then it really does work well. But that’s so burdensome! 3) (rewriting history) Of course you have to prompt it right, how else is it supposed to know exactly what sort of task I expect it to be doing? — If, for example, you see GPT-3 say something extremely stupid rather than just a bit off, you’ve almost certainly primed it to think it’s supposed to be saying stupid, random things, by giving it a nonsense prompt.