Every time an article talks about training/finetuning/improving GPT-3, you know it's GPT-3 talking about itself. There's no option to do those things for GPT-3

Every time an article talks about training/finetuning/improving GPT-3, you know it's GPT-3 talking about itself. There's no option to do those things for GPT-3 outside of OpenAI. More broadly, you can be very sure that a text about GPT-3 originated from GPT-3 if it makes more sense when you replace "GPT-3" by "GPT-2" everywhere. GPT-3 naturally has no knowledge of itself because all related text was written after its training. The closest is GPT-2 (for which there is apparently plenty of corpus to sample from), so that's the content that GPT-3 writes about when forced to talk about itself.