3 ms·
I suppose I could have guessed that GPT-2 is a plausible name for the predecessor of GPT-3, and the other predicted predecessor name (PTB) is not really correct
by Imnimo 6y ago
I suppose I could have guessed that GPT-2 is a plausible name for the predecessor of GPT-3, and the other predicted predecessor name (PTB) is not really correct. But I'm still surprised that it can hit a believable tone about its own capabilities. I don't think I could have done that if you had asked me before GPT-2 was published (which is essentially what we're doing if GPT-3's training data predates GPT-2). I think I would have under-estimated and it would be immediately clear to a reader familiar with GPT-3 that I didn't actually know what it could or could not do.
- nullc 6y ago> I think I would have under-estimated Keep in mind, the prompt specified that it was the biggest thing since bitcoin and had disruptive potential. Given that you wouldn't write that it sucks. It also did underestimate its own performance: > ... nor do its predicted replies tend to be relevant or even grammatically correct. A prototype that had predicted replies that were convincing in most cases would be much more impressive than the GPT-3 I describe here, although that would probably require many years of training and many iterations of improvements on the model It essentially adopts a common convention for hype articles: A pile of breathless exaggerated hype and then a brief conclusion that suggests that the approach hardly works at the current time, at least if you're reading between the lines. :) I think it's more surprising that it didn't stray further into fantasy. I'd be interested in seeing the author's rejected samples.