3 ms·
GPT-3 is not a 100x improvement over GPT-2.
by minimaxir 6y ago
GPT-3 is not a 100x improvement over GPT-2.
- TomMarius 6y agoAre you sure? When I look at the GPT-2 examples, and remember my experience with it when you could try on talktotransformer.com, I would say it is. GPT-2 would start to ramble incomprehensibly almost every time. It would forget context after a few words. It would get into loops. To make something resembling a conversation you had to heavily influence it.
- minimaxir 6y agoIt's better, just not 100x better, which is a big difference in practice. Quantitatively about 5x of the output is more useful in my usage, which is not enough to upend society.
- dflock 6y agoDoesn't really matter, for the purposes of this question.