GPT2 is trained on huge amounts of text. it contains every book people haven't read, and can more or less stitch pieces together seamlessly. This is exciting fr

GPT2 is trained on huge amounts of text. it contains every book people haven't read, and can more or less stitch pieces together seamlessly. This is exciting from a research perspective but the hype is out of control already. Despite breaking previosu benchmarks, language benchmarks themselves are not great ways to describe language. I get the excitement but this overselling to the general public is soon gonna lead to a trough of disillusionment.