Usually text transformers use reddit posts for some training data alongside wikipedia and others. In The Pile, a large text dataset, Hackernews is included too

Usually text transformers use reddit posts for some training data alongside wikipedia and others. In The Pile, a large text dataset, Hackernews is included too if I'm not mistaken. https://media.arxiv-vanity.com/render-output/5356367/relative_bpb_gpt3.png https://media.arxiv-vanity.com/render-output/5356367/relativ...