I'm not sure if you're being tongue-in-cheek, but: * Training on Sci-hub or book torrents would probably get them sued, so that's right out. * The point of tr

I'm not sure if you're being tongue-in-cheek, but: * Training on Sci-hub or book torrents would probably get them sued, so that's right out. * The point of training the model on reddit-linked-articles with 3+ karma (not reddit comments, as the article suggests) is to filter out worthless content (spam or non-text pages) while still getting a large, diverse sampling of human writing. They're training huge models and they need as much data as possible to do so. * Every native english speaker is a "domain expert" for the purpose of evaluating the model's results. The point at which we need subject-matter experts to evaluate the quality of neural-net generated research papers is many decades away; the excitement around GPT-2 is that it can generate coherent English sentences at all.