I think some of these examples are interesting because they show that GPT-2 was trained on data (web sites) that were optimized to be interesting, rather than l

I think some of these examples are interesting because they show that GPT-2 was trained on data (web sites) that were optimized to be interesting, rather than lists of facts or logical inferences. Hmm, now I wonder if you could take GPT-2, add on a little bit of training on some boring rote lists of logical inferences, and get something useful out of it.