> it has encountered the patterns you’re picking out in lots and lots of text written by humans? In pre-training data, yes There are post-training datasets, w

> it has encountered the patterns you’re picking out in lots and lots of text written by humans? In pre-training data, yes There are post-training datasets, where the weights are changed to conform to human preference. These datasets are created by groups of thousands of people all following a 40-page guide, and these guides have example. People over-index on these examples and so sample sentences with these structures are over represented in these datasets and used for post-training.