SOTA language models are trained with reinforcement learning now too, so the model distributions can be far removed from the underlying data distribution. Not t

SOTA language models are trained with reinforcement learning now too, so the model distributions can be far removed from the underlying data distribution. Not that I think your overall point is wrong (it's obviously a very strong bias), but things are getting more complex with time.