> It depends on whether you'd like the model to learn casual or correlative relationships. I expect that in the practical limit of scale achievable, the regula

> It depends on whether you'd like the model to learn casual or correlative relationships. I expect that in the practical limit of scale achievable, the regularization pressure inherent to the process of training these models converges to https://en.wikipedia.org/wiki/Minimum_description_length https://en.wikipedia.org/wiki/Minimum_description_length and the correlative relationships become optimized away, leaving mostly true causal relationships inherent to data-generating process.