> [Regularisation] tends to be more robust, because of the bitter lesson. This seems circular? Over-parametrisation and stochastic gradient decent provides aut

> [Regularisation] tends to be more robust, because of the bitter lesson. This seems circular? Over-parametrisation and stochastic gradient decent provides automatic regularisation, and it turns out this regularisation is very good in practice. The bitter lesson is simply the a restatement of the empirical observation that this specific regularisation often works great, it doesn't come with any explanation.