> One of the strangest things in AI, to me, is that you can average the weights of multiple different models to create a single model that’s better than any of

> One of the strangest things in AI, to me, is that you can average the weights of multiple different models to create a single model that’s better than any of the individuals. I think this is one of the least strange things in AI. All you're doing is taking N overfitted models (unlikely to be overfit in the same way) and then asserting that the average of those predictions is probably not overfitted as much (regularization). Overfitting as a concept is not restricted to some number of dimensions.