The global optimum would be taken in reference to the training data (because that's all you have to set the weights). Unless the training data represents all re

The global optimum would be taken in reference to the training data (because that's all you have to set the weights). Unless the training data represents all real world data perfectly, fully optimizing for it will pessimize the model in relation to some set of real world data.