In simpler worlds, there's a variety of criteria such as Akaike's Information Criterion (AIC) and the Bayesian Information Criterion (BIC). Neither applies dire

In simpler worlds, there's a variety of criteria such as Akaike's Information Criterion (AIC) and the Bayesian Information Criterion (BIC). Neither applies directly to NN optimization because they amount to "every additional parameter costs _this_ much". The more philosophical Minimal Description Length (MDL) principles and friends might yield something applicable for NN training, but hasn't yet AFAIK.