I'll come out and say what's on many practicioners' minds: It could very well be that generations of academics have been WRONG about the relationship between t

I'll come out and say what's on many practicioners' minds: It could very well be that generations of academics have been WRONG about the relationship between the number of parameters (or complexity) of a model and its ability to generalize to new, previously unseen data, particularly when the data is drawn from many naturally occurring distributions -- as opposed to, say, distributions randomly chosen from the space of all possible distributions, as assumed by many theoretical frameworks (e.g., No Free Lunch theorem). It could be that generations of students have been taught The Wrong Thing™. In many cases we must increase, not decrease, model complexity to improve generalization!