At some point you will hit the interpolation threshold and your model will overfit perfectly to the training set. The gargantuan # of parameters is what buys y

At some point you will hit the interpolation threshold and your model will overfit perfectly to the training set. The gargantuan # of parameters is what buys you the generalization properties that everyone is interested in. A very reduced model may still look & sound competent on the surface, but extensive use by domain experts would quickly highlight the cost of this.