It's not inherently true. Technically, deep learning is essentially any neural network model with hidden layers (i.e., one layer in between the input layer and

It's not inherently true. Technically, deep learning is essentially any neural network model with hidden layers (i.e., one layer in between the input layer and the output layer). You could have a "deep learning" model with a couple dozen parameters, perhaps. But at that end of the scale, most people would probably reach for other approaches that are more easily interpretable (e.g., logistic regression, random forest). So in practice, yes, virtually any deep learning model you see out there in the wild, even most "toy examples" used to teach machine learning, are going to be overparameterized.