Machine learning is more accurately described (in my view) as an interdisciplinary region that involves computation and statistics both. It's when you have to u

Machine learning is more accurately described (in my view) as an interdisciplinary region that involves computation and statistics both. It's when you have to use statistics to push the bounds of computational work, or CS to tease out statistical relationships that require so much data that the size of the data itself is part of the problem. When I think of "statistics", I think of problems where there are a few well-studied parametric approaches, in large part because there aren't enough data (in most cases) to do anything extremely complicated. If your data set is small and you need to build a generally adequate model, you can often use linear regression to get something good enough, and that may the best you can do, because model simplicity is often valuable in its own right (low risk of overfitting). Many of the more non-parametric machine learning algorithms (e.g. neural nets) are computationally intense and often a bitch to debug. They work best in situations where (a) you have a lot of data, but (b) you have no idea what the structure of the relationship should be. Parametric models perform extremely well when the structure is known. You have inputs X1, X2, X3 and Y, and you know that a linear model will work well. You fit one, and it captures 65% of the variance. Good. That may be enough. What do you use, however, if 65% isn't good enough, or if there's a special case that becomes economically important. You can look at the residuals. They may be random noise. If they're truly "random", then the linear model is the absolute best you can get. That may not be, however. There might be a region of the X2-X3 space where atypically high Y-values occur for a structural reason. These atypical points may be connected to an X4 that would otherwise be discarded (because it had no general correlation with Y). I think that machine learning approaches start to be worth their additional complexity in these cases where (a) there are important relationships yet to be discovered-- no one knows about them!-- and (b) those are, or might be, economically critical.