3 ms·
This view isn't really accurate – the relationship is one of non-empty intersection rather than inclusion in either direction. Machine learning algorithms are m
by StefanKarpinski 10y ago
This view isn't really accurate – the relationship is one of non-empty intersection rather than inclusion in either direction. Machine learning algorithms are meta-algorithms where a large portion of the specific algorithm to be applied is filled in based on training data. Most ML algorithms are statistically unsound and many statistical methods aren't machine learning.
To make the case that ML - stats isn't empty, consider neural networks and singular value decomposition (Netflix winners used SVD): both are wildly successful ML techniques – and neither is remotely statistically sound. Their correctness is at best heuristic, yet, partly because they can be efficiently applied to huge amounts of data – far more than classical statistical methods can handle – they are very effective.
Statistics has traditionally focused on making the most of a modest amount of data – because in previous eras, data was the limiting factor. Huge data sets today render most statistical tests useless since no hypothesis is strictly true given enough precision, and the focus on eking the most out of every data point is overkill. It turns out that a naive, imprecise method with a huge amount of data is often more effective than a sophisticated method with far less data. As ML gets better at building statistically sound models and stats gets better at scaling, the two disciplines are converging slowly – but we are still very far away from inclusion in either direction.
- eli_gottlieb 10y ago>Huge data sets today render most statistical tests useless since no hypothesis is strictly true given enough precision, and the focus on eking the most out of every data point is overkill. Mind, huge data sets and huge amounts of computing power also make the use of Bayesian methods feasible, which don't have the NHST problem that anything is significant with enough data-points.