4 ms·
> Neural networks just happen to suck less than all other known methods. Or, perhaps, the best demonstrated performance of NNs exceeds the best demonstrated pe
by abeppu 5y ago
> Neural networks just happen to suck less than all other known methods.
Or, perhaps, the best demonstrated performance of NNs exceeds the best demonstrated performance of other known methods for many tasks. But ... the amount of compute, investment in tooling, and attention that have been thrown at deep learning in the past decade is at a scale where ... do we actually know that other methods would perform worse with the same resources? Is there some alternate timeline where in 2012 someone figured out how to run MCMC for bayesian non-parametrics over much larger datasets or something, and the whole field of ML just tilted in a different direction?
- credit_guy 5y agoThat's a very good observation. However, deep learning didn't just get the share of the first mover. Before DL was popular, Support Vector Machines used to be where all the ML fun research was happening. And just out of nowhere, Random Forests and XGBoost came and took the crown if only for a fleeting moment. Gaussian Processes always showed promise, but I'm not sure they delivered. Deep Learning just delivered. I guess it's because of the composability. But you are absolutely right that there's no proof and now way of knowing right now if DL is the best there possibly can be.