3 ms·
I've done some work in this area and have indeed found that simpler statistical models often out perform ML/DL. This is true for single time series, where we a
by ford 4y ago
I've done some work in this area and have indeed found that simpler statistical models often out perform ML/DL.
This is true for single time series, where we are predicting P(x_t+1 | x_0..t)
DL has advantages when you
a) have additional context at each time step, or
b) you have multiple related time series.
For example, consider Amazon who predicts demands for all of their products. At each time step, they know about inventory, marketing efforts, and could even model higher dimensional attributes like persuasiveness of the item's description with NLP.
It's also true they have items that are highly correlated. Skis, Snowboards, and Ski jackets all likely have similar sales patterns. Leveraging this correlation can increase accuracy, and is especially useful when you have items with limited history.
Including all of that context is hard with a statistical model, and whatever equation a human can come up with to combine them is probably worse than a learned, embedding-based DL model.
Statistical models are a great starting point & baseline for most problems, but as you add real world complexity beyond the general case time-series that's not as true.
I might not be aware of it, but I wish there were more benchmarks/research on higher complexity problems.
- fedegr 4y agoI completely agree with your perspective. It is a reality that deep learning models might offer certain advantages over classical statistical models. We are building benchmarks and comparisons to clarify when the more complex models are better. We also want to show with this experiment the importance of creating benchmarks. In many use cases, practitioners choose more sophisticated models because they think this will give them better accuracy. The main idea is that robust benchmarks should always be created.
- Salgat 4y agoMy understanding is that the biggest advantage between traditional machine learning and neural networks is that neural networks are useful when features either need to be generated or are poorly understood (such as a binary blob of an image or sound sample). So for data that is already neatly organized and labeled in a spreadsheet, DL loses its main advantage.
- hackernewds 4y agoHow do you refer to traditional ML, DL and neutral networks in this context? Realize they are rather nebulously defined IRL
- Salgat 4y agoDeep Learning is just Neural networks with more than one hidden layer, so in this case I refer to NN/DL grouped together versus everything else.
- nojito 4y agoAmazon for years used nothing but a random forest for their forecasting.