5 ms·

Are RNNs ideal to process non-textual time series information? We are looking to replace our Arima models with RNNs and the results so far has been far from sa

Are RNNs ideal to process non-textual time series information? We are looking to replace our Arima models with RNNs and the results so far has been far from satisfactory. The usecase is: based on sale quantity in past year, predict the sale quantity tomorrow. Regression does not consider weekdays or weekends or similar bumps and we thought RNN w/LSTM would be well suited for this problem

xgb84j 9y ago
I would try to include additional indicator features for weekdays or weekends directly into your Arima model. Also I believe that RNNs are useful mainly for highly non-linear problems. Non-linearities in a problem such as sales forecasting are best handled through interaction terms or by including non-linear transformations (e.g. logarithm) of existing features into your model.
- ganeshkrishnan 9y ago
  The problem with ARIMA is that it's more of an exponential regression rather than deep learning. We already have added weekday,weekend,month variable to it but its more of regression than deep learning. So you are saying that RNN won't be suited for non-linear sales forecasting?
  - xgb84j 9y ago
    I don't have the details of your exact use case, but I cannot imagine any complex non-linearities involved in your sales process. If ARIMA models produce decent results for your use case I would try to improve the ARIMA model through additional data, rather than switching to deep learning. If you are convinced that there are complex non-linearities that an ARIMA model cannot describe, then I would try to use RNNs to find a pattern in your ARIMA models' residuals and try to augment your ARIMA model with an RNN.
    - nonbel 9y ago
      >"I cannot imagine any complex non-linearities involved in your sales process" Knowing nothing about his sales process and features, I would assume there are many complex non-linearities (to be possibly leveraged for better predictions). I find this statement bizarre.
      - xgb84j 9y ago
        What I mean by this statement is that there are lots of "tricks" such as interaction terms, regime switching and non-linear transformation of features to handle non-linearities in linear models (e.g. different food sold before Christmas). But if you can give me an example of a non-linearity in sales forecasting that cannot be fit by a linear model but can by an RNN I'd honestly be really interested in that.
    - ganeshkrishnan 9y ago
      > If ARIMA models produce decent results for your use case I would try to improve the ARIMA model through additional data, rather than switching to deep learning. ARIMA models cannot take into consideration special scenarios or even multivariate features (Like location and time). It's good for simple forecasting but when you need more "human like" predictions we are betting on neural networks. For example we would instinctively know that sales of turkey goes up on Thanksgiving. ARIMA model cannot factor this but RNN/LSTM should be able to (Theoretically speaking).
      - xgb84j 9y ago
        There are techniques regime switching and indicator functions that can handle exactly this type of interaction in a linear model. https://en.wikipedia.org/wiki/Dummy_variable_(statistics) https://en.wikipedia.org/wiki/Dummy_variable_(statistics) https://en.wikipedia.org/wiki/Interaction_(statistics) https://en.wikipedia.org/wiki/Interaction_(statistics)
superfx 9y ago
May wanna look at Phased LSTMs: https://arxiv.org/abs/1610.09513 https://arxiv.org/abs/1610.09513
- ganeshkrishnan 9y ago
  Keras has support for this. We will give this a shot too. Thanks!
alexcnwy 9y ago
I'd suggest lecture 14 of the fast.ai mooc on some advice on feature engineering for timeseries data to make it possible to model with something like regression (or a non-recurrent neural net - can work surprisingly well)
- bhuthesh_r 9y ago
  Where do I find the lecture 14? I was able to find only 7 lectures here - http://course.fast.ai/ http://course.fast.ai/ .
  - alexcnwy 9y ago
    I'm not sure if it's officially out yet but if you search YouTube, you will find it ;) https://m.youtube.com/watch?v=6lTyqrrWVQ0 https://m.youtube.com/watch?v=6lTyqrrWVQ0

xgb84j 9y ago

I would try to include additional indicator features for weekdays or weekends directly into your Arima model. Also I believe that RNNs are useful mainly for highly non-linear problems. Non-linearities in a problem such as sales forecasting are best handled through interaction terms or by including non-linear transformations (e.g. logarithm) of existing features into your model.

ganeshkrishnan 9y ago
The problem with ARIMA is that it's more of an exponential regression rather than deep learning. We already have added weekday,weekend,month variable to it but its more of regression than deep learning. So you are saying that RNN won't be suited for non-linear sales forecasting?
- xgb84j 9y ago
  I don't have the details of your exact use case, but I cannot imagine any complex non-linearities involved in your sales process. If ARIMA models produce decent results for your use case I would try to improve the ARIMA model through additional data, rather than switching to deep learning. If you are convinced that there are complex non-linearities that an ARIMA model cannot describe, then I would try to use RNNs to find a pattern in your ARIMA models' residuals and try to augment your ARIMA model with an RNN.
  - nonbel 9y ago
    >"I cannot imagine any complex non-linearities involved in your sales process" Knowing nothing about his sales process and features, I would assume there are many complex non-linearities (to be possibly leveraged for better predictions). I find this statement bizarre.
    - xgb84j 9y ago
      What I mean by this statement is that there are lots of "tricks" such as interaction terms, regime switching and non-linear transformation of features to handle non-linearities in linear models (e.g. different food sold before Christmas). But if you can give me an example of a non-linearity in sales forecasting that cannot be fit by a linear model but can by an RNN I'd honestly be really interested in that.
  - ganeshkrishnan 9y ago
    > If ARIMA models produce decent results for your use case I would try to improve the ARIMA model through additional data, rather than switching to deep learning. ARIMA models cannot take into consideration special scenarios or even multivariate features (Like location and time). It's good for simple forecasting but when you need more "human like" predictions we are betting on neural networks. For example we would instinctively know that sales of turkey goes up on Thanksgiving. ARIMA model cannot factor this but RNN/LSTM should be able to (Theoretically speaking).
    - xgb84j 9y ago
      There are techniques regime switching and indicator functions that can handle exactly this type of interaction in a linear model. https://en.wikipedia.org/wiki/Dummy_variable_(statistics) https://en.wikipedia.org/wiki/Dummy_variable_(statistics) https://en.wikipedia.org/wiki/Interaction_(statistics) https://en.wikipedia.org/wiki/Interaction_(statistics)

superfx 9y ago

May wanna look at Phased LSTMs: https://arxiv.org/abs/1610.09513 https://arxiv.org/abs/1610.09513

ganeshkrishnan 9y ago
Keras has support for this. We will give this a shot too. Thanks!

alexcnwy 9y ago

I'd suggest lecture 14 of the fast.ai mooc on some advice on feature engineering for timeseries data to make it possible to model with something like regression (or a non-recurrent neural net - can work surprisingly well)

bhuthesh_r 9y ago
Where do I find the lecture 14? I was able to find only 7 lectures here - http://course.fast.ai/ http://course.fast.ai/ .
- alexcnwy 9y ago
  I'm not sure if it's officially out yet but if you search YouTube, you will find it ;) https://m.youtube.com/watch?v=6lTyqrrWVQ0 https://m.youtube.com/watch?v=6lTyqrrWVQ0