3 ms·

Can you reframe the problem to suit a more classical approach - regression using xgboost or lgbm? If so, go for that! As an example, imagine you want to calcul

Can you reframe the problem to suit a more classical approach - regression using xgboost or lgbm? If so, go for that! As an example, imagine you want to calculate only a single sample into the future. Say furthermore that you have six input timeseries sampled hourly, and you don't expect meaningful correlation beyond 48h old samples. You create 6x48 input features, take the single target value that you want to predict as output, and feed this into your run of the mill gradient boosted tree. The above gives you a less complex approach than reaching for bespoke time-series stuff; I've personally have had success doing something like this. If your regressor does not support multiple outputs, you can always wrap it in sklearns MultiOutputRegressor (or optionally RegressorChain; check it out). This is useful if, in the above example, you are not looking to predict only the next sample, but maybe the next 12 samples. https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputRegressor.html https://scikit-learn.org/stable/modules/generated/sklearn.mu...

em500 4y ago
fbprophet is mostly just regression though, with features for trend, yearly and weekly periodicity (smoothed a bit using trigonometric regressors), and holiday features. The only non-standard linear regression part is that it includes a flexible piecewise linear trend, with regularization to select where the trend is allowed to change. Once the change points are selected, it's literally linear regression, that you could fit with anything that can handle regression (statsmodels, sklearn, xgboost, keras, tf, scipy or even just plain numpy). So you could roll your own using lots of different libraries and do the feature engineering, but for plain jane business time series, fbprophet usually works pretty decently with minimum effort. Because most of the predictable variation in business time series is are patterns in human behavior, which are mostly regulated by the weekly and yearly cycle, interrupted a few times per year by holidays. (And the 24h cycle if you forecast intra-day). That said, fbprophet is somewhat tuned for a few years of regular data (ideally not interrupted by pandemics), sampled at the daily frequency. If your data is different (e.g. weekly or intra-day) it can start to break in unexpected ways, and it becomes worthwhile to dive into the model and customize or roll your own.
- nerdponx 4y ago
  I generally think it's better to start with something even simpler than Prophet, like EMA and ETS. One of the "auto ARIMA" packages in R (Forecast or Fable) could also be a good choice. However I think Prophet is a good next step from there. It's at least a sensible starting point for your own feature engineering.
nerdponx 4y ago
Your description of this technique as "6x48" reminds me of 1D convolutional neural networks. I always had a gut feeling that there was some kind of unifying principle between NN and tree ensemble models. I wonder if the latter is kind of like a highly quantized, compressed version of the former, which explains why tree ensembles seem to work better than NNs on "lower resolution" data.

em500 4y ago

fbprophet is mostly just regression though, with features for trend, yearly and weekly periodicity (smoothed a bit using trigonometric regressors), and holiday features. The only non-standard linear regression part is that it includes a flexible piecewise linear trend, with regularization to select where the trend is allowed to change. Once the change points are selected, it's literally linear regression, that you could fit with anything that can handle regression (statsmodels, sklearn, xgboost, keras, tf, scipy or even just plain numpy). So you could roll your own using lots of different libraries and do the feature engineering, but for plain jane business time series, fbprophet usually works pretty decently with minimum effort. Because most of the predictable variation in business time series is are patterns in human behavior, which are mostly regulated by the weekly and yearly cycle, interrupted a few times per year by holidays. (And the 24h cycle if you forecast intra-day). That said, fbprophet is somewhat tuned for a few years of regular data (ideally not interrupted by pandemics), sampled at the daily frequency. If your data is different (e.g. weekly or intra-day) it can start to break in unexpected ways, and it becomes worthwhile to dive into the model and customize or roll your own.

nerdponx 4y ago
I generally think it's better to start with something even simpler than Prophet, like EMA and ETS. One of the "auto ARIMA" packages in R (Forecast or Fable) could also be a good choice. However I think Prophet is a good next step from there. It's at least a sensible starting point for your own feature engineering.

nerdponx 4y ago

Your description of this technique as "6x48" reminds me of 1D convolutional neural networks. I always had a gut feeling that there was some kind of unifying principle between NN and tree ensemble models. I wonder if the latter is kind of like a highly quantized, compressed version of the former, which explains why tree ensembles seem to work better than NNs on "lower resolution" data.