4 ms·
As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly u
by lr1970 5y ago
As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly useless in many import practical applications such as FinTech, e-commerce, sports betting, etc. Point-forecast models such as Prophet fail to give you a meaningful measure of uncertainty of the predicted value. A much better approach are probabilistic forecasting models that predict the probability distribution of the random variable of interest [0]. Probability distribution is the right language to express prediction values and their uncertainty at the same time. And the decisions based on forecasts oftentimes take the uncertainty measures in consideration, e.g. portfolio risk optimization or buying decisions in e-commerce.
[0] https://en.wikipedia.org/wiki/Probabilistic_forecasting https://en.wikipedia.org/wiki/Probabilistic_forecasting
- dcl 5y agoDo you have any favourite libraries for producing such?
- microprediction 5y agohttps://www.microprediction.com/blog/popular-timeseries-packages https://www.microprediction.com/blog/popular-timeseries-pack...
- lr1970 5y ago> Do you have any favourite libraries for producing such? For modern Deep Learning based probabilistic forecasting you can try DeepAR with parametric likelihood function [0] or Multi-Horizon Quantile RNN (non-parametric) [1]. The implementations of these models in Pytorch and MXnet are scattered all over the place. [0] https://arxiv.org/abs/1704.04110 https://arxiv.org/abs/1704.04110 [1] https://arxiv.org/abs/1711.11053 https://arxiv.org/abs/1711.11053 EDIT: formatting
- frakt0x90 5y agoAnd if you want to play with this the gluonts python library makes it very easy. It also includes some other time series models as well! I've been playing with it at work and it's enjoyable.
- glial 5y agoCan you elaborate a bit? Prophet can use MCMC sampling and includes uncertainty in its forecasts.
- lr1970 5y agoProphet with MCMC can produce probabilistic forecasts. But how to choose MCMC priors and measure accuracy of the probabilistic forecast are open questions.
- deleted 5y ago[deleted]
- lr1970 5y ago> Can you elaborate a bit? Prophet can use MCMC sampling and includes uncertainty in its forecasts. Prophet is a GAM (Generalised Additive Model). It decomposes time series in additive components: trend, seasonality, holidays and noise. Most interesting time-series are not so simply decomposable. Making Prophet Bayesian and producing probabilistic forecast by MCMC sampling from trend/seasonality/holiday posteriors still keeps its GAM structure. Might be for a simple exploratory analysis Prophet is a good go-to tool but all the research action is now in Deep Learning Forecasting Models. Also, IMHO, Prophet deals with individual TS and teaching it to produce vector forecast for multiple TSes at the same time is tricky (or not even possible).
- glial 5y agoYes, that’s my understanding too. GAMs have structural limitations that harm flexibility and help interpretability. One benefit of Bayesian models that they work with relatively little data - and generally provide greater uncertainty in those cases. Do you happen to know of some DL frameworks that behave similarly? I’m eager to learn.
- chasely 5y agoIt's worth noting that one of the original authors of Prophet (Sean Taylor) basically agrees with this and is now on a team at Lyft doing probabilistic forecasting for their marketplace [0]. [0] https://twimlai.com/causal-models-in-practice-at-lyft-with-sean-taylor/ https://twimlai.com/causal-models-in-practice-at-lyft-with-s...
- kqr 5y agoThis is a good point and true for many things other than timeseries analysis. Any time uncertainty is involved the language of probability is the only thing that makes sense. And as you say, what matters is generally not the specific values you are uncertain of, but what consequences they can potentially have for your decisions. To know this, you really have to know most values that are possible and how likely they are.
- CuriouslyC 5y agoGaussian processes are great for this. Another easy option is to bootstrap the estimator, though for large models that isn't feasible.
- devit 5y agoThis is a Bayesian model so it of course provides a probability distribution for both the parameters and the forecasted values. P(model | historic_data) = P(historic_data | model) P(model) / (sum(model) P(historic_data | model) P(model)) P(future_data | historic_data) = sum(model) P(future_data | model) P(model | historic_data) = (sum(model) P(future_data | model) P(historic_data | model) P(model)) / (sum(model) P(historic_data | model) P(model))
- microprediction 5y agoFun fact: if you publish a data point regularly at https://www.microprediction.com/get-predictions https://www.microprediction.com/get-predictions then over time, distributional predictions will come to you. It's like a nano-market for live distributional prediction.