6 ms·

Sorry ARIMA, but I’m Going Bayesian

173 points by stared 11y ago

tomrod 11y ago
I like the content. Does anyone else have a _really_ hard time reading the light gray-on-white font?
- banku_brougham 11y ago
  I liked it on iPhone 6 plus
- FreeFull 11y ago
  There definitely isn't sufficient contrast.
- chestervonwinch 11y ago
  It is ironic that your off-topic comment lead to down-votes, reducing the contrast of your comment. I can't imagine the nightmare you must be living right now.
  - tomrod 11y ago
    Mostly, I'm unconcerned about karma. But at present, the contrast is fine. I suppose the vote count has increased.
uptownfunk 11y ago
Excellent walkthrough I'll be sure to give this a try on my next project. Any good references for an intro to the traditional arima models?
- stared 11y ago
  Short: http://a-little-book-of-r-for-time-series.readthedocs.org/en/latest/src/timeseries.html http://a-little-book-of-r-for-time-series.readthedocs.org/en... or http://people.duke.edu/~rnau/411arim.htm http://people.duke.edu/~rnau/411arim.htm And for a longer, a book by the author of R forecast library: https://www.otexts.org/fpp/ https://www.otexts.org/fpp/
  - dandermotj 11y ago
    This book is extremely practical - I would definitely recommend it for doing actual time series analysis. That said, it's not a learn time series book. It's a do basic time series in R book.
- em500 11y ago
  Hyndman's online forecasting book https://www.otexts.org/fpp https://www.otexts.org/fpp has an ARIMA section that covers pretty much everything a practitioner would need.
- bachmeier 11y ago
  Try one of the forecasting books on Frank Diebold's page: http://www.ssc.upenn.edu/~fdiebold/Textbooks.html http://www.ssc.upenn.edu/~fdiebold/Textbooks.html I use the Elements of Forecasting book in an undergrad forecasting class that I teach. Forecasting will be better, but I'm not sure if it's completed.
toxik 11y ago
This was very interesting to read, especially as I'm currently taking a course in time series analysis. The course doesn't touch on any Bayesian treatment of the topic at all, and barely goes beyond non-stationary models (e.g. ARIMA.)
hoodwink 11y ago
Like others, I enjoyed this. I work with time series data every day in the domain of commercial real estate. One of my constant struggles is to extract an underlying long-term trend from the real estate cycle. I would love to try this here. Can anyone suggest some Bayesian learning resources for a non-statistician?
- fludlight 11y ago
  http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/ http://ocw.mit.edu/courses/electrical-engineering-and-comput...
  - hoodwink 11y ago
    Thank you!
- alex_hirner 11y ago
  http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/ http://camdavidsonpilon.github.io/Probabilistic-Programming-...
  - dandermotj 11y ago
    Another open source Bayesian book/course all in python: http://www.greenteapress.com/thinkbayes/t http://www.greenteapress.com/thinkbayes/t
    - pjscott 11y ago
      If you can program, this is the best probability book. I think you messed up the URL, though: http://greenteapress.com/wp/think-bayes/ http://greenteapress.com/wp/think-bayes/
      - tmptmp 11y ago
        Thanks a ton you all posters of these book names and links. Helped a lot.
MrQuincle 11y ago
Really nice story. All the Bayesian stuff now gets swamped by big data method. There must be a time machine learning people will come back to the Bayesian womb. :-) I'm currently using it to define priors on measure spaces. I think it's awesome to have so few abstractions in a discipline and be able to do inference anyway. I'd definitely recommend to look into Dirichlet Processes if you haven't before. It's a nice entry point.
- deleted 11y ago
  [deleted]
- huac 11y ago
  Wharton's Pete Fader would agree with you somewhat that big data-style methods are overblown: http://www.datanami.com/2012/05/03/wharton_professor_pokes_hole_in_big_data_balloon/ http://www.datanami.com/2012/05/03/wharton_professor_pokes_h... he also has some very cool uses of dirichlet processes (modelling entire competitive industries)
dia80 11y ago
Thanks for this. If you wanted to get started doing this kind of analysis in python check out the examples for the maturing pyMC3 https://pymc-devs.github.io/pymc3/stochastic_volatility/ https://pymc-devs.github.io/pymc3/stochastic_volatility/
- stared 11y ago
  I second for PyMC3 - a great library. And just recently I went through https://github.com/markdregan/Bayesian-Modelling-in-Python https://github.com/markdregan/Bayesian-Modelling-in-Python.
dandermotj 11y ago
This is a really interesting post. I have to say the data science team at Stitchfix are clearly doing really good, applied work that is central to their business. It's so cool to see. Here's a tip for any R users who read through the code and (like me) is pained by repetition. Instead of using library(lubricate) library(bts) library(...) Just use apply! packages <- c("lubricate", "bts", "...") lapply(packages, library, character.only = TRUE)
- GFK_of_xmaspast 11y ago
  That doesn't seem to me to be a huge efficiency win.
  - dandermotj 11y ago
    In terms of managing packages in session, readability and keystrokes, I think it definitely wins out over successive library calls. It's common to have >5 packages in any one script, especially if you avoid base R like many do.
- huac 11y ago
  or library(pacman) pacman::p_load("lubridate", "bts", ...) with the added benefit of installing missing packages (I find this especially useful because my school's computer lab deletes user-installed packages weekly)
- RockyMcNuts 11y ago
  it's clever but is it more readable? these sorts of discussions, people who write blog posts about 'library' vs. 'require', kind of feel like 'R smell'. wouldn't it be better for a language to just take a list of libraries for import, maybe with a readable syntax? so... have we got to where Julia can re-use R packages yet?
  - dandermotj 11y ago
    I understand where you're coming from, but these two lines are exactly 'taking a list of libraries for import'. packages is a character vector of package names, lapply is by definition 'list apply'. We're taking a list of packages and applying the library function on them. This seems complicated if you're not used to it but R is a functional language. Approaching R from this perspective makes it a powerful, flexible.
radicality 11y ago
I was very surprised when I went to the root domain to see what stitchfix is. Completely not what I was expecting! I guess my model didn't predict a clothes delivery website to be writing about cool stuff like this. But now a bayesian update to this belief is in order :)
liviu- 11y ago
I found it quite interesting to compare HN and /r/statistics in terms of the tone difference in the comments when discussing this article: https://www.reddit.com/r/statistics/comments/4fw1bn/sorry_arima_but_im_going_bayesian/ https://www.reddit.com/r/statistics/comments/4fw1bn/sorry_ar...
- TheLogothete 11y ago
  Because "data science".

173 points by stared 11y ago

tomrod 11y ago

I like the content. Does anyone else have a _really_ hard time reading the light gray-on-white font?

banku_brougham 11y ago
I liked it on iPhone 6 plus
FreeFull 11y ago
There definitely isn't sufficient contrast.
chestervonwinch 11y ago
It is ironic that your off-topic comment lead to down-votes, reducing the contrast of your comment. I can't imagine the nightmare you must be living right now.
- tomrod 11y ago
  Mostly, I'm unconcerned about karma. But at present, the contrast is fine. I suppose the vote count has increased.

uptownfunk 11y ago

Excellent walkthrough I'll be sure to give this a try on my next project. Any good references for an intro to the traditional arima models?

stared 11y ago
Short: http://a-little-book-of-r-for-time-series.readthedocs.org/en/latest/src/timeseries.html http://a-little-book-of-r-for-time-series.readthedocs.org/en... or http://people.duke.edu/~rnau/411arim.htm http://people.duke.edu/~rnau/411arim.htm And for a longer, a book by the author of R forecast library: https://www.otexts.org/fpp/ https://www.otexts.org/fpp/
- dandermotj 11y ago
  This book is extremely practical - I would definitely recommend it for doing actual time series analysis. That said, it's not a learn time series book. It's a do basic time series in R book.
em500 11y ago
Hyndman's online forecasting book https://www.otexts.org/fpp https://www.otexts.org/fpp has an ARIMA section that covers pretty much everything a practitioner would need.
bachmeier 11y ago
Try one of the forecasting books on Frank Diebold's page: http://www.ssc.upenn.edu/~fdiebold/Textbooks.html http://www.ssc.upenn.edu/~fdiebold/Textbooks.html I use the Elements of Forecasting book in an undergrad forecasting class that I teach. Forecasting will be better, but I'm not sure if it's completed.

toxik 11y ago

This was very interesting to read, especially as I'm currently taking a course in time series analysis. The course doesn't touch on any Bayesian treatment of the topic at all, and barely goes beyond non-stationary models (e.g. ARIMA.)

hoodwink 11y ago

Like others, I enjoyed this. I work with time series data every day in the domain of commercial real estate. One of my constant struggles is to extract an underlying long-term trend from the real estate cycle. I would love to try this here. Can anyone suggest some Bayesian learning resources for a non-statistician?

fludlight 11y ago
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/ http://ocw.mit.edu/courses/electrical-engineering-and-comput...
- hoodwink 11y ago
  Thank you!
alex_hirner 11y ago
http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/ http://camdavidsonpilon.github.io/Probabilistic-Programming-...
- dandermotj 11y ago
  Another open source Bayesian book/course all in python: http://www.greenteapress.com/thinkbayes/t http://www.greenteapress.com/thinkbayes/t
  - pjscott 11y ago
    If you can program, this is the best probability book. I think you messed up the URL, though: http://greenteapress.com/wp/think-bayes/ http://greenteapress.com/wp/think-bayes/
    - tmptmp 11y ago
      Thanks a ton you all posters of these book names and links. Helped a lot.

MrQuincle 11y ago

Really nice story. All the Bayesian stuff now gets swamped by big data method. There must be a time machine learning people will come back to the Bayesian womb. :-) I'm currently using it to define priors on measure spaces. I think it's awesome to have so few abstractions in a discipline and be able to do inference anyway. I'd definitely recommend to look into Dirichlet Processes if you haven't before. It's a nice entry point.

deleted 11y ago
[deleted]
huac 11y ago
Wharton's Pete Fader would agree with you somewhat that big data-style methods are overblown: http://www.datanami.com/2012/05/03/wharton_professor_pokes_hole_in_big_data_balloon/ http://www.datanami.com/2012/05/03/wharton_professor_pokes_h... he also has some very cool uses of dirichlet processes (modelling entire competitive industries)

dia80 11y ago

Thanks for this. If you wanted to get started doing this kind of analysis in python check out the examples for the maturing pyMC3 https://pymc-devs.github.io/pymc3/stochastic_volatility/ https://pymc-devs.github.io/pymc3/stochastic_volatility/

stared 11y ago
I second for PyMC3 - a great library. And just recently I went through https://github.com/markdregan/Bayesian-Modelling-in-Python https://github.com/markdregan/Bayesian-Modelling-in-Python.

dandermotj 11y ago

This is a really interesting post. I have to say the data science team at Stitchfix are clearly doing really good, applied work that is central to their business. It's so cool to see. Here's a tip for any R users who read through the code and (like me) is pained by repetition. Instead of using library(lubricate) library(bts) library(...) Just use apply! packages <- c("lubricate", "bts", "...") lapply(packages, library, character.only = TRUE)

GFK_of_xmaspast 11y ago
That doesn't seem to me to be a huge efficiency win.
- dandermotj 11y ago
  In terms of managing packages in session, readability and keystrokes, I think it definitely wins out over successive library calls. It's common to have >5 packages in any one script, especially if you avoid base R like many do.
huac 11y ago
or library(pacman) pacman::p_load("lubridate", "bts", ...) with the added benefit of installing missing packages (I find this especially useful because my school's computer lab deletes user-installed packages weekly)
RockyMcNuts 11y ago
it's clever but is it more readable? these sorts of discussions, people who write blog posts about 'library' vs. 'require', kind of feel like 'R smell'. wouldn't it be better for a language to just take a list of libraries for import, maybe with a readable syntax? so... have we got to where Julia can re-use R packages yet?
- dandermotj 11y ago
  I understand where you're coming from, but these two lines are exactly 'taking a list of libraries for import'. packages is a character vector of package names, lapply is by definition 'list apply'. We're taking a list of packages and applying the library function on them. This seems complicated if you're not used to it but R is a functional language. Approaching R from this perspective makes it a powerful, flexible.

radicality 11y ago

I was very surprised when I went to the root domain to see what stitchfix is. Completely not what I was expecting! I guess my model didn't predict a clothes delivery website to be writing about cool stuff like this. But now a bayesian update to this belief is in order :)

liviu- 11y ago

I found it quite interesting to compare HN and /r/statistics in terms of the tone difference in the comments when discussing this article: https://www.reddit.com/r/statistics/comments/4fw1bn/sorry_arima_but_im_going_bayesian/ https://www.reddit.com/r/statistics/comments/4fw1bn/sorry_ar...

TheLogothete 11y ago
Because "data science".