4 ms·

Can you elaborate on that? If you mean that ML stuff isn't taught much in coursework, that's probably true --- but getting that stuff to work for economic quest

Can you elaborate on that? If you mean that ML stuff isn't taught much in coursework, that's probably true --- but getting that stuff to work for economic questions isn't exactly trivial (e.g. we only have observational data but want to estimate the effects of hypothetical policy interventions). If you mean something else by "modern statistical methods" I'd be curious to know what you mean.

huac 10y ago
Well, we talked about using cross-validation and the bootstrap/bagging methods. I was surprised that he hadn't heard of these ideas, much less using. Deep learning variants are definitely farther out; I think the current lack of interpretability makes for a real strenous case wrt economic applications. I don't really want to draw conclusions from n=1 convos but a couple other links had piqued my curiosity towards this relationship: https://www.quora.com/How-will-Machine-Learning-affect-economics https://www.quora.com/How-will-Machine-Learning-affect-econo..., https://news.ycombinator.com/item?id=11460412 https://news.ycombinator.com/item?id=11460412
- dmagee 10y ago
  These methods are not usually taught in economics courses. Why? Not enough data. In most cases economists are dealing with survey data, time series data and panel data. The benefits of cross validation and bootstrap/bagging grow with data size. When you're dealing with minimal amounts of messy/misbehaving data these methods lose their power. Other methods become more important ie: Instrumental Variable estimation, Probit and Tobit models, Vector Autoregressions, Vector Error Correction models. Im sure your econ PhD student friend would know what these are. A different tool-kit to solve different problems.
  - grayclhn 10y ago
    "Not enough time" and "harder to do inference" are bigger reasons. It's hard to see why cross validation would do worse than the AIC or BIC for lag length selection in VARs, the bootstrap is widely used for inference for all of the models you mentioned, IV isn't known for its exceptional small sample properties, etc. People are working on this stuff, but it takes a little while to get it to work well for Econ research, and it takes some clear new empirical findings before it becomes mainstream enough to teach it in classes. Everyone recognizes that there's a lot of promise, though.
    - dmagee 10y ago
      grayclhn, you might find Rob Hyndman's article on cross validation helpful. http://robjhyndman.com/hyndsight/crossvalidation/ http://robjhyndman.com/hyndsight/crossvalidation/ " Asymptotically, minimizing the AIC is equivalent to minimizing the CV value. This is true for any model (Stone 1977), not just linear models. It is this property that makes the AIC so useful in model selection when the purpose is prediction."
- grayclhn 10y ago
  Thanks, that's a bit surprising, but definitely helped me make up my mind on what I'll teach in the last few weeks of my class next year. The bootstrap is a weird omission, but economists usually only see it for variance estimation and inference, not in other contexts.
- RockyMcNuts 10y ago
  Interesting, I think bootstrapping is overused in financial modeling, because most long term financial time series don't obey the assumptions, covariance stationarity, normal distribution of residuals etc. A lot of regime changes, fat tails, latent predictors that you don't realize are important until they break your model. disclaimer: hand-waving OTOH there are a lot of situations where you need to model something with a lot of potential predictors and limited data. e.g. testing a macro model with 100s of potential predictors and 100 years of relatively poor macro data. ML might find interesting relationships in those cases. Traditional statistics has a strong theoretical foundation, you assume a bunch of things about the shape of the data, and you can prove your estimator is best and what the error looks like based on the amount of data. It makes heroic assumptions about underlying data that we know don't apply. So it often doesn't work well but we know why. ML just wants to find things that work well in cross-validation without worrying too much about proofs... ML is a little like QE ... it works but we don't really know why. Anyway, any sufficiently complex ecosystem is a market design, makes sense that if you have a sufficiently valuable ecosystem you would want some people who study markets.

huac 10y ago

Well, we talked about using cross-validation and the bootstrap/bagging methods. I was surprised that he hadn't heard of these ideas, much less using. Deep learning variants are definitely farther out; I think the current lack of interpretability makes for a real strenous case wrt economic applications. I don't really want to draw conclusions from n=1 convos but a couple other links had piqued my curiosity towards this relationship: https://www.quora.com/How-will-Machine-Learning-affect-economics https://www.quora.com/How-will-Machine-Learning-affect-econo..., https://news.ycombinator.com/item?id=11460412 https://news.ycombinator.com/item?id=11460412

dmagee 10y ago
These methods are not usually taught in economics courses. Why? Not enough data. In most cases economists are dealing with survey data, time series data and panel data. The benefits of cross validation and bootstrap/bagging grow with data size. When you're dealing with minimal amounts of messy/misbehaving data these methods lose their power. Other methods become more important ie: Instrumental Variable estimation, Probit and Tobit models, Vector Autoregressions, Vector Error Correction models. Im sure your econ PhD student friend would know what these are. A different tool-kit to solve different problems.
- grayclhn 10y ago
  "Not enough time" and "harder to do inference" are bigger reasons. It's hard to see why cross validation would do worse than the AIC or BIC for lag length selection in VARs, the bootstrap is widely used for inference for all of the models you mentioned, IV isn't known for its exceptional small sample properties, etc. People are working on this stuff, but it takes a little while to get it to work well for Econ research, and it takes some clear new empirical findings before it becomes mainstream enough to teach it in classes. Everyone recognizes that there's a lot of promise, though.
  - dmagee 10y ago
    grayclhn, you might find Rob Hyndman's article on cross validation helpful. http://robjhyndman.com/hyndsight/crossvalidation/ http://robjhyndman.com/hyndsight/crossvalidation/ " Asymptotically, minimizing the AIC is equivalent to minimizing the CV value. This is true for any model (Stone 1977), not just linear models. It is this property that makes the AIC so useful in model selection when the purpose is prediction."
grayclhn 10y ago
Thanks, that's a bit surprising, but definitely helped me make up my mind on what I'll teach in the last few weeks of my class next year. The bootstrap is a weird omission, but economists usually only see it for variance estimation and inference, not in other contexts.
RockyMcNuts 10y ago
Interesting, I think bootstrapping is overused in financial modeling, because most long term financial time series don't obey the assumptions, covariance stationarity, normal distribution of residuals etc. A lot of regime changes, fat tails, latent predictors that you don't realize are important until they break your model. disclaimer: hand-waving OTOH there are a lot of situations where you need to model something with a lot of potential predictors and limited data. e.g. testing a macro model with 100s of potential predictors and 100 years of relatively poor macro data. ML might find interesting relationships in those cases. Traditional statistics has a strong theoretical foundation, you assume a bunch of things about the shape of the data, and you can prove your estimator is best and what the error looks like based on the amount of data. It makes heroic assumptions about underlying data that we know don't apply. So it often doesn't work well but we know why. ML just wants to find things that work well in cross-validation without worrying too much about proofs... ML is a little like QE ... it works but we don't really know why. Anyway, any sufficiently complex ecosystem is a market design, makes sense that if you have a sufficiently valuable ecosystem you would want some people who study markets.