4 ms·

> 'Econometric models' have so many levers and dials on them that they be tuned to forecast whatever their operator wants them to forecast. I am going to guess

> 'Econometric models' have so many levers and dials on them that they be tuned to forecast whatever their operator wants them to forecast. I am going to guess you have never written down an economic model, much less presented one at an economics seminar. If your model can be "tuned to forecast whatever", you are going to be annihilated. You will never be able to get away with presenting a model like that. There is no place for it in the field. Quite on the contrary, a premium is placed on parsimonious, well-identified models. > Appendix B, Macroeconomic Simulations of NAFTA This is a useful piece, but to make exactly the opposite of the point you wanted to make. Right there in the executive summary it says "although the overall effect of NAFTA on the US economy would be positive, some firms would contract. Industries in the US that make intensive use of low-wage labor and are now protected by trade barriers are likely to be put at a disadvantage." Or see the entirety of Chapter 5 in the same document, where the authors repeatedly note that there will be industries where employment decreases. Indeed, the worst part for your uninformed initial comment is page 87, wherein the following claim is made: "In sum, despite all of the analyses that have been conducted on various aspects of NAFTA, good information for answering the question 'How many workers would be displaced if NAFTA were implemented?' simply does not exist." A substantial fraction of their Appendix B is devoted to noting weaknesses in the then-current literature about NAFTA. These users of econometrics are well aware of the limitations of the models. Forecasting job losses at that time and today is extremely hard, and the authors of this study acknowledge that. This is literally the opposite of what you initially claimed about econometrics.

Kranar 5y ago
To be fair, the purpose of referencing the Google Scholar citation was not to support the claim that econometricians predicted no decline in manufacturing, it was to refute your claim that: "Econometricians are unlikely to have made predictions about NAFTA." I don't know why you would have made such a claim since it should be fairly obvious that econometricians made numerous predictions of all sorts. I will respect your stance on the issue, you are obviously very informed on the subject and seem passionate about it and that deserves respect, but I'm more sympathetic to OP's point of view that econometricians have not produced much in the way of a consensus on any kind of prediction. Whether we look at the 90s at NAFTA, or the rise of China or other third world countries, or Brexit, or even less global topics and instead pick more local and focused subjects such as the minimum wage, rent control, tax policy... you can basically pick any position on such topics and you won't find anything close to a consensus on it.
- huitzitziltzin 5y ago
  I didn’t address that point because it would seem extremely pedantic. To an economist, “an econometrician” is almost always someone who spends their time developing new estimators and proving things about their properties. The people who wrote the CBO report are very unlikely to have been econometricians according to nearly any economist. The CBO mostly hires applied people of various kinds, whether in micro or macro. They are users of econometrics, as we all are, but to say they are “econometricians” as economists use that term would be wrong. But that seemed like an extremely pedantic objection to the OPs point.
yourqiwi 5y ago
> I am going to guess you have never written down an economic model, much less presented one at an economics seminar. If your model can be "tuned to forecast whatever", you are going to be annihilated. You will never be able to get away with presenting a model like that. There is no place for it in the field. A poorly specified model will get ripped apart when presented in an academic context. I've seen it happen in seminars. Questions about the standard errors (robust? clustered?), omitted variable bias, whether things have been detrended, fixed effects, quality of the instrumental variable, etc. When econometrics is done properly and by the book, it can be a useful tool for analyzing observational data. But these tools get regularly applied outside of academia and the results are presented to people who can't critically evaluate them. Nobody knows that you ran a million other regressions in STATA, and kept the significant one. Or that your result isn't robust when you excluded/included such-and-such variables. When research is motivated in a particular direction (as it often is in politics and consulting), then you can often pull the levers and turn the dials until you get the result you want.
- huitzitziltzin 5y ago
  > Nobody knows that you ran a million other regressions in STATA, and kept the significant one. Totally fair criticism and I agree with it. Not at all specific to econometrics though! > Or that your result isn't robust when you excluded/included such-and-such variables. 100%! Also definitely not specific to econometrics. > When research is motivated in a particular direction (as it often is in politics and consulting) ... The CBO is non-partisan. I have more faith in them than that (unless you have inside information?). The whole tone of the specific report linked is very much "here is what we know and don't know and here is our best estimate, which has limits." That is research as it should be at its best. Litigation consulting is different. > ... then you can often pull the levers and turn the dials until you get the result you want. Here I don't agree. There are limits. It's not that as simple as that. I can't turn an estimated -5 into positive 500 just because I want to.
  - yourqiwi 5y ago
    > I can't turn an estimated -5 into positive 500 just because I want to. I didn't have quite that much precision in mind. More like "flip the coefficient's sign" or "make the coefficient bigger" or "make it statistically significant", which are all often doable with enough experimentation (as much as I hate to admit it).

Kranar 5y ago

To be fair, the purpose of referencing the Google Scholar citation was not to support the claim that econometricians predicted no decline in manufacturing, it was to refute your claim that: "Econometricians are unlikely to have made predictions about NAFTA." I don't know why you would have made such a claim since it should be fairly obvious that econometricians made numerous predictions of all sorts. I will respect your stance on the issue, you are obviously very informed on the subject and seem passionate about it and that deserves respect, but I'm more sympathetic to OP's point of view that econometricians have not produced much in the way of a consensus on any kind of prediction. Whether we look at the 90s at NAFTA, or the rise of China or other third world countries, or Brexit, or even less global topics and instead pick more local and focused subjects such as the minimum wage, rent control, tax policy... you can basically pick any position on such topics and you won't find anything close to a consensus on it.

huitzitziltzin 5y ago
I didn’t address that point because it would seem extremely pedantic. To an economist, “an econometrician” is almost always someone who spends their time developing new estimators and proving things about their properties. The people who wrote the CBO report are very unlikely to have been econometricians according to nearly any economist. The CBO mostly hires applied people of various kinds, whether in micro or macro. They are users of econometrics, as we all are, but to say they are “econometricians” as economists use that term would be wrong. But that seemed like an extremely pedantic objection to the OPs point.

yourqiwi 5y ago

> I am going to guess you have never written down an economic model, much less presented one at an economics seminar. If your model can be "tuned to forecast whatever", you are going to be annihilated. You will never be able to get away with presenting a model like that. There is no place for it in the field. A poorly specified model will get ripped apart when presented in an academic context. I've seen it happen in seminars. Questions about the standard errors (robust? clustered?), omitted variable bias, whether things have been detrended, fixed effects, quality of the instrumental variable, etc. When econometrics is done properly and by the book, it can be a useful tool for analyzing observational data. But these tools get regularly applied outside of academia and the results are presented to people who can't critically evaluate them. Nobody knows that you ran a million other regressions in STATA, and kept the significant one. Or that your result isn't robust when you excluded/included such-and-such variables. When research is motivated in a particular direction (as it often is in politics and consulting), then you can often pull the levers and turn the dials until you get the result you want.

huitzitziltzin 5y ago
> Nobody knows that you ran a million other regressions in STATA, and kept the significant one. Totally fair criticism and I agree with it. Not at all specific to econometrics though! > Or that your result isn't robust when you excluded/included such-and-such variables. 100%! Also definitely not specific to econometrics. > When research is motivated in a particular direction (as it often is in politics and consulting) ... The CBO is non-partisan. I have more faith in them than that (unless you have inside information?). The whole tone of the specific report linked is very much "here is what we know and don't know and here is our best estimate, which has limits." That is research as it should be at its best. Litigation consulting is different. > ... then you can often pull the levers and turn the dials until you get the result you want. Here I don't agree. There are limits. It's not that as simple as that. I can't turn an estimated -5 into positive 500 just because I want to.
- yourqiwi 5y ago
  > I can't turn an estimated -5 into positive 500 just because I want to. I didn't have quite that much precision in mind. More like "flip the coefficient's sign" or "make the coefficient bigger" or "make it statistically significant", which are all often doable with enough experimentation (as much as I hate to admit it).