4 ms·

Economists do plenty of experiments. As for Econometrics, I agree that studies employing observational causal inference in statistics are much harder (and hard

by zwaps 5y ago

Economists do plenty of experiments. As for Econometrics, I agree that studies employing observational causal inference in statistics are much harder (and harder to swallow) than experiments. Usually. But perhaps, if you dig a bit deeper and ask yourself things like: "what makes a hypothesis falsifyable" or, "could there be a hypothesis falsifyable by observational data?". Have your ever written down the exact exclusion condition (or exogeneity condition or whatever) you use in your experiment? You might find out something interesting. Controlled experiments, much like any econometric (or statistic) technique, require identification assumptions. And of course, these can not be proven. Even though many people believe it is so, experiments are not really different in kind, the assumptions are just usually easier to believe. Not always, though, which can also be fun. Even in an experiment, you - in fact - observe reality only once. In other cases, however, we find out that our assumptions about the identification in the experiment were wrong, due to factors we didn't know about. This happens even in physics (shocker, I know). Keep an open mind, would be my advise. Your critiques are well known, and in the broad sense tackling them is the business of statistics. By the way, there are indeed smart people here and there. A good heuristic: If your find yourself rejecting entire disciplines, like econometrics, all while implictily using auction theory, regulation, pricing etc. in your amazon, ebay, cellphone, or energy grid, you are probably on the wrong peak of a Dunning Kruger curve.

mbar84 5y ago
I understand the argument and I accept the idea of natural experiments in principle. The most difficult thing with all of these is the ceteris paribus (holding all else constant), or as a second best, correcting for known confounding factors. This is in such stark contrast to a lab setting, where an adversarial scientist can regard a hypothesis as wrong and construct an experiment to prove it so, only to find out that the hypothesis isn't falsified. With statistics and models on the other hand it always devolves into discussions about variables, confounding factors and there's never a satisfying resolution, especially on controversial results. The number of places for bias to creep in are way to many. I think the econometrics people have gone down a hopeless path, guided by wishful thinking and demand from people who want us to be able to make statements about things which we just cannot do. The parts of economics that are valuable are not based on econometrics, but upon reasoning from first principles: Incentives, supply and demand, game theory etc. under which I would argue all of your examples fall.
- zwaps 5y ago
  Then I don't get it. If you accept first principles, then the world of structural econometrics is open to you. Cake having, and cake eating - at the cost of structural assumptions. All you are doing then is filling your models with some actual data. Could you be more specific what is a lab setting for you? Natural experiments are not what I meant when I spoke of experiments, as natural experiments are observational inferential methods! btw did they ever tell you about that physics experiment in a lab that was built just over a subway line? So much confounding! Jokes aside, your assumptions amount to assumptions about what "controlled" and "in a lab" means. I hope you see it makes more sense to define the requirement for experimental identification mathematically, in which case you'd see that the conditions are not different in kind, only in degree. etc. Edit: If that counts as revealed preference data, I can assure you that econometrics are used to great effect in pricing, auctions etc - one of the reasons certain orange and blue companies hiring experts on this have become mucho rich in the past couple of years. Some oldschool econ models may not be the best to describe the altruistic, boundedly rational present-preferrer. But oh boy, can you make some serious dough if you can mechanism design the auction platforms and the best the firms can do is to try to train some machine learning models against it.
  - mbar84 5y ago
    The only thing to which I will object here is the use of the word "only" from "not different in kind, only in degree". There is a substantial difference in degree when dealing with atoms, molecules, homogenous materials vs. people who think, face incentives and understand game theory.
- 7thaccount 5y ago
  Not an economist, but my job role involves lots of economic theory and I'm a few classes into an econ masters. As far as economic reasoning goes, there are some studies you can do (like optimal auction clearing) where you can hold the inputs equal and then adjust some aspects of the optimization (the objective, constraints, or decision variables) and get a very accurate picture of how market changes would impact the results. I think that is probably an exception rather than the normal case though. In general, my thoughts during my studies has been that the economic theories of supply & demand, incentives, elasticity, theory of the firm, game theory, government intervention...etc all has more use as a generic framework of thinking (ex: if we put in a price cap, here are the expected impacts), rather than an equation I can use to get an exact answer (basically exactly what you're saying above). Doing some kind of analysis on things as complex as the US economy must have massive inaccuracies involved as there are too many factors and even behavioral economics doesn't have all the answers for why people don't always act rationally.
deleted 5y ago
[deleted]

by zwaps 5y ago

mbar84 5y ago

I understand the argument and I accept the idea of natural experiments in principle. The most difficult thing with all of these is the ceteris paribus (holding all else constant), or as a second best, correcting for known confounding factors. This is in such stark contrast to a lab setting, where an adversarial scientist can regard a hypothesis as wrong and construct an experiment to prove it so, only to find out that the hypothesis isn't falsified. With statistics and models on the other hand it always devolves into discussions about variables, confounding factors and there's never a satisfying resolution, especially on controversial results. The number of places for bias to creep in are way to many. I think the econometrics people have gone down a hopeless path, guided by wishful thinking and demand from people who want us to be able to make statements about things which we just cannot do. The parts of economics that are valuable are not based on econometrics, but upon reasoning from first principles: Incentives, supply and demand, game theory etc. under which I would argue all of your examples fall.

zwaps 5y ago
Then I don't get it. If you accept first principles, then the world of structural econometrics is open to you. Cake having, and cake eating - at the cost of structural assumptions. All you are doing then is filling your models with some actual data. Could you be more specific what is a lab setting for you? Natural experiments are not what I meant when I spoke of experiments, as natural experiments are observational inferential methods! btw did they ever tell you about that physics experiment in a lab that was built just over a subway line? So much confounding! Jokes aside, your assumptions amount to assumptions about what "controlled" and "in a lab" means. I hope you see it makes more sense to define the requirement for experimental identification mathematically, in which case you'd see that the conditions are not different in kind, only in degree. etc. Edit: If that counts as revealed preference data, I can assure you that econometrics are used to great effect in pricing, auctions etc - one of the reasons certain orange and blue companies hiring experts on this have become mucho rich in the past couple of years. Some oldschool econ models may not be the best to describe the altruistic, boundedly rational present-preferrer. But oh boy, can you make some serious dough if you can mechanism design the auction platforms and the best the firms can do is to try to train some machine learning models against it.
- mbar84 5y ago
  The only thing to which I will object here is the use of the word "only" from "not different in kind, only in degree". There is a substantial difference in degree when dealing with atoms, molecules, homogenous materials vs. people who think, face incentives and understand game theory.
7thaccount 5y ago
Not an economist, but my job role involves lots of economic theory and I'm a few classes into an econ masters. As far as economic reasoning goes, there are some studies you can do (like optimal auction clearing) where you can hold the inputs equal and then adjust some aspects of the optimization (the objective, constraints, or decision variables) and get a very accurate picture of how market changes would impact the results. I think that is probably an exception rather than the normal case though. In general, my thoughts during my studies has been that the economic theories of supply & demand, incentives, elasticity, theory of the firm, game theory, government intervention...etc all has more use as a generic framework of thinking (ex: if we put in a price cap, here are the expected impacts), rather than an equation I can use to get an exact answer (basically exactly what you're saying above). Doing some kind of analysis on things as complex as the US economy must have massive inaccuracies involved as there are too many factors and even behavioral economics doesn't have all the answers for why people don't always act rationally.

deleted 5y ago

[deleted]