4 ms·
How is this different than typical multi-variable regressions with control variables? The problem with social science is that it tries to be a science but in r
by bokonist 17y ago
How is this different than typical multi-variable regressions with control variables?
The problem with social science is that it tries to be a science but in reality its closest relative is product management. Using even the best statistical methods still runs into the keys/lamp post problem. There are too many variables, the variables are not well defined, the variables may have strange interactions, there may be too much noise in the data, etc. In almost all social science situations, you cannot find truth solely through regressions.
For instance, recently I was studying the issue of whether the crash in the stock market caused the fall in consumer spending, or the slow down in consumer spending caused the fall in the stock market. If you run a straight regression you'll have a lot of problems because a) even if the stock market falls first, it could have been because of expectations of future consumption b) the data on expenditures is not collected frequently enough, and c) there are other potentially confounding variables.
But if you take the product management approach you can listen to what people say. And when you listen, you find that people say stuff like, "After the stock market fell, I had to cut back my plans to travel." or "Since the fall in our endowment, we have had to suspend spending on all new projects." By simply asking people you can figure out the causality.
Note that this is how every company in the world makes most of its decisions ( except perhaps Google, because they actually enough controlled data to meaningfully do regressions). When we're trying to figure out what features to build, we do customer interviews, follow-me-homes, surveys, feature requests, talk to customer support, etc. We make very modest use of statistics, because we do not have large amounts of controlled data. Did the customer not use this part of the app because it was not useful, or because we got the label wrong, or because of something else? We could do a bunch of statistics, but what's the point? It won't answer those questions. Asking people may be subjective, but it's the only way (keys/lamppost).
The problem is that social scientists do not get promoted for making accurate judgments and being right. A product manager does. A social scientist/academic can only be judged by his methods. Thus there is the preference within the professions for methods that can be objectively evaluated, and methods that only a select elite can perform.
Being right is not part of the selection process for being an academic.
- cousin_it 17y agoHow is this different than typical multi-variable regressions with control variables? I think it is different. Do you understand the smoking/tar/cancer example? See here for the example: http://bayes.cs.ucla.edu/IJCAI99/ijcai-99.pdf http://bayes.cs.ucla.edu/IJCAI99/ijcai-99.pdf (pages 34-38 and 46), and here for a better introductory text: http://cscs.umich.edu/~crshalizi/notebooks/graphical-models.html http://cscs.umich.edu/~crshalizi/notebooks/graphical-models.... . Basically, if the situation allows you to reason that some causal arrows are impossible (e.g. due to time or space separation), you can often infer which of the remaining causal arrows are real and which aren't, using only non-experimental correlation data and some formal inference rules. Of course it doesn't always work - sometimes there really isn't enough data and you have to do a controlled experiment. But on the other hand, people often need to reason about causality in areas where controlled experiments are impossible or unethical, like smoking/tar/cancer.
- zby 17y agoThe too many variables problem might be more evident in the social sciences - but it is present everywhere in science, even the most 'strict' science branches - like physics. You always need to make some assumptions. Much more eloquently - this idea is laid down in: http://falkenblog.blogspot.com/2009/09/nonscientists-naive-about-science.html http://falkenblog.blogspot.com/2009/09/nonscientists-naive-a...
- alan-crowe 17y ago> How is this different than typical multi-variable regressions with control variables? As different as astrology and astronomy. Multi-variable regressions with control variables are known to incapable of supporting causal inference. The network based techniques of Pearl et. al. do permit causal inferences. Notice though that the network based techniques will quite often rigorously prove that the data you have are inadequate to make a causal inference. They do this by discovering multiple causal networks, all consistent with the patterns of correlation and conditional correlation in the data, but with causal links going one way in one net work and the other way in another network.