23 ms·
As a former PhD student in economics I can tell you that this kind of research is often not very well done. Having not read the paper I will reserve final judge
by stanleydrew 17y ago
As a former PhD student in economics I can tell you that this kind of research is often not very well done. Having not read the paper I will reserve final judgement, but based on the journal it's published in this probably doesn't hold much weight. Here's why:
In order to say anything causal (as the author of this paper would like to) in a field where you can't run experiments and only have observational data you need to have a source of identification. It's not enough to compare means. Yes this has everything to do with unobserved-variable bias.
In labor economics the huge unobserved variable is ability. People who are naturally more "able" will likely earn more than less able people and also will likely obtain more education. If that is true then it will look like education is much more important in determining earnings/wages than it actually is.
So there are all these ways of trying to eliminate that unobserved-variable bias. The popular one in the 90s and early 00s (and still popular today despite a lot of criticism) is the instrumental variables technique. Here's the idea:
We need to introduce a source of randomness into our otherwise non-random data. If we can find a variable that is correlated with the causal variable we are interested in, but (arguably) uncorrelated with the unobserved variable, then we can use that new variable as an "instrument." I won't get into the technical details of how to run the regressions with actual data, but this method basically uses an external source of randomness to attempt to say something causal.
The classic paper on this is Josh Angrist's and Alan Kreuger's paper estimating returns to education. They used Vietnam draft lottery numbers as their instrument, since your draft lottery number should be completely uncorrelated with your natural "ability," but definitely would correlate with how much education you got (those having to serve couldn't go to school). More on that paper is here: http://econpapers.repec.org/paper/priindrel/670.htm http://econpapers.repec.org/paper/priindrel/670.htm.
So basically I doubt that this community college paper does much of the required analysis to actually "identify" the causal relationship it is claiming to have found.
UPDATE: Having read the methodology (page 7) portion of the paper (link to the pdf at the bottom of this page: http://userpages.umbc.edu/~marcotte/pubs.htm http://userpages.umbc.edu/~marcotte/pubs.htm) I can confirm that my doubts were justified (the author also makes some highly questionable assumptions in order to estimate individuals' work experience as well, but I won't get into that). The author does acknowledge that unobserved variables will confound the results, and attempts to eliminate the bias by controlling for underlying ability with standardized high school math and reading exam scores. This is a common method, but it's really not good enough. We are still dealing with observational data, and we still haven't been presented a source of identification. That is, even if two people scored exactly the same on their high school math and reading exams, but one went to community college and the other didn't, you still haven't accounted for whatever underlying differences existed to lead each to make a different choice.
- mediaman 17y agoWonderful commentary, thank you! Can you describe some other techniques used to eliminate unobserved variable bias, or do you know of a resource that describes these? I remember in one of the other studies on the impact of "elite" education versus state-provided university education, they used people who were accepted to an elite university, but chose not to attend. Is that using "acceptance" as an instrumental variable?
- stanleydrew 17y agoI've been out of the econ phd world for a little while now (see blog post here: http://hellosorld.com/personal/2009/07/16/why-i-quit-my-phd-in-economics.html http://hellosorld.com/personal/2009/07/16/why-i-quit-my-phd-...) so it's hard to remember what all the resources were, but i'll ask around for some good material and post it here. Natural random experiments are another good source of identification (i.e. elimination of unobserved variable bias). For the elite vs state university question, choosing to attend isn't an instrument since it's still an observed decision. If there was some source of randomness that was uncorrelated with all your other variables besides attending vs not, then that would be an instrumental variable. Say the government decided to take a pool of 10,000 elite college acceptees, gave them each a number from one to 10,000 and then said only the even numbers could actually attend. That would be your instrument. Trouble is that those kinds of natural random experiments don't occur often. That's why economists can get tenure simply by finding a good instrument for some study.