3 ms·

That doesn't make the study any less flawed. I'm questioning whether we should even accept this study if there wasn't a big enough pool of black people to actua

by gdix 10y ago

That doesn't make the study any less flawed. I'm questioning whether we should even accept this study if there wasn't a big enough pool of black people to actually run the experiment properly. Isn't anyone at least a bit curious how an all black team would have performed? If the all black team performs as badly as the all white team, it would only bolster the argument that diversity is important. Which is a good result, because you've made a more convincing study. But what if the all black team performs the best? It would cause you to look at the study completely different, wouldn't it? Maybe for cultural reasons, black people are better at these kinds of puzzles. We'll never know because it seems like they have already decided this study in its current form is enough to conclude that "diversity=good".

gnicholas 10y ago
I'm not saying that one can't imagine a better study. I'm saying that the notion that the researchers committed a "sin" and that it did "not even occur to the researchers" to use a different design is unsupported. The author has no idea what was or was not considered by the researchers, but she is happy to impugn their motives. Given the choice between (1) run the study they ran, and have a large enough sample to get a statistically significant result and (2) don't run a study at all, which would you have preferred? With a population that is 5% black, it is likely that this is the practical choice the researchers faced. Note that (1) doesn't mean we get all the answers to everything we might wonder about, but it would give an answer that could help direct future research. That in and of itself is a useful step to take, IMO.
- gdix 10y ago
  The obvious answer is (2) don't run a study at all (assuming the methodology is truly flawed). Or (3) run the study but take the results with a grain of salt and don't draw broad conclusions from it. And (4) we can comment on hacker news about how "wouldn't it be nice if they tested this or that because that would have convinced me, personally." I think that we're kind of saying the same thing. It's an thought-provoking study and I agree that it is a useful step. But I also agree that further research is required to really draw a conclusion.

by gdix 10y ago

gnicholas 10y ago

I'm not saying that one can't imagine a better study. I'm saying that the notion that the researchers committed a "sin" and that it did "not even occur to the researchers" to use a different design is unsupported. The author has no idea what was or was not considered by the researchers, but she is happy to impugn their motives. Given the choice between (1) run the study they ran, and have a large enough sample to get a statistically significant result and (2) don't run a study at all, which would you have preferred? With a population that is 5% black, it is likely that this is the practical choice the researchers faced. Note that (1) doesn't mean we get all the answers to everything we might wonder about, but it would give an answer that could help direct future research. That in and of itself is a useful step to take, IMO.

gdix 10y ago
The obvious answer is (2) don't run a study at all (assuming the methodology is truly flawed). Or (3) run the study but take the results with a grain of salt and don't draw broad conclusions from it. And (4) we can comment on hacker news about how "wouldn't it be nice if they tested this or that because that would have convinced me, personally." I think that we're kind of saying the same thing. It's an thought-provoking study and I agree that it is a useful step. But I also agree that further research is required to really draw a conclusion.