25 ms·

one issue could be the study design. I can't quite tell from the NYT writeup, but it seems that the test graded by the teacher was a different test than the one

one issue could be the study design. I can't quite tell from the NYT writeup, but it seems that the test graded by the teacher was a different test than the one graded by outside teachers. That could definitely have an impact

gwern 12y ago
Fulltext: http://sci-hub.org/downloads/277d/10.0000@www.nber.org@generic-7242DB741D24.pdf http://sci-hub.org/downloads/277d/10.0000@www.nber.org@gener... Correct, the tests are very different: "We measure teachers’ gender biased behavior by comparing their average marking of boys’ and girls’ in a “non-blind” classroom exam to the respective means in a “blind” national exam marked anonymously. To construct a measure of teachers' biased behavior we combine the scores from the GEMS 5th grade external exam with those of internal exams held in the middle of 6 th grade. The GEMS test scores is a “blind” assessment since the GEMS exams are graded by an independent agency where at no stage are the identity and gender of the student revealed. In contrast, the internal exam is graded by the student’s teacher and therefore it is a “non-blind” assessment. We assume that this measure of teacher’s stereotypes captures her/his overall perception about gender cognitive differences and we use it as a proxy for her/his level of prejudice and discriminatory behavior in class . 15" (The paper is surprisingly quiet about what the "internal exams" actually are.)
- jquery 12y ago
  Doesn't matter how badly the study was constructed, the mission has been accomplished. Coming soon to a recommended article on Medium near you.
  - gwern 12y ago
    Yeah, I've learned to not expect anything good of any study which is about gap-busting. The topic draws terrible studies like flies. (Seriously, it's kind of boggling going back and looking. The entire study rests on the exact comparability of the two sets of tests allowing claims of teacher bias by gender to then give later causal effects... and not only can they not establish the comparability in any remotely rigorous way, they don't even tell you what one set really is!)

gwern 12y ago

Fulltext: http://sci-hub.org/downloads/277d/10.0000@www.nber.org@generic-7242DB741D24.pdf http://sci-hub.org/downloads/277d/10.0000@www.nber.org@gener... Correct, the tests are very different: "We measure teachers’ gender biased behavior by comparing their average marking of boys’ and girls’ in a “non-blind” classroom exam to the respective means in a “blind” national exam marked anonymously. To construct a measure of teachers' biased behavior we combine the scores from the GEMS 5th grade external exam with those of internal exams held in the middle of 6 th grade. The GEMS test scores is a “blind” assessment since the GEMS exams are graded by an independent agency where at no stage are the identity and gender of the student revealed. In contrast, the internal exam is graded by the student’s teacher and therefore it is a “non-blind” assessment. We assume that this measure of teacher’s stereotypes captures her/his overall perception about gender cognitive differences and we use it as a proxy for her/his level of prejudice and discriminatory behavior in class . 15" (The paper is surprisingly quiet about what the "internal exams" actually are.)

jquery 12y ago
Doesn't matter how badly the study was constructed, the mission has been accomplished. Coming soon to a recommended article on Medium near you.
- gwern 12y ago
  Yeah, I've learned to not expect anything good of any study which is about gap-busting. The topic draws terrible studies like flies. (Seriously, it's kind of boggling going back and looking. The entire study rests on the exact comparability of the two sets of tests allowing claims of teacher bias by gender to then give later causal effects... and not only can they not establish the comparability in any remotely rigorous way, they don't even tell you what one set really is!)