2 ms·

I'm not saying the claims aren't plausible, but it sounds like this could be p-hacking? "No other factor the researchers analyzed showed a statistically signifi

• ASpring 8y ago
  To me it looks more like due diligence. If you present only the main result every single reviewer will ask "Well couldn't the instructor's gender/ethnicity/tenure/experience explain this result?" Then you're forced to include this analysis either way.
  • foota 8y ago
    That's true. We can't know which came first though, I think the proposed solution to this is pre recording of methodology and hypothesis. Honestly, I'm inclined to believe this study was done ethically, but I wanted to point out the possibility.
    • cortesoft 8y ago
      Right, but I don’t think them saying they examined the other variables makes it more or less likely they were p-hacking.
• treis 8y ago
  It's not p-hacking as much as it is conflating statistically significant and actually significant. The fixed teachers gave whites/asians 7% higher marks while the growth teachers gave them 5% higher. That might be a statistically significant gap, but a 2% expected increase in grades is nothing. And way less of a difference than the gap between the grades fixed and growth teachers gave. Any gap can also be easily explained by grading styles. A fixed teacher probably places more emphasis on tests and the entirety of the work over the course of the semester. A growth teacher probably places more emphasis on homework and gives weight to improvement over the course of a semester. Asian/Whites with their relative head start over URMs would on average do a bit better under the first teacher.