4 ms·
Thanks for doing that, I found myself wishing they had put error bars on when I saw that graph but was too lazy to do it myself. With respect to the statistica
by JunkDNA 15y ago
Thanks for doing that, I found myself wishing they had put error bars on when I saw that graph but was too lazy to do it myself.
With respect to the statistical power issue, it may be they just didn't have enough students. Unlike your typical clinical trial or scientific study where you're recruiting people, I got the impression this was from an existing pool of students for a specific school. Once you go beyond this, the design gets tricky since there are practical issues to how big a class can be. Then you need more teachers, and you need to normalize for per-teacher variability. Not saying a study like that wouldn't be great, but it would be complex to get right.
- john_horton 15y agono problem & agreed about what probably happened - my point was more that knowing they had that sample size (n=40), someone could have told them that even if Kahn academy was extraordinarily effective, there was almost no chance they'd detect statistically significant effect. If they knew that, they might have pursued a more qualitative approach rather than have people see the take-away as "no effect."