I'll have to look at the prior reports, but this 3rd report is somewhat disappointing in terms of its statistical rigor. They do provide weightings based on op

I'll have to look at the prior reports, but this 3rd report is somewhat disappointing in terms of its statistical rigor. They do provide weightings based on optimal predictions for maximum accuracy with predictions for state tests scores, although those weights appear to assume a simple linear correlation. Even then, the data suggests that in class observations are almost a complete waste of time (single digit weightings), and that student surveys decline in value as the kids mature (no shocker: with younger kids, how much they like the teacher has a much more profound impact on their success at learning). In all cases the kids seem to be a more reliable indicator than the observations of professionals! This may just be an indication that assessments based on more frequent observations are more accurate, but the only way we can really get a sense of that is if they also had peer observations from team teaching environments... which are completely absent. That's not the worst of it though. The worst is the rest of the data. It is based on some fairly arbitrary presupposed models for combining these different assessments. No breakdown of the individual components of these assessments (other than "English" & "Math") and no attempt to discern a model from first principles. There are some other factors as well. Aside from some outliers, the accuracy of predictions seems kind of suspect. Most of the outcomes are within +/-0.05 standard deviations, and a majority of them are within what looks like +/-0.025 standard deviations. These are pretty minuscule variances (0.25 was supposed to represent a year of schooling, so 0.025 represents maybe 15 school days worth of progress at best), and it is not hard to see that without some of those outliers, the regression line doesn't look so pretty (particularly for English). I also didn't see any metrics on the variance of the data within a classroom, which I suspect makes any predictive value look limited at best. Ugh. I really hoped that Gates would make sure there was more statistical rigor with an analysis like this. :-( I hate to say it, but the conclusions of the study are not well supported by at least what is in the final research report.