3 ms·

The author is arguing that because the measurement is noisy, we should ignore it. Not quite.. Indeed - if 1/3 of my salary was determined by the number of kid

The author is arguing that because the measurement is noisy, we should ignore it. Not quite.. Indeed - if 1/3 of my salary was determined by the number of kids in my class who can read better than when they are predicted to read at this age, I'd definitely try to make sure that reading skills improved. Keep in mind that "score on a reading test" and "reading skills" are not the same things. For instance, one would be significantly improved by spending valuable class time teaching students tricks about how to succeed on the standardized reading test. It's similar to the incentives you'd get by determining salary by LOC - even if LOC correlates with performance, basing pay on it encourages all kinds of nonsense that doesn't actually benefit anyone. So in that context, we have a metric that very loosely correlates with this clearly flawed marker of success. And we're supposed to spend ungodly sums of money implementing this strategy? Come on. Conversely, if a stupid metric such as student/principal opinion were used, I'd focus on jokes and friendliness over education. Well, they do propose to use student and principal evaluations..

yummyfajitas 14y ago
Keep in mind that "score on a reading test" and "reading skills" are not the same things. True. Score on a reading test might be lower than actual reading skills due to a lack of test taking skills. Then again, I can't think of any measurable quantity more highly correlated with reading skills than scoring high on a reading test. Do you have evidence that "principal's opinion of teacher" is better? ...we have a metric that very loosely correlates... You are confused. GR shows that year on year correlations are weak (0.3), not that correlation between reading test and reading skills are weak. You are correct that a certain nonzero amount of teaching test skills will be effective at improving scores. I see no reason to believe this amount will be large - do you? Again, standardized tests are not the only gameable metric. Principal/student opinion is as well. Why do you believe they are less gameable and more accurate than directly measuring student ability?
- streptomycin 14y ago
  It's not that I think this is the only flawed metric or that there are other methods that are quantifiably better. It's that we've spent ridiculous sums of time and money on this complicated new metric, despite some pretty clear arguments against it, and we still are very far away from actually proving that it is better than the status quo, even if that status quo obviously falls somewhat short of perfection. It's like.. how much money and effort do you think is worthwhile to spend on something with such uncertain prospects? There are tons of great grant proposals to funding agencies which are rejected. Maybe we would be better off funding some of those. Opportunity costs exist, and it seems like undue sums of money are being wasted on this stuff.

yummyfajitas 14y ago

Keep in mind that "score on a reading test" and "reading skills" are not the same things. True. Score on a reading test might be lower than actual reading skills due to a lack of test taking skills. Then again, I can't think of any measurable quantity more highly correlated with reading skills than scoring high on a reading test. Do you have evidence that "principal's opinion of teacher" is better? ...we have a metric that very loosely correlates... You are confused. GR shows that year on year correlations are weak (0.3), not that correlation between reading test and reading skills are weak. You are correct that a certain nonzero amount of teaching test skills will be effective at improving scores. I see no reason to believe this amount will be large - do you? Again, standardized tests are not the only gameable metric. Principal/student opinion is as well. Why do you believe they are less gameable and more accurate than directly measuring student ability?

streptomycin 14y ago
It's not that I think this is the only flawed metric or that there are other methods that are quantifiably better. It's that we've spent ridiculous sums of time and money on this complicated new metric, despite some pretty clear arguments against it, and we still are very far away from actually proving that it is better than the status quo, even if that status quo obviously falls somewhat short of perfection. It's like.. how much money and effort do you think is worthwhile to spend on something with such uncertain prospects? There are tons of great grant proposals to funding agencies which are rejected. Maybe we would be better off funding some of those. Opportunity costs exist, and it seems like undue sums of money are being wasted on this stuff.