3 ms·

I don't really understand removal of graduation rates. Obviously that's meant to act as a proxy for learning, especially for those 38 states without proficiency

I don't really understand removal of graduation rates. Obviously that's meant to act as a proxy for learning, especially for those 38 states without proficiency exams.

bigpicture 8y ago
Right, surely these states have some sort of standard that must be met for graduation even if it is not passing some exam.
- saudioger 8y ago
  the standards for graduation vary wildly so isn't really comparable
  - bigpicture 8y ago
    That's a good point
grandmczeb 8y ago
The standard to graduate isn’t uniform, so using graduation rates biases towards states with weak requirements.
ergothus 8y ago
> Obviously that's meant to act as a proxy for learning That is the INTENT, sure. But does it work? If you have someone that graduated from a school A and someone that failed to graduate from school B that is in an entirely different area, can you draw any meaningful conclusions? Particularly knowing that school A has no particular requirements on why you graduate, while school B does? I'm far more likely to complain about "But when we disaggregate student performance scores by racial categories (white, black, Hispanic, and Asian), the rankings change dramatically." I wouldn't want to go too far with that before ensuring that I wasn't trying to curb swimming deaths by keeping Nick Cage out of movies: http://tylervigen.com/view_correlation?id=359 http://tylervigen.com/view_correlation?id=359
- kolbe 8y ago
  The data is pretty useful in penalizing states that don't educate their students well enough to even get to the standardized testing stage.
btilly 8y ago
First a point about bias. They are quick to point out the bias of traditional scores. But they do not point out that their bias is for libertarian causes. Had they not found that more libertarian states perform well by their measure, they would never have published this. Removing graduation rates is definitely troublesome. Any school system could then improve its numbers by encouraging the worst students to drop out. Problem students both take more resources to educate and produce worse scores. That's bad for efficiency, but it is not obviously bad for society. I am also troubled that they are using measures of performance in grades 4 and 8, but not something like SAT scores. I understand that SAT scores could be hard to get. But a lot of what people care about in an education system is how well prepared you are for college, and not how far along you were in grade 8.
- the_watcher 8y ago
  > But they do not point out that their bias is for libertarian causes. Had they not found that more libertarian states perform well by their measure, they would never have published this. The first statement is true. The piece would be a bit stronger if the researchers noted that by publishing in Reason, they're at the very least giving the appearance of potential confirmation bias. Your second statement makes an unsupported assumption though. RE: SAT scores - not all students take the SAT, some students take it multiple times, the SAT is taken at different points in time per student, and it's a test that is so influential in college admissions that success on it is strongly correlated with wealth, as scores are best improved by tutoring on test taking strategy, not increasing general knowledge. I agree that grade 4 and 8 are probably not optimal to get a complete picture of education quality, but there is no reward to the student for doing well, so they're less likely to have spent substantial time investing in beating the test.

I don't really understand removal of graduation rates. Obviously that's meant to act as a proxy for learning, especially for those 38 states without proficiency exams.

bigpicture 8y ago

Right, surely these states have some sort of standard that must be met for graduation even if it is not passing some exam.

saudioger 8y ago
the standards for graduation vary wildly so isn't really comparable
- bigpicture 8y ago
  That's a good point

grandmczeb 8y ago

The standard to graduate isn’t uniform, so using graduation rates biases towards states with weak requirements.

ergothus 8y ago

> Obviously that's meant to act as a proxy for learning That is the INTENT, sure. But does it work? If you have someone that graduated from a school A and someone that failed to graduate from school B that is in an entirely different area, can you draw any meaningful conclusions? Particularly knowing that school A has no particular requirements on why you graduate, while school B does? I'm far more likely to complain about "But when we disaggregate student performance scores by racial categories (white, black, Hispanic, and Asian), the rankings change dramatically." I wouldn't want to go too far with that before ensuring that I wasn't trying to curb swimming deaths by keeping Nick Cage out of movies: http://tylervigen.com/view_correlation?id=359 http://tylervigen.com/view_correlation?id=359

kolbe 8y ago
The data is pretty useful in penalizing states that don't educate their students well enough to even get to the standardized testing stage.

btilly 8y ago

First a point about bias. They are quick to point out the bias of traditional scores. But they do not point out that their bias is for libertarian causes. Had they not found that more libertarian states perform well by their measure, they would never have published this. Removing graduation rates is definitely troublesome. Any school system could then improve its numbers by encouraging the worst students to drop out. Problem students both take more resources to educate and produce worse scores. That's bad for efficiency, but it is not obviously bad for society. I am also troubled that they are using measures of performance in grades 4 and 8, but not something like SAT scores. I understand that SAT scores could be hard to get. But a lot of what people care about in an education system is how well prepared you are for college, and not how far along you were in grade 8.

the_watcher 8y ago
> But they do not point out that their bias is for libertarian causes. Had they not found that more libertarian states perform well by their measure, they would never have published this. The first statement is true. The piece would be a bit stronger if the researchers noted that by publishing in Reason, they're at the very least giving the appearance of potential confirmation bias. Your second statement makes an unsupported assumption though. RE: SAT scores - not all students take the SAT, some students take it multiple times, the SAT is taken at different points in time per student, and it's a test that is so influential in college admissions that success on it is strongly correlated with wealth, as scores are best improved by tutoring on test taking strategy, not increasing general knowledge. I agree that grade 4 and 8 are probably not optimal to get a complete picture of education quality, but there is no reward to the student for doing well, so they're less likely to have spent substantial time investing in beating the test.