4 ms·

Who would have thought that statistics could be improved by eliminating bad data points?

by aredox 1y ago

pc86 1y ago
I don't want to strawman your argument but it sounds like you're saying that if you're in 3rd grade one year you should be in 4th grade the next year no matter what. That there's nothing you actually need to learn in 3rd grade in order to be advanced to 4th grade. Is that what you're saying?
- bombcar 1y ago
  I would say that the point is that you can't just look at one datapoint, especially if there are other things affecting it. The most obvious case of this is comparing private vs public schools, where the private schools can be selective and kick out anyone who doesn't perform or they don't like, but the public schools have to accept everyone by law. Obviously failing anyone who cannot read from getting to 4th grade will greatly improve 8th grade reading scores.
  - seanmcdirmid 1y ago
    Those failing kids eventually make it to the 8th grade, however, and affect statistics. Still, having lived there and attending one of the better middle and high schools near Vicksburg, I wouldn’t be surprised if they were gaming the system in some way (I hope they aren’t and these gains are real, though).
    - directevolve 1y ago
      If a kid achieves a great 8th grade test score at age 18, is that a success or a failure of the system? What we care about is the level of achievement by a given age. To determine that, we need to be comparing states using standardized tests given to age groups, not grade levels. It is fine to hold students back, if we think that will do them more good than advancing them. But they still need to be tested the same way as their age group if we want to do a meaningful comparison between states.
      - ryathal 1y ago
        If an 18 year old achieves a great score on an 8th grade test they are above average for adults.
      - seanmcdirmid 1y ago
        If the kid is held back and not failed forward, at least they get a chance to fix things.
- aredox 1y ago
  You are strawmanning my argument as I didn't say anything like that. I said that if you are going to evaluate a policy with statistics, you need to compare apples to apples because statistics are easily biased. See this example of a paradox that applies a lot in educational settings: you can raise the average level of two classes just by shuffling students from one to another: https://en.wikipedia.org/wiki/Will_Rogers_phenomenon https://en.wikipedia.org/wiki/Will_Rogers_phenomenon
  - pc86 1y ago
    So explain to me what "eliminating bad data points" in this context means. Should MS schools not hold back failing 3rd graders?
    - motorest 1y ago
      > So explain to me what "eliminating bad data points" in this context means. Should MS schools not hold back failing 3rd graders The data point is the number of 3rd graders failing. If you insist in filtering out those 3rd graders, limiting your analysis to the subset of kids who didn't failed does not represent a success story. It represents an attempt to arbitrarily remove inconvenient data points to portray a false idea if success.
      - pc86 1y ago
        I disagree, I think it points to a core educational policy difference between states. Some states will not fail a 3rd grader, and Mississippi will. This has an obvious impact on 4th grade scores, yes, but I'm willing to bet if you followed those "failed" 3rd graders in MS and compared to other states where they were pushed ahead, holding under-achieving students back is a net positive.
        motorest 1y ago
        > (...) holding under-achieving students back is a net positive. Even if we assume that's the case, that's not the problem. The problem is that the school system fails to provide the necessary and sufficient services that would prevent a statistically significant number of 3rd graders from being held back. Feeling the need to hold kids back is a symptom of the problem, not a solution.
- motorest 1y ago
  > I don't want to strawman your argument but it sounds like you're saying that if you're in 3rd grade one year you should be in 4th grade the next year no matter what. If a school system is designed so that the average kid in 3rd grade is expected to be in 4th grade the following year, the fact that a statistically significant subset of kids is not able to meet that bar is a sign that the system is failing those kids. What's the goal here? Is it to get pretty metrics by filtering out the failures, or is it to provide an effective education to all kids?
  - pc86 1y ago
    How do you know its statistically significant? Nothing in the article (or anywhere else I looked) suggests a "statistically significant" portion of 3rd graders, whatever that means, are being held back.
    - motorest 1y ago
      > How do you know its statistically significant? Because I bothered to look it up. In the last few years, Mississippi has been holding back between 5-10% of it's students.
deleted 1y ago
[deleted]

Who would have thought that statistics could be improved by eliminating bad data points?

by aredox 1y ago

Who would have thought that statistics could be improved by eliminating bad data points?

pc86 1y ago

I don't want to strawman your argument but it sounds like you're saying that if you're in 3rd grade one year you should be in 4th grade the next year no matter what. That there's nothing you actually need to learn in 3rd grade in order to be advanced to 4th grade. Is that what you're saying?

bombcar 1y ago
I would say that the point is that you can't just look at one datapoint, especially if there are other things affecting it. The most obvious case of this is comparing private vs public schools, where the private schools can be selective and kick out anyone who doesn't perform or they don't like, but the public schools have to accept everyone by law. Obviously failing anyone who cannot read from getting to 4th grade will greatly improve 8th grade reading scores.
- seanmcdirmid 1y ago
  Those failing kids eventually make it to the 8th grade, however, and affect statistics. Still, having lived there and attending one of the better middle and high schools near Vicksburg, I wouldn’t be surprised if they were gaming the system in some way (I hope they aren’t and these gains are real, though).
  - directevolve 1y ago
    If a kid achieves a great 8th grade test score at age 18, is that a success or a failure of the system? What we care about is the level of achievement by a given age. To determine that, we need to be comparing states using standardized tests given to age groups, not grade levels. It is fine to hold students back, if we think that will do them more good than advancing them. But they still need to be tested the same way as their age group if we want to do a meaningful comparison between states.
    - ryathal 1y ago
      If an 18 year old achieves a great score on an 8th grade test they are above average for adults.
    - seanmcdirmid 1y ago
      If the kid is held back and not failed forward, at least they get a chance to fix things.
aredox 1y ago
You are strawmanning my argument as I didn't say anything like that. I said that if you are going to evaluate a policy with statistics, you need to compare apples to apples because statistics are easily biased. See this example of a paradox that applies a lot in educational settings: you can raise the average level of two classes just by shuffling students from one to another: https://en.wikipedia.org/wiki/Will_Rogers_phenomenon https://en.wikipedia.org/wiki/Will_Rogers_phenomenon
- pc86 1y ago
  So explain to me what "eliminating bad data points" in this context means. Should MS schools not hold back failing 3rd graders?
  - motorest 1y ago
    > So explain to me what "eliminating bad data points" in this context means. Should MS schools not hold back failing 3rd graders The data point is the number of 3rd graders failing. If you insist in filtering out those 3rd graders, limiting your analysis to the subset of kids who didn't failed does not represent a success story. It represents an attempt to arbitrarily remove inconvenient data points to portray a false idea if success.
    - pc86 1y ago
      I disagree, I think it points to a core educational policy difference between states. Some states will not fail a 3rd grader, and Mississippi will. This has an obvious impact on 4th grade scores, yes, but I'm willing to bet if you followed those "failed" 3rd graders in MS and compared to other states where they were pushed ahead, holding under-achieving students back is a net positive.
      - motorest 1y ago
        > (...) holding under-achieving students back is a net positive. Even if we assume that's the case, that's not the problem. The problem is that the school system fails to provide the necessary and sufficient services that would prevent a statistically significant number of 3rd graders from being held back. Feeling the need to hold kids back is a symptom of the problem, not a solution.
motorest 1y ago
> I don't want to strawman your argument but it sounds like you're saying that if you're in 3rd grade one year you should be in 4th grade the next year no matter what. If a school system is designed so that the average kid in 3rd grade is expected to be in 4th grade the following year, the fact that a statistically significant subset of kids is not able to meet that bar is a sign that the system is failing those kids. What's the goal here? Is it to get pretty metrics by filtering out the failures, or is it to provide an effective education to all kids?
- pc86 1y ago
  How do you know its statistically significant? Nothing in the article (or anywhere else I looked) suggests a "statistically significant" portion of 3rd graders, whatever that means, are being held back.
  - motorest 1y ago
    > How do you know its statistically significant? Because I bothered to look it up. In the last few years, Mississippi has been holding back between 5-10% of it's students.

deleted 1y ago

[deleted]