3 ms·

> But spending is never a good measurement of achievement. Nobody said it was. > You think it is impossible to measure educational performance at all? Surely

> But spending is never a good measurement of achievement. Nobody said it was. > You think it is impossible to measure educational performance at all? Surely you don't believe something that silly. Obviously not. That's a ridiculous strawman and I think you know it. > But i'm not sure what your point is here. My point is that the availability of resources IS a useful metric. > Certainly imperfect measurements don't mean we should just measure random, unrelated variables like spending. How is availability of resources unrelated? The idea that the level of resources available is irrelevant strikes me as fairly insane. You're really saying that you'd have the same quality teachers as $40k/yr as at $90k/yr or $500,000k/yr? CLEARLY resources effect outcomes. It's not the whole story, but it is a useful feature. To claim it's completely irrelevant because "oh look multiple choice exams that everyone has been systematically gaming for 20+ years" seems absolutely insane to me. > I also don't see how it means we shouldn't try to measure, and improve our measurement standards. Sure. > And it certainly doesn't mean, that the conclusion of this article is in any way incorrect (which is not that the measurements are good, but that the way other people were using these measurements were even less good than using them correctly). I think the fundamental thesis of the article -- that we can scientifically manage a system of universal education -- is fundamentally and impossibly flawed. Especially given the current state of assessments and the incentive structures built around them.

darawk 8y ago
Oh, but they did, from the article: > We fixed two serious problems common to traditional rankings. First, we removed factors that do not measure K–12 student performance or teaching effectiveness, such as spending per student (intentions to raise performance are not the same as raising performance)
- thrmsforbfast 8y ago
  If you want to build a school system around maximizing scores on a very particular type of standardized test then their methodology makes sense. Otherwise, it's just weighing one questionable feature more than another questionable feature. Test scores do NOT provide a holistic picture of the quality of a school system. Neither do funding levels. BOTH of those things can be increased while not improving or even harming actual educational achievement. They are FEATURES, and reasonable models can weigh them in different ways. The assertion that funding levels might be positively correlated with (unmeasurable -- you've conceded this point!) true outcomes is not unreasonable. The assessments the authors are criticizing actually have it completely right. Both test scores and funding levels are PROXIES for the actual quality of education that's happening. They are features. It's unclear how to weight these features against one another, but the conjecture that either of those features might be correlative or even causative is totally reasonable. The authors, however, go off the deep end and completely mistake one particular map (and a damn bad one at that) for the territory. If the authors' assertion made any sense, then we could replace all teachers with DRL.
  - darawk 8y ago
    > The assertion that funding levels might be positively correlated with (unmeasurable -- you've conceded this point!) true outcomes is not unreasonable. No it's not unreasonable, as a hypothesis. But if you wanted to, you know, test that hypothesis, guess what you'd need: performance measurement. Which means that the only way you can prove that spending matters is by measuring its impact on performance, which is already being measured by the other variables. So, there is zero reason to believe that incorporating spending as an additional variable can improve your assessment of quality.
    - thrmsforbfast 8y ago
      Unless those tests are NOT measuring performance in a meaningful way. Once we assert that those tests are not measuring performance, the claim that performance is "already being measured" is just wrong and we're back to square one.
      - sgc 8y ago
        To add to this, they need not be wrong (and really should not be to make the argument), just significantly incomplete. From what I gathered from this thread, this was already agreed upon by everyone. So in the face of a known insufficiency of objective measures, and an identified correlation between a common statistic and some identified metrics, it is legitimate to use that correlating factor to help identify better performing school systems which include the as yet unidentified measures until better measurements can be objectively identified. But one must acknowledge at the same time this is not ideal, and much more effort should be made to understand the true nature of education beyond simple test or economic results, ie, in my opinion, a better understanding of philosophy and the nature of the human being, so that we can understand what truly embodies a good education. As a tangent, may all multiple choice tests die an eternal death. They have only one redeeming quality. They are easier to grade. And even that metric is dead for many subjects in today's world. One of their more unredeeming qualities is they hide ignorance and are just too easy. All you need is a minimum of a bullshit radar to get to a 50-50 chance of being right most of the time (so if you know 70%, you will probably score 85%). Another good chunk of it, that same radar is enough for 100% accuracy due to the ridiculous alternative choices. Hell, I got bored on my PSAT and got a Mensa invitation for my score because I (intuitively) picked up on a pattern in the answers and filled it in for the last 30-40% of the test.

darawk 8y ago

Oh, but they did, from the article: > We fixed two serious problems common to traditional rankings. First, we removed factors that do not measure K–12 student performance or teaching effectiveness, such as spending per student (intentions to raise performance are not the same as raising performance)

thrmsforbfast 8y ago
If you want to build a school system around maximizing scores on a very particular type of standardized test then their methodology makes sense. Otherwise, it's just weighing one questionable feature more than another questionable feature. Test scores do NOT provide a holistic picture of the quality of a school system. Neither do funding levels. BOTH of those things can be increased while not improving or even harming actual educational achievement. They are FEATURES, and reasonable models can weigh them in different ways. The assertion that funding levels might be positively correlated with (unmeasurable -- you've conceded this point!) true outcomes is not unreasonable. The assessments the authors are criticizing actually have it completely right. Both test scores and funding levels are PROXIES for the actual quality of education that's happening. They are features. It's unclear how to weight these features against one another, but the conjecture that either of those features might be correlative or even causative is totally reasonable. The authors, however, go off the deep end and completely mistake one particular map (and a damn bad one at that) for the territory. If the authors' assertion made any sense, then we could replace all teachers with DRL.
- darawk 8y ago
  > The assertion that funding levels might be positively correlated with (unmeasurable -- you've conceded this point!) true outcomes is not unreasonable. No it's not unreasonable, as a hypothesis. But if you wanted to, you know, test that hypothesis, guess what you'd need: performance measurement. Which means that the only way you can prove that spending matters is by measuring its impact on performance, which is already being measured by the other variables. So, there is zero reason to believe that incorporating spending as an additional variable can improve your assessment of quality.
  - thrmsforbfast 8y ago
    Unless those tests are NOT measuring performance in a meaningful way. Once we assert that those tests are not measuring performance, the claim that performance is "already being measured" is just wrong and we're back to square one.
    - sgc 8y ago
      To add to this, they need not be wrong (and really should not be to make the argument), just significantly incomplete. From what I gathered from this thread, this was already agreed upon by everyone. So in the face of a known insufficiency of objective measures, and an identified correlation between a common statistic and some identified metrics, it is legitimate to use that correlating factor to help identify better performing school systems which include the as yet unidentified measures until better measurements can be objectively identified. But one must acknowledge at the same time this is not ideal, and much more effort should be made to understand the true nature of education beyond simple test or economic results, ie, in my opinion, a better understanding of philosophy and the nature of the human being, so that we can understand what truly embodies a good education. As a tangent, may all multiple choice tests die an eternal death. They have only one redeeming quality. They are easier to grade. And even that metric is dead for many subjects in today's world. One of their more unredeeming qualities is they hide ignorance and are just too easy. All you need is a minimum of a bullshit radar to get to a 50-50 chance of being right most of the time (so if you know 70%, you will probably score 85%). Another good chunk of it, that same radar is enough for 100% accuracy due to the ridiculous alternative choices. Hell, I got bored on my PSAT and got a Mensa invitation for my score because I (intuitively) picked up on a pattern in the answers and filled it in for the last 30-40% of the test.