3 ms·

If you want to build a school system around maximizing scores on a very particular type of standardized test then their methodology makes sense. Otherwise, it'

If you want to build a school system around maximizing scores on a very particular type of standardized test then their methodology makes sense. Otherwise, it's just weighing one questionable feature more than another questionable feature. Test scores do NOT provide a holistic picture of the quality of a school system. Neither do funding levels. BOTH of those things can be increased while not improving or even harming actual educational achievement. They are FEATURES, and reasonable models can weigh them in different ways. The assertion that funding levels might be positively correlated with (unmeasurable -- you've conceded this point!) true outcomes is not unreasonable. The assessments the authors are criticizing actually have it completely right. Both test scores and funding levels are PROXIES for the actual quality of education that's happening. They are features. It's unclear how to weight these features against one another, but the conjecture that either of those features might be correlative or even causative is totally reasonable. The authors, however, go off the deep end and completely mistake one particular map (and a damn bad one at that) for the territory. If the authors' assertion made any sense, then we could replace all teachers with DRL.

darawk 8y ago
> The assertion that funding levels might be positively correlated with (unmeasurable -- you've conceded this point!) true outcomes is not unreasonable. No it's not unreasonable, as a hypothesis. But if you wanted to, you know, test that hypothesis, guess what you'd need: performance measurement. Which means that the only way you can prove that spending matters is by measuring its impact on performance, which is already being measured by the other variables. So, there is zero reason to believe that incorporating spending as an additional variable can improve your assessment of quality.
- thrmsforbfast 8y ago
  Unless those tests are NOT measuring performance in a meaningful way. Once we assert that those tests are not measuring performance, the claim that performance is "already being measured" is just wrong and we're back to square one.
  - sgc 8y ago
    To add to this, they need not be wrong (and really should not be to make the argument), just significantly incomplete. From what I gathered from this thread, this was already agreed upon by everyone. So in the face of a known insufficiency of objective measures, and an identified correlation between a common statistic and some identified metrics, it is legitimate to use that correlating factor to help identify better performing school systems which include the as yet unidentified measures until better measurements can be objectively identified. But one must acknowledge at the same time this is not ideal, and much more effort should be made to understand the true nature of education beyond simple test or economic results, ie, in my opinion, a better understanding of philosophy and the nature of the human being, so that we can understand what truly embodies a good education. As a tangent, may all multiple choice tests die an eternal death. They have only one redeeming quality. They are easier to grade. And even that metric is dead for many subjects in today's world. One of their more unredeeming qualities is they hide ignorance and are just too easy. All you need is a minimum of a bullshit radar to get to a 50-50 chance of being right most of the time (so if you know 70%, you will probably score 85%). Another good chunk of it, that same radar is enough for 100% accuracy due to the ridiculous alternative choices. Hell, I got bored on my PSAT and got a Mensa invitation for my score because I (intuitively) picked up on a pattern in the answers and filled it in for the last 30-40% of the test.
    - darawk 8y ago
      > So in the face of a known insufficiency of objective measures, and an identified correlation between a common statistic and some identified metrics, it is legitimate to use that correlating factor to help identify better performing school systems which include the as yet unidentified measures until better measurements can be objectively identified. How do you figure that? The correlating metric only explains the variance it correlates with. But we're already measuring that variance directly.
      - thrmsforbfast 8y ago
        > But we're already measuring that variance directly. >> To add to this, they need not be wrong (and really should not be to make the argument), just significantly incomplete. From what I gathered from this thread, this was already agreed upon by everyone. ...and... >>> Unless those tests are NOT measuring performance in a meaningful way. ...and... >>>> Otherwise, it's just weighing one questionable feature more than another questionable feature. ...and... >>>>(or something)>>> I think the fundamental thesis of the article -- that we can scientifically manage a system of universal education -- is fundamentally and impossibly flawed. Especially given the current state of assessments and the incentive structures built around them. And so on... So there's a general agreement that the current tests (and/or the social system they're embedded in) are extremely flawed and at least a contest over assertions about whether good enough tests event exist. Which is to say, for the fifth(?) time, the basic problem is that standardized tests are not doing even a remotely reasonable job at "measuring that variance directly". So pretending like they are legitimate metrics (and/or any more legitimate than other fairly broken metrics like funding levels) for measuring overall educational attainment is at best flawed and at worst -- I assert very likely! -- the very source of the whole problem with K12 in this country. On that note, I would like to close out this thread by imploring you to read the essay I linked to in my first post: https://www.maa.org/external_archive/devlin/LockhartsLament.pdf https://www.maa.org/external_archive/devlin/LockhartsLament....

darawk 8y ago

> The assertion that funding levels might be positively correlated with (unmeasurable -- you've conceded this point!) true outcomes is not unreasonable. No it's not unreasonable, as a hypothesis. But if you wanted to, you know, test that hypothesis, guess what you'd need: performance measurement. Which means that the only way you can prove that spending matters is by measuring its impact on performance, which is already being measured by the other variables. So, there is zero reason to believe that incorporating spending as an additional variable can improve your assessment of quality.

thrmsforbfast 8y ago
Unless those tests are NOT measuring performance in a meaningful way. Once we assert that those tests are not measuring performance, the claim that performance is "already being measured" is just wrong and we're back to square one.
- sgc 8y ago
  To add to this, they need not be wrong (and really should not be to make the argument), just significantly incomplete. From what I gathered from this thread, this was already agreed upon by everyone. So in the face of a known insufficiency of objective measures, and an identified correlation between a common statistic and some identified metrics, it is legitimate to use that correlating factor to help identify better performing school systems which include the as yet unidentified measures until better measurements can be objectively identified. But one must acknowledge at the same time this is not ideal, and much more effort should be made to understand the true nature of education beyond simple test or economic results, ie, in my opinion, a better understanding of philosophy and the nature of the human being, so that we can understand what truly embodies a good education. As a tangent, may all multiple choice tests die an eternal death. They have only one redeeming quality. They are easier to grade. And even that metric is dead for many subjects in today's world. One of their more unredeeming qualities is they hide ignorance and are just too easy. All you need is a minimum of a bullshit radar to get to a 50-50 chance of being right most of the time (so if you know 70%, you will probably score 85%). Another good chunk of it, that same radar is enough for 100% accuracy due to the ridiculous alternative choices. Hell, I got bored on my PSAT and got a Mensa invitation for my score because I (intuitively) picked up on a pattern in the answers and filled it in for the last 30-40% of the test.
  - darawk 8y ago
    > So in the face of a known insufficiency of objective measures, and an identified correlation between a common statistic and some identified metrics, it is legitimate to use that correlating factor to help identify better performing school systems which include the as yet unidentified measures until better measurements can be objectively identified. How do you figure that? The correlating metric only explains the variance it correlates with. But we're already measuring that variance directly.
    - thrmsforbfast 8y ago
      > But we're already measuring that variance directly. >> To add to this, they need not be wrong (and really should not be to make the argument), just significantly incomplete. From what I gathered from this thread, this was already agreed upon by everyone. ...and... >>> Unless those tests are NOT measuring performance in a meaningful way. ...and... >>>> Otherwise, it's just weighing one questionable feature more than another questionable feature. ...and... >>>>(or something)>>> I think the fundamental thesis of the article -- that we can scientifically manage a system of universal education -- is fundamentally and impossibly flawed. Especially given the current state of assessments and the incentive structures built around them. And so on... So there's a general agreement that the current tests (and/or the social system they're embedded in) are extremely flawed and at least a contest over assertions about whether good enough tests event exist. Which is to say, for the fifth(?) time, the basic problem is that standardized tests are not doing even a remotely reasonable job at "measuring that variance directly". So pretending like they are legitimate metrics (and/or any more legitimate than other fairly broken metrics like funding levels) for measuring overall educational attainment is at best flawed and at worst -- I assert very likely! -- the very source of the whole problem with K12 in this country. On that note, I would like to close out this thread by imploring you to read the essay I linked to in my first post: https://www.maa.org/external_archive/devlin/LockhartsLament.pdf https://www.maa.org/external_archive/devlin/LockhartsLament....