4 ms·

If the test is good, then “teaching to the test” is desirable.

SketchySeaBeast 1y ago
Have we found an exception to Goodhart's Law? "When a measure becomes a target, it ceases to be a good measure, except when the measure is good."
- DougN7 1y ago
  … so if the test is good … Reading is such a basic skill it seems like a pretty reasonable thing to come up with a decent test.
  - SketchySeaBeast 1y ago
    "When a measure becomes a target, it ceases to be a good measure, except when the measure is something basic, like teaching reading"? Has the last twenty years of teaching to the test proven that out?
    - anon291 1y ago
      'Teaching to the test' has not caused the drop in literacy. Rather, it's teaching reading using bizarre methods, like whole word recognition instead of phonics. This has nothing to do with 'teaching to the test', since phonics would teach more to the test than whole word anything. A phonetically competent adult would be able to make out almost any English word. A whole word one would not.
      - SketchySeaBeast 1y ago
        "When a measure becomes a target, it ceases to be a good measure, unless the teacher is poor"?
        anon291 1y ago
        This 'truth' is not always true. For binary questions, teaching to measure makes sense
        deleted 1y ago
        [deleted]
        SketchySeaBeast 1y ago
        Unfortunately, "is someone literate" is not a binary question. There's a whole range of possible answers there.
    - DougN7 1y ago
      Is there a way to teach reading, which increases reading test scores, that is bad? I’m having a hard time imagining what that would look like. Just because someone comes up with a pithy saying that some people call a “law” doesn’t make it a universal truth.
- throwawaymaths 1y ago
  you trying to establish some sort of measure for Goodhart's law?
- viraptor 1y ago
  I don't think so. You can still teach to a good test badly. For example: reading a variety of books and letting people do more on their own would be would be better overall than reading just things on the level of the test, asking just the test style questions and spending time learning the answering style to get 100%. It's kind of like those articles about young kids passing IT certifications. Those tests are reasonably good, but those kids just memorised lots of material and never actually worked as, for example, a networking engineer. Or like in IELTS style language test you can easily score higher (even than a native English speaker) if you learn the format enough.
andrewflnr 1y ago
The test is never that good.
- larrled 1y ago
  In education, nothing is ever good enough.
  - no_wizard 1y ago
    This is my take: the real issue with education in the US goes back to trying to square two ideals. One is the meritocracy[0]. I believe that No Child Left Behind Act was sold to the public as a way to promote meritocracy in education and having some reasonably unbiased[1] way of doing so. The foundation of which is a way for parents to hold institutions responsible much more easily[2][3] The other is social and political stratification. This isn't merit based. Good examples of this would be the history of slavery, Jim Crow laws, the formerly legal practice of segregation by race, the drawing school district lines that keep students of wealthy parents away from folks who come from less economically advantaged backgrounds and enforcing this with district based school eligibility and funding. These two don't exactly square very well. Which is why the top level idea sold to the public - meritocracy - is used to sell these policies, but in practice they are using it to pull levers to further political and social stratification. [0]: Americans generally have a positive view of the term. I personally believe that most people don't have a proper understanding of the term to begin with, nor conceive what issues it could have in practice if they were given it. [1]: Leaving out the other half of the narrative, which is how much macro and micro levels of a students environment will influence how well they learn. [2]: Which made it easier for Republicans to continue to go after public education. They've been pushing for the voucher system for decades as a backdoor way to dismantle the public school system, reasons for wanting to do so vary but it all converges on this point. [3]: Not to mention, it completely ignores micro and macro issues that students can have, be it poverty, domestic issues, systemic issues and a whole host of other things.
    - throwawaymaths 1y ago
      > Which made it easier for Republicans to continue to go after public education. They've been pushing for the voucher system for decades as a backdoor way to dismantle the public school system, reasons for wanting to do so vary but it all converges on this point. in practice, republicans are perfectly happy with public schools when the schools are are "good enough" for their kids. so this caricature seems incorrect. there's a lot wrong with USA public schools, so something must be done. some very not-for-profit voucher schools have been doing excellent work where the public school systens have utterly failed, for example green dot animo in inglewood, or, dramatically, green dot locke in watts. what is not a caricature: the vast majority of policymakers who want to keep status quo in the public school system send their kids to private schools.
reverendsteveii 1y ago
that's a downright ponderous conditional at the beginning of this comment
no_wizard 1y ago
The problem with standardized testing in education is the following question: standardized for who?[0] The ultimate "sin", if you will, is that they tied test outcomes to funding and in practice is used to hold teachers to an arbitrary standard, not to help understand and guide the learning for students. For example, what could be done with standardized tests is that schools could use them at regular intervals to understand where each student is at, and adjust their learning resources accordingly to help students that aren't at grade level reach appropriate grade level academically, like if a student tests poorly in Math but not English Comprehension, it would make sense to adjust their schedule to give them more time to learn what they're struggling with and assigning resources appropriately, be it an extra half hour of math learning time with a tutor. That would actually make them useful. Instead, they're used to bludgeon teachers and school districts, and really the student outcomes are at best secondary to the whole operation, and since so much critical funding comes from sources tied to these outcomes, both good and bad outcomes mind you, hence the reason for the deliberate bell curve. Thats the real issue. In the US, education has become incredibly politicized to the point that I firmly feel it stopped being about actually educating students to be productive members of society and its about entrenched interests using it as a tool to push a broader political battle at the cost of student outcomes [0]: Not to mention that the pace of learning and aptitudes is varied by individual, some students will excel in X but not Y. This is a related, but for the purpose of this discussion, separate issue. Not to mention how much environment plays a role (a good home vs bad home situation for example).
- potato3732842 1y ago
  >In the US, education has become incredibly politicized to the point that I firmly feel it stopped being about actually educating students to be productive members of society and its about entrenched interests using it as a tool to push a broader political battle at the cost of student outcomes I.e. exactly what those pesky naysayers decades ago said would happen eventually.
- anon291 1y ago
  This nuance becomes reasonable in high school / college. For elementary schoolers, it's actually a great binary. Can you do two-digit addition or not is a pretty straightforwards question? Same with... can you identify grammar errors in this English sentence? Binary questions. The assessments to guide student learning are not the standardize tests meant to measure school performance and do state-level planning
BobaFloutist 1y ago
No, that's simply not true. You can very easily overfit even good metrics.

Have we found an exception to Goodhart's Law? "When a measure becomes a target, it ceases to be a good measure, except when the measure is good."

DougN7 1y ago
… so if the test is good … Reading is such a basic skill it seems like a pretty reasonable thing to come up with a decent test.
- SketchySeaBeast 1y ago
  "When a measure becomes a target, it ceases to be a good measure, except when the measure is something basic, like teaching reading"? Has the last twenty years of teaching to the test proven that out?
  - anon291 1y ago
    'Teaching to the test' has not caused the drop in literacy. Rather, it's teaching reading using bizarre methods, like whole word recognition instead of phonics. This has nothing to do with 'teaching to the test', since phonics would teach more to the test than whole word anything. A phonetically competent adult would be able to make out almost any English word. A whole word one would not.
    - SketchySeaBeast 1y ago
      "When a measure becomes a target, it ceases to be a good measure, unless the teacher is poor"?
      - anon291 1y ago
        This 'truth' is not always true. For binary questions, teaching to measure makes sense
        deleted 1y ago
        [deleted]
        SketchySeaBeast 1y ago
        Unfortunately, "is someone literate" is not a binary question. There's a whole range of possible answers there.
  - DougN7 1y ago
    Is there a way to teach reading, which increases reading test scores, that is bad? I’m having a hard time imagining what that would look like. Just because someone comes up with a pithy saying that some people call a “law” doesn’t make it a universal truth.
throwawaymaths 1y ago
you trying to establish some sort of measure for Goodhart's law?
viraptor 1y ago
I don't think so. You can still teach to a good test badly. For example: reading a variety of books and letting people do more on their own would be would be better overall than reading just things on the level of the test, asking just the test style questions and spending time learning the answering style to get 100%. It's kind of like those articles about young kids passing IT certifications. Those tests are reasonably good, but those kids just memorised lots of material and never actually worked as, for example, a networking engineer. Or like in IELTS style language test you can easily score higher (even than a native English speaker) if you learn the format enough.

andrewflnr 1y ago

The test is never that good.

larrled 1y ago
In education, nothing is ever good enough.
- no_wizard 1y ago
  This is my take: the real issue with education in the US goes back to trying to square two ideals. One is the meritocracy[0]. I believe that No Child Left Behind Act was sold to the public as a way to promote meritocracy in education and having some reasonably unbiased[1] way of doing so. The foundation of which is a way for parents to hold institutions responsible much more easily[2][3] The other is social and political stratification. This isn't merit based. Good examples of this would be the history of slavery, Jim Crow laws, the formerly legal practice of segregation by race, the drawing school district lines that keep students of wealthy parents away from folks who come from less economically advantaged backgrounds and enforcing this with district based school eligibility and funding. These two don't exactly square very well. Which is why the top level idea sold to the public - meritocracy - is used to sell these policies, but in practice they are using it to pull levers to further political and social stratification. [0]: Americans generally have a positive view of the term. I personally believe that most people don't have a proper understanding of the term to begin with, nor conceive what issues it could have in practice if they were given it. [1]: Leaving out the other half of the narrative, which is how much macro and micro levels of a students environment will influence how well they learn. [2]: Which made it easier for Republicans to continue to go after public education. They've been pushing for the voucher system for decades as a backdoor way to dismantle the public school system, reasons for wanting to do so vary but it all converges on this point. [3]: Not to mention, it completely ignores micro and macro issues that students can have, be it poverty, domestic issues, systemic issues and a whole host of other things.
  - throwawaymaths 1y ago
    > Which made it easier for Republicans to continue to go after public education. They've been pushing for the voucher system for decades as a backdoor way to dismantle the public school system, reasons for wanting to do so vary but it all converges on this point. in practice, republicans are perfectly happy with public schools when the schools are are "good enough" for their kids. so this caricature seems incorrect. there's a lot wrong with USA public schools, so something must be done. some very not-for-profit voucher schools have been doing excellent work where the public school systens have utterly failed, for example green dot animo in inglewood, or, dramatically, green dot locke in watts. what is not a caricature: the vast majority of policymakers who want to keep status quo in the public school system send their kids to private schools.

reverendsteveii 1y ago

that's a downright ponderous conditional at the beginning of this comment

no_wizard 1y ago

The problem with standardized testing in education is the following question: standardized for who?[0] The ultimate "sin", if you will, is that they tied test outcomes to funding and in practice is used to hold teachers to an arbitrary standard, not to help understand and guide the learning for students. For example, what could be done with standardized tests is that schools could use them at regular intervals to understand where each student is at, and adjust their learning resources accordingly to help students that aren't at grade level reach appropriate grade level academically, like if a student tests poorly in Math but not English Comprehension, it would make sense to adjust their schedule to give them more time to learn what they're struggling with and assigning resources appropriately, be it an extra half hour of math learning time with a tutor. That would actually make them useful. Instead, they're used to bludgeon teachers and school districts, and really the student outcomes are at best secondary to the whole operation, and since so much critical funding comes from sources tied to these outcomes, both good and bad outcomes mind you, hence the reason for the deliberate bell curve. Thats the real issue. In the US, education has become incredibly politicized to the point that I firmly feel it stopped being about actually educating students to be productive members of society and its about entrenched interests using it as a tool to push a broader political battle at the cost of student outcomes [0]: Not to mention that the pace of learning and aptitudes is varied by individual, some students will excel in X but not Y. This is a related, but for the purpose of this discussion, separate issue. Not to mention how much environment plays a role (a good home vs bad home situation for example).

potato3732842 1y ago
>In the US, education has become incredibly politicized to the point that I firmly feel it stopped being about actually educating students to be productive members of society and its about entrenched interests using it as a tool to push a broader political battle at the cost of student outcomes I.e. exactly what those pesky naysayers decades ago said would happen eventually.
anon291 1y ago
This nuance becomes reasonable in high school / college. For elementary schoolers, it's actually a great binary. Can you do two-digit addition or not is a pretty straightforwards question? Same with... can you identify grammar errors in this English sentence? Binary questions. The assessments to guide student learning are not the standardize tests meant to measure school performance and do state-level planning

BobaFloutist 1y ago

No, that's simply not true. You can very easily overfit even good metrics.