2 ms·

Whenever teachers realize they're being evaluated based on some metric, they almost immediately begin gaming that metric and with that it ceases to be a good me

Whenever teachers realize they're being evaluated based on some metric, they almost immediately begin gaming that metric and with that it ceases to be a good metric. I had teachers give a lot of "group tests", walk out of the classroom during tests, etc. Standardized testing aims to correct this, but I think it's now considered by many teachers to be a joke. Even when exams are administered by independent proctors, teachers can still game the metric by "teaching the test" instead of imparting meaningful understanding of the material. I think this is a subject that will continue to defy rigorous objective scientific analysis.

pingyong 7y ago
Tbh if you can "teach the test" without imparting meaningful understanding of the material, the test wasn't good the first place.
- catalogia 7y ago
  It turns out creating "good tests" isn't so easy. Generally tests suffer from one or more of the following: subjective grading, game-able question formats, or superficial coverage of the material. Most tests administered by school teachers have very subjective grading. Basically every essay question has subjective grading, as do "show your work" math test questions. Multiple choice questions remove much of that subjectivity, but at the expense of a system that can often be gamed and often covers the subject matter in a superficial way. If incentives are correctly aligned, tests with subjective grading can work reasonably well. But when tests taken by students are used to evaluate the performance of teachers, the incentives of the teachers align against you. For wide scale experimentation, you want a test that actually measures meaningful understanding of the subject matter and at the same time can be graded objectively and at the same time defies the attempts of students, teachers, and school administerators to subvert the results. That's basically the holy grail of tests.
  - pingyong 7y ago
    It's definitely not trivial, but I think a solution would be to have tests that are, to a certain extent, subjective (I don't think they end up being that subjective in Math, and they'll always be subjective in English), and then they get graded by teachers from different schools.

pingyong 7y ago

Tbh if you can "teach the test" without imparting meaningful understanding of the material, the test wasn't good the first place.

catalogia 7y ago
It turns out creating "good tests" isn't so easy. Generally tests suffer from one or more of the following: subjective grading, game-able question formats, or superficial coverage of the material. Most tests administered by school teachers have very subjective grading. Basically every essay question has subjective grading, as do "show your work" math test questions. Multiple choice questions remove much of that subjectivity, but at the expense of a system that can often be gamed and often covers the subject matter in a superficial way. If incentives are correctly aligned, tests with subjective grading can work reasonably well. But when tests taken by students are used to evaluate the performance of teachers, the incentives of the teachers align against you. For wide scale experimentation, you want a test that actually measures meaningful understanding of the subject matter and at the same time can be graded objectively and at the same time defies the attempts of students, teachers, and school administerators to subvert the results. That's basically the holy grail of tests.
- pingyong 7y ago
  It's definitely not trivial, but I think a solution would be to have tests that are, to a certain extent, subjective (I don't think they end up being that subjective in Math, and they'll always be subjective in English), and then they get graded by teachers from different schools.