4 ms·

For an example of how it can go wrong: I know for a fact that the competitive math class I teach makes a couple of standardized test measures go down-- especia

by mlyle 3y ago

For an example of how it can go wrong: I know for a fact that the competitive math class I teach makes a couple of standardized test measures go down-- especially "procedural knowledge". I know this because I've compared changes in performance to similar control students not taking the courses. I also know that it has a big positive impact on performance overall. And our students do pretty well in comp and a lot reach state championships. If I were strictly evaluated on standardized test outcomes, I would be asked to figure out how to pull those procedural knowledge scores up.

taeric 3y ago
Right, and I am not really arguing against you on this. For an example of places where not teaching to the test can go wrong, though, I need only look at some extended family that were only taught evolution, as it would be on some tests. Anecdotes being what they are, I'm interested in knowing if there is a large study on this. I'm also very open to thoughts on how to fix things. Knowing that it typically comes around to better tests... Edit: To be clear, my evolution example was supposed to be that they were only taught it because it would be on the tests. Not that they were only taught evolution.
- mlyle 3y ago
  > if there is a large study on this I'm not sure what "this" is, though. The question isn't well-posed, so I'm not sure what you would study. Absolutely there are many public school teachers that report that they are spending a lot of time teaching "to the test" rather than what they consider important. And this is a case of misaligned incentives. Overall mastery is important for the student's outcome going forward; the outcome of this, say, 5th grade standardized test doesn't affect their educational trajectory at all. But the outcome of that test is very important to the school's perception and the teacher's career. We'd all love better tests that really measure what we're trying to inculcate in students. But we also want tests that are easy to grade and very quantitative, which is in opposition to the other goal.
  - taeric 3y ago
    "This" would be how many places are harmed by "teaching to the tests." For example, you are asserting that "there are many public school teachers that report that they are spending a lot of time.." but... where is this tabulated? Are there biases mitigated in the self reporting side of what you are talking about? I don't want to be dismissive of the idea that it is harmful. I have seen very little compelling evidence that it is as harmful as it is asserted. Both as a student and as a parent. Is akin to employees that are upset about long term compromises for short term gain. Yes, it happens. All too often, short term survival is the best thing for long term survival. (With the ack that it is until it isn't.) Which is why I would love to see data and studies on this. Indeed, if "teaching to the test" is the concern, how do I then interpret declining test scores? Failing to teach either to the test or the subject?
    - mlyle 3y ago
      > where is this tabulated? How would one tabulate this or conduct such a study? Usually the main outcome measure you're looking at is something like the test. > if "teaching to the test" is the concern, how do I then interpret declining test scores? Exactly what you mentioned before and dismissed: short term gain is not necessarily long term gain. There's not enough high quality educational research. But one area we have strong research is on what kinds of learning are durable. The National Training Laboratory Institute has done a whole lot of research on what results in retention at 6 months, and the things you do for immediate test performance (rote practice, mnemonics, tricks to answer specific subject-specific questions, etc) are the least durable things. Test focused classrooms spend a whole lot of time on this stuff, and less time on things that work to create durable understanding (think-pair-share, low intensity practice built around spaced repetition, mastery-based methods, etc). Note that I am not a career teacher. I am a retired entrepreneur who now happens to teach. I have extraordinary programs that get extraordinary results-- despite being a small school, we're routinely having multiple students reaching state level championships in things I teach. Under test-focused pressure I could not achieve these things. Similarly, if I was spending a huge portion of my energy on classroom management and behavioral norms, I couldn't do them either. (Of course, on most test measures my students do pretty well, too). Indeed, I've written here: https://news.ycombinator.com/item?id=36423142 https://news.ycombinator.com/item?id=36423142 (and in the grandchild comment) about my experience teaching a very heavily test-focused course. I know I could have given my students a better and more durable knowledge of economics with less of the test-focused pressure.

by mlyle 3y ago

taeric 3y ago

Right, and I am not really arguing against you on this. For an example of places where not teaching to the test can go wrong, though, I need only look at some extended family that were only taught evolution, as it would be on some tests. Anecdotes being what they are, I'm interested in knowing if there is a large study on this. I'm also very open to thoughts on how to fix things. Knowing that it typically comes around to better tests... Edit: To be clear, my evolution example was supposed to be that they were only taught it because it would be on the tests. Not that they were only taught evolution.

mlyle 3y ago
> if there is a large study on this I'm not sure what "this" is, though. The question isn't well-posed, so I'm not sure what you would study. Absolutely there are many public school teachers that report that they are spending a lot of time teaching "to the test" rather than what they consider important. And this is a case of misaligned incentives. Overall mastery is important for the student's outcome going forward; the outcome of this, say, 5th grade standardized test doesn't affect their educational trajectory at all. But the outcome of that test is very important to the school's perception and the teacher's career. We'd all love better tests that really measure what we're trying to inculcate in students. But we also want tests that are easy to grade and very quantitative, which is in opposition to the other goal.
- taeric 3y ago
  "This" would be how many places are harmed by "teaching to the tests." For example, you are asserting that "there are many public school teachers that report that they are spending a lot of time.." but... where is this tabulated? Are there biases mitigated in the self reporting side of what you are talking about? I don't want to be dismissive of the idea that it is harmful. I have seen very little compelling evidence that it is as harmful as it is asserted. Both as a student and as a parent. Is akin to employees that are upset about long term compromises for short term gain. Yes, it happens. All too often, short term survival is the best thing for long term survival. (With the ack that it is until it isn't.) Which is why I would love to see data and studies on this. Indeed, if "teaching to the test" is the concern, how do I then interpret declining test scores? Failing to teach either to the test or the subject?
  - mlyle 3y ago
    > where is this tabulated? How would one tabulate this or conduct such a study? Usually the main outcome measure you're looking at is something like the test. > if "teaching to the test" is the concern, how do I then interpret declining test scores? Exactly what you mentioned before and dismissed: short term gain is not necessarily long term gain. There's not enough high quality educational research. But one area we have strong research is on what kinds of learning are durable. The National Training Laboratory Institute has done a whole lot of research on what results in retention at 6 months, and the things you do for immediate test performance (rote practice, mnemonics, tricks to answer specific subject-specific questions, etc) are the least durable things. Test focused classrooms spend a whole lot of time on this stuff, and less time on things that work to create durable understanding (think-pair-share, low intensity practice built around spaced repetition, mastery-based methods, etc). Note that I am not a career teacher. I am a retired entrepreneur who now happens to teach. I have extraordinary programs that get extraordinary results-- despite being a small school, we're routinely having multiple students reaching state level championships in things I teach. Under test-focused pressure I could not achieve these things. Similarly, if I was spending a huge portion of my energy on classroom management and behavioral norms, I couldn't do them either. (Of course, on most test measures my students do pretty well, too). Indeed, I've written here: https://news.ycombinator.com/item?id=36423142 https://news.ycombinator.com/item?id=36423142 (and in the grandchild comment) about my experience teaching a very heavily test-focused course. I know I could have given my students a better and more durable knowledge of economics with less of the test-focused pressure.