3 ms·

> where is this tabulated? How would one tabulate this or conduct such a study? Usually the main outcome measure you're looking at is something like the test.

by mlyle 3y ago

> where is this tabulated? How would one tabulate this or conduct such a study? Usually the main outcome measure you're looking at is something like the test. > if "teaching to the test" is the concern, how do I then interpret declining test scores? Exactly what you mentioned before and dismissed: short term gain is not necessarily long term gain. There's not enough high quality educational research. But one area we have strong research is on what kinds of learning are durable. The National Training Laboratory Institute has done a whole lot of research on what results in retention at 6 months, and the things you do for immediate test performance (rote practice, mnemonics, tricks to answer specific subject-specific questions, etc) are the least durable things. Test focused classrooms spend a whole lot of time on this stuff, and less time on things that work to create durable understanding (think-pair-share, low intensity practice built around spaced repetition, mastery-based methods, etc). Note that I am not a career teacher. I am a retired entrepreneur who now happens to teach. I have extraordinary programs that get extraordinary results-- despite being a small school, we're routinely having multiple students reaching state level championships in things I teach. Under test-focused pressure I could not achieve these things. Similarly, if I was spending a huge portion of my energy on classroom management and behavioral norms, I couldn't do them either. (Of course, on most test measures my students do pretty well, too). Indeed, I've written here: https://news.ycombinator.com/item?id=36423142 https://news.ycombinator.com/item?id=36423142 (and in the grandchild comment) about my experience teaching a very heavily test-focused course. I know I could have given my students a better and more durable knowledge of economics with less of the test-focused pressure.

taeric 3y ago
My assertion, then, would be to point that "teaching to the tests" concerns started and continued through a time that tests scores have decreased. If they really are teaching to the tests, they are doing a bad job at that, as well. :D I didn't dismiss the idea that short term gain can be a long term loss. I question if that is a panacea of a claim. Specifically, all too often, I see it pushed by fellow programmers when they are pushing for a harder solution and have stalled out on it. I have been said programmer doing that, before. It is best exemplified in the nonsense talk people have about how "worse is better." Antifragility appeals to me. Total ordering of ideas, required for "worse is better," do not. Worse, I've seen students that did a lot of these supposed better methods fall flat on their faces when in a new environment. And then it falls back to an obnoxious "true scotsman" debate about how they must not have been in these superior settings, as otherwise they would have succeeded. To that end, I would challenge that you could have really given them more durable knowledge of anything, right off. If only because I have yet to see anyone succeed at giving durable knowledge of anything that wasn't flat facts. I'd love to be shown how that is wrong, so please don't let my skepticism keep you from doing what you do. I'll go further and say that it is very likely that my skepticism is unhealthy at some levels. And again, I am interested in seeing more exploration of all of these ideas. As a parent, I'd love for my kids to get good at all they want to do. Rote practice and simple following of recipes and such is far better at that than it often gets stated. (And a big "f you" to "sight words." I can't believe how misguided that is and how it set back one of my kids in reading.)
- mlyle 3y ago
  > would be to point that "teaching to the tests" concerns started and continued through a time that tests scores have decreased This is false. The concerns of teaching to the tests, from my memory, peaked around 2005. No Child Left Behind was a major driver of these concerns, passed in 2001. Test scores peaked in 2012. This is exactly what you would expect if "teaching to the test" yielded a short term benefit but longer term harms, and was increasing during this interval. > Worse, I've seen students that did a lot of these supposed better methods fall flat on their faces when in a new environment. NTL did pretty simple experiments: teach something various ways, and see what ones resulted in best performance 6mos and 12mos later. Having students themselves pair off, teach, and explain ideas is the most durable thing we know. It requires active engagement with the material in order to reformulate it to help someone else through understanding. It requires being able to do this while multitasking and attending to social cues. And it can be a powerful motivator for students to fully understand before undertaking explanation. > (And a big "f you" to "sight words." I can't figure out what you're saying. "sight words" are a classic example of a hyper-focused rote memorization practice. I'm not saying rote is worthless. From my perspective, a perfect math class equally emphasizes intuition, rote practice, and rigorous explanation. What we tend to get, instead, is that once a student falls at all behind, they get 80+% rote practice. Worse, this practice is often subtly faulty and on topics that the student doesn't have the higher level explanations necessary to monitor their own performance.
  - taeric 3y ago
    Teaching to the tests never stopped being a concern from the circles I can remember. No Child Left Behind is an interesting fence post on this, though. "Sold a Story" was rather effective at showing the major opposition to all things Bush in that legislation and how it backfired heavily for reading. As so many opponents of that act railed against how it was pushing for reading to be tested, and how we are still struggling from that. That said, I do not have hard data here. Happy to be shown I was wrong that that peaked about that time. For teaching/learning, I confess I'm more partial to https://www.scientificamerican.com/article/getting-it-wrong/ https://www.scientificamerican.com/article/getting-it-wrong/. I'm curious to know more on studies like this, so I'll be looking up some of the NTL studies you are referencing. Would be delighted if you have a good reference on them. My "f you" to sight words is just hearkening back to the "Sold a Story" thing. So many of the "best practices" for teaching reading from teachers centered around "sight words" and such. Which, I suppose you can frame that to teaching to a test. But I think it is more teaching the wrong things. And flat out not teaching any technique to reading. Which, that last is fully to your point. I don't want techniques ignored or passed over. So if there is large scale evidence that that is happening, I'm very interested in the evidence.

by mlyle 3y ago

taeric 3y ago

My assertion, then, would be to point that "teaching to the tests" concerns started and continued through a time that tests scores have decreased. If they really are teaching to the tests, they are doing a bad job at that, as well. :D I didn't dismiss the idea that short term gain can be a long term loss. I question if that is a panacea of a claim. Specifically, all too often, I see it pushed by fellow programmers when they are pushing for a harder solution and have stalled out on it. I have been said programmer doing that, before. It is best exemplified in the nonsense talk people have about how "worse is better." Antifragility appeals to me. Total ordering of ideas, required for "worse is better," do not. Worse, I've seen students that did a lot of these supposed better methods fall flat on their faces when in a new environment. And then it falls back to an obnoxious "true scotsman" debate about how they must not have been in these superior settings, as otherwise they would have succeeded. To that end, I would challenge that you could have really given them more durable knowledge of anything, right off. If only because I have yet to see anyone succeed at giving durable knowledge of anything that wasn't flat facts. I'd love to be shown how that is wrong, so please don't let my skepticism keep you from doing what you do. I'll go further and say that it is very likely that my skepticism is unhealthy at some levels. And again, I am interested in seeing more exploration of all of these ideas. As a parent, I'd love for my kids to get good at all they want to do. Rote practice and simple following of recipes and such is far better at that than it often gets stated. (And a big "f you" to "sight words." I can't believe how misguided that is and how it set back one of my kids in reading.)

mlyle 3y ago
> would be to point that "teaching to the tests" concerns started and continued through a time that tests scores have decreased This is false. The concerns of teaching to the tests, from my memory, peaked around 2005. No Child Left Behind was a major driver of these concerns, passed in 2001. Test scores peaked in 2012. This is exactly what you would expect if "teaching to the test" yielded a short term benefit but longer term harms, and was increasing during this interval. > Worse, I've seen students that did a lot of these supposed better methods fall flat on their faces when in a new environment. NTL did pretty simple experiments: teach something various ways, and see what ones resulted in best performance 6mos and 12mos later. Having students themselves pair off, teach, and explain ideas is the most durable thing we know. It requires active engagement with the material in order to reformulate it to help someone else through understanding. It requires being able to do this while multitasking and attending to social cues. And it can be a powerful motivator for students to fully understand before undertaking explanation. > (And a big "f you" to "sight words." I can't figure out what you're saying. "sight words" are a classic example of a hyper-focused rote memorization practice. I'm not saying rote is worthless. From my perspective, a perfect math class equally emphasizes intuition, rote practice, and rigorous explanation. What we tend to get, instead, is that once a student falls at all behind, they get 80+% rote practice. Worse, this practice is often subtly faulty and on topics that the student doesn't have the higher level explanations necessary to monitor their own performance.
- taeric 3y ago
  Teaching to the tests never stopped being a concern from the circles I can remember. No Child Left Behind is an interesting fence post on this, though. "Sold a Story" was rather effective at showing the major opposition to all things Bush in that legislation and how it backfired heavily for reading. As so many opponents of that act railed against how it was pushing for reading to be tested, and how we are still struggling from that. That said, I do not have hard data here. Happy to be shown I was wrong that that peaked about that time. For teaching/learning, I confess I'm more partial to https://www.scientificamerican.com/article/getting-it-wrong/ https://www.scientificamerican.com/article/getting-it-wrong/. I'm curious to know more on studies like this, so I'll be looking up some of the NTL studies you are referencing. Would be delighted if you have a good reference on them. My "f you" to sight words is just hearkening back to the "Sold a Story" thing. So many of the "best practices" for teaching reading from teachers centered around "sight words" and such. Which, I suppose you can frame that to teaching to a test. But I think it is more teaching the wrong things. And flat out not teaching any technique to reading. Which, that last is fully to your point. I don't want techniques ignored or passed over. So if there is large scale evidence that that is happening, I'm very interested in the evidence.