3 ms·

> If the concern is that they are "teaching to the tests", my bigger concern is why are they so bad at that, even? Again, I think purely teaching to the tests

by mlyle 3y ago

> If the concern is that they are "teaching to the tests", my bigger concern is why are they so bad at that, even? Again, I think purely teaching to the tests is short term gains for long term losses. To take my example: I expect my students are going to be mostly 4's and 5's on the AP Microeconomics exam (this is good). But if I could have taught them why the functions looked the way they did, they'd have a better understanding of microeconomics. This would have been a dangerous thing to actually teach them, because if they used this knowledge on the test, there's no guarantee a pair of readers would understand it and give them the point. Better to dumb it down and teach the graph-based and algebra-based reasoning rather than the calculus-based reasoning of how it actually works, even though my students were capable of it. I got to teach students stuff that they'll have to unlearn for deeper study, but maximized their short term performance. I'm totally willing to do this because it is in the students' interest: they really want a "5" on that exam to prove their capability to colleges and to skip a course. But if the test results weren't important to the students (like most of these standardized tests) but were important to -me-, that would be a terrible conflict of interest. > I'm assuming the idea is that the kids are passing lower level tests better, but failing later tests? 4th grade scores kept climbing longer and have been flat for the past 7-8 years even as 7th grade scores have slowly slid downwards.

taeric 3y ago
Right, but where are the short term gains? Scores are down. The charts I saw on the NAEP site had 4th grade scores basically flat. Which chart do you see them going up in? Specifically, their charts do show we are up in math compared to 90, but it has been flat since about 2003. And 4th grade reading has been flat basically forever, which is laughably in contrast with the older kids. If your assertion is that they are having to teach to the test just to keep things flat, and that is why 4th graders are treading water and then starting to fall off by grade 8, I can buy that. I'm a little susp that the charts between 4th and 8th graders seem to not have a time lag on them. In that, whatever we are doing wrong seems to impact both grades at the same time. If that is the assertion, though, I stand by my "if they are teaching to the tests, they are doing a bad job at that."
- mlyle 3y ago
  First off: I think this is a really silly set of weeds we've gotten into. Trying to eyeball an effect of an intervention from a time series is "what not to do in statistics 101." It's massive confounded: underlying student demographics have massively shifted But if we're going to do it: If you increase the amount of effort spent "teaching to the tests" over the span of 2000-2010, and spot a rise in scores over that time and then a fall in older students' performance, then it sure looks like you've prioritized a short term gain. > "if they are teaching to the tests, they are doing a bad job at that." Well, test scores are massively higher than 1978, the first year that we have real data for. At the same time, it's not too obvious to me that we're really doing massively better than 1978. In which case: what are we measuring? What are we spending all this precious instructional time and effort on?
  - taeric 3y ago
    I'm smiling at that first. :D I really do think this is more "violent agreement" than it is otherwise. I don't see an increase in 2000-2010, though? I see a jump in 2000ish, but the rest looks like noise. Even 2000ish is probably debatable, but if there is clear evidence that we made short term gains at long term loss, I'm game. My kneecapping here is that they didn't even get the short term gains. If you blame both on "teaching to the test" I will make no real defense of that. My attack is simply that whatever we are doing isn't working. In any real way. That is, my main posit is that if they were at least making gains on the tests, that would be a gain we could see somewhere. But... we aren't seeing that. So much of the "teaching to the test" rhetoric is a moot topic without clearer evidence of short term gains. So, yeah, violent agreement with your last paragraph. I have no idea what we are doing. Be it teaching to the test or otherwise, it isn't working. Evidence seems mostly flat for the past few decades. (Covid notwithstanding, as I feel that has its own causal factors that are beyond what we are talking about.)
    - mlyle 3y ago
      > I don't see an increase in 2000-2010, though? I see a jump in 2000ish, but the rest looks like noise There's a few points of improvement from '04 to '12. The funny thing here is basically all of the drop we've seen since '12-- and especially post '20-- has been in the attainment of the bottom 10th percentile. Everyone else has been pretty flat. > So much of the "teaching to the test" rhetoric is a moot topic without clearer evidence of short term gains. We have gains on the tests during the mid 90's to 2010 timeframe. Big gains in mathematics. But does anyone seriously believe that mathematical literacy is much better? Of course, it all depends what you decide is important. In the past 5 years that I've been following things, even very high performing students have fallen in arithmetical capabilities. (But maybe we just don't care if they're still doing higher math? I feel like I care)

by mlyle 3y ago

taeric 3y ago

Right, but where are the short term gains? Scores are down. The charts I saw on the NAEP site had 4th grade scores basically flat. Which chart do you see them going up in? Specifically, their charts do show we are up in math compared to 90, but it has been flat since about 2003. And 4th grade reading has been flat basically forever, which is laughably in contrast with the older kids. If your assertion is that they are having to teach to the test just to keep things flat, and that is why 4th graders are treading water and then starting to fall off by grade 8, I can buy that. I'm a little susp that the charts between 4th and 8th graders seem to not have a time lag on them. In that, whatever we are doing wrong seems to impact both grades at the same time. If that is the assertion, though, I stand by my "if they are teaching to the tests, they are doing a bad job at that."

mlyle 3y ago
First off: I think this is a really silly set of weeds we've gotten into. Trying to eyeball an effect of an intervention from a time series is "what not to do in statistics 101." It's massive confounded: underlying student demographics have massively shifted But if we're going to do it: If you increase the amount of effort spent "teaching to the tests" over the span of 2000-2010, and spot a rise in scores over that time and then a fall in older students' performance, then it sure looks like you've prioritized a short term gain. > "if they are teaching to the tests, they are doing a bad job at that." Well, test scores are massively higher than 1978, the first year that we have real data for. At the same time, it's not too obvious to me that we're really doing massively better than 1978. In which case: what are we measuring? What are we spending all this precious instructional time and effort on?
- taeric 3y ago
  I'm smiling at that first. :D I really do think this is more "violent agreement" than it is otherwise. I don't see an increase in 2000-2010, though? I see a jump in 2000ish, but the rest looks like noise. Even 2000ish is probably debatable, but if there is clear evidence that we made short term gains at long term loss, I'm game. My kneecapping here is that they didn't even get the short term gains. If you blame both on "teaching to the test" I will make no real defense of that. My attack is simply that whatever we are doing isn't working. In any real way. That is, my main posit is that if they were at least making gains on the tests, that would be a gain we could see somewhere. But... we aren't seeing that. So much of the "teaching to the test" rhetoric is a moot topic without clearer evidence of short term gains. So, yeah, violent agreement with your last paragraph. I have no idea what we are doing. Be it teaching to the test or otherwise, it isn't working. Evidence seems mostly flat for the past few decades. (Covid notwithstanding, as I feel that has its own causal factors that are beyond what we are talking about.)
  - mlyle 3y ago
    > I don't see an increase in 2000-2010, though? I see a jump in 2000ish, but the rest looks like noise There's a few points of improvement from '04 to '12. The funny thing here is basically all of the drop we've seen since '12-- and especially post '20-- has been in the attainment of the bottom 10th percentile. Everyone else has been pretty flat. > So much of the "teaching to the test" rhetoric is a moot topic without clearer evidence of short term gains. We have gains on the tests during the mid 90's to 2010 timeframe. Big gains in mathematics. But does anyone seriously believe that mathematical literacy is much better? Of course, it all depends what you decide is important. In the past 5 years that I've been following things, even very high performing students have fallen in arithmetical capabilities. (But maybe we just don't care if they're still doing higher math? I feel like I care)