2 ms·

First off: I think this is a really silly set of weeds we've gotten into. Trying to eyeball an effect of an intervention from a time series is "what not to do

by mlyle 3y ago

First off: I think this is a really silly set of weeds we've gotten into. Trying to eyeball an effect of an intervention from a time series is "what not to do in statistics 101." It's massive confounded: underlying student demographics have massively shifted But if we're going to do it: If you increase the amount of effort spent "teaching to the tests" over the span of 2000-2010, and spot a rise in scores over that time and then a fall in older students' performance, then it sure looks like you've prioritized a short term gain. > "if they are teaching to the tests, they are doing a bad job at that." Well, test scores are massively higher than 1978, the first year that we have real data for. At the same time, it's not too obvious to me that we're really doing massively better than 1978. In which case: what are we measuring? What are we spending all this precious instructional time and effort on?

taeric 3y ago
I'm smiling at that first. :D I really do think this is more "violent agreement" than it is otherwise. I don't see an increase in 2000-2010, though? I see a jump in 2000ish, but the rest looks like noise. Even 2000ish is probably debatable, but if there is clear evidence that we made short term gains at long term loss, I'm game. My kneecapping here is that they didn't even get the short term gains. If you blame both on "teaching to the test" I will make no real defense of that. My attack is simply that whatever we are doing isn't working. In any real way. That is, my main posit is that if they were at least making gains on the tests, that would be a gain we could see somewhere. But... we aren't seeing that. So much of the "teaching to the test" rhetoric is a moot topic without clearer evidence of short term gains. So, yeah, violent agreement with your last paragraph. I have no idea what we are doing. Be it teaching to the test or otherwise, it isn't working. Evidence seems mostly flat for the past few decades. (Covid notwithstanding, as I feel that has its own causal factors that are beyond what we are talking about.)
- mlyle 3y ago
  > I don't see an increase in 2000-2010, though? I see a jump in 2000ish, but the rest looks like noise There's a few points of improvement from '04 to '12. The funny thing here is basically all of the drop we've seen since '12-- and especially post '20-- has been in the attainment of the bottom 10th percentile. Everyone else has been pretty flat. > So much of the "teaching to the test" rhetoric is a moot topic without clearer evidence of short term gains. We have gains on the tests during the mid 90's to 2010 timeframe. Big gains in mathematics. But does anyone seriously believe that mathematical literacy is much better? Of course, it all depends what you decide is important. In the past 5 years that I've been following things, even very high performing students have fallen in arithmetical capabilities. (But maybe we just don't care if they're still doing higher math? I feel like I care)
  - taeric 3y ago
    I've seen the fall in arithmetical capabilities. But I've also seen that largely in myself. Oddly, I think credit cards and never having to make change for things is a big driver in that. To wit, you get worse at the things you don't have to regularly do. Interesting point on the differences in the percentiles. I'd hazard that speaks more to exposure? Could be bias in what I was able to see, but I know every richer person I knew was able to get their kids in a private school, often with a personal tutor, during the '20 chaos. I should say, thanks for sticking on this thread! Interesting conversation.

by mlyle 3y ago

taeric 3y ago

I'm smiling at that first. :D I really do think this is more "violent agreement" than it is otherwise. I don't see an increase in 2000-2010, though? I see a jump in 2000ish, but the rest looks like noise. Even 2000ish is probably debatable, but if there is clear evidence that we made short term gains at long term loss, I'm game. My kneecapping here is that they didn't even get the short term gains. If you blame both on "teaching to the test" I will make no real defense of that. My attack is simply that whatever we are doing isn't working. In any real way. That is, my main posit is that if they were at least making gains on the tests, that would be a gain we could see somewhere. But... we aren't seeing that. So much of the "teaching to the test" rhetoric is a moot topic without clearer evidence of short term gains. So, yeah, violent agreement with your last paragraph. I have no idea what we are doing. Be it teaching to the test or otherwise, it isn't working. Evidence seems mostly flat for the past few decades. (Covid notwithstanding, as I feel that has its own causal factors that are beyond what we are talking about.)

mlyle 3y ago
> I don't see an increase in 2000-2010, though? I see a jump in 2000ish, but the rest looks like noise There's a few points of improvement from '04 to '12. The funny thing here is basically all of the drop we've seen since '12-- and especially post '20-- has been in the attainment of the bottom 10th percentile. Everyone else has been pretty flat. > So much of the "teaching to the test" rhetoric is a moot topic without clearer evidence of short term gains. We have gains on the tests during the mid 90's to 2010 timeframe. Big gains in mathematics. But does anyone seriously believe that mathematical literacy is much better? Of course, it all depends what you decide is important. In the past 5 years that I've been following things, even very high performing students have fallen in arithmetical capabilities. (But maybe we just don't care if they're still doing higher math? I feel like I care)
- taeric 3y ago
  I've seen the fall in arithmetical capabilities. But I've also seen that largely in myself. Oddly, I think credit cards and never having to make change for things is a big driver in that. To wit, you get worse at the things you don't have to regularly do. Interesting point on the differences in the percentiles. I'd hazard that speaks more to exposure? Could be bias in what I was able to see, but I know every richer person I knew was able to get their kids in a private school, often with a personal tutor, during the '20 chaos. I should say, thanks for sticking on this thread! Interesting conversation.