13 ms·

"California isn’t alone in its stagnating scores and in fact compares favorably with the 12 other states, plus the U.S. Virgin Islands, that administered the sa

"California isn’t alone in its stagnating scores and in fact compares favorably with the 12 other states, plus the U.S. Virgin Islands, that administered the same tests last spring from the Smarter Balanced Assessment Consortium. Scores decreased in every state, and California’s decline was about the smallest, retired educational testing specialist Douglas McRae said." This is the most important sentence in the article. It points to a problem in the test itself - California's students didn't just get magically stupider. Standardized testing isn't a particularly good way to gauge schools' ability to prepare students for college or for the 'real world.' Unfortunately, our K-12 public education system is broken and the politics of fixing it are even more broken, so this is what we get.

bitxbitxbitcoin 9y ago
It's almost as if the standard of what is standard changes drastically as the world does.
- swivelmaster 9y ago
  Or the process by which they design and score the test isn't very good.
lr4444lr 9y ago
So what's your recommended alternative metric? I don't doubt there is a good case to improve certain standardized tests, but that doesn't mean there isn't a valuable correlation even in a flawed one, nor that it's better to keep moving the goal posts into more subjective territory to explain away bad outcomes.
- swivelmaster 9y ago
  They're mentioned in the article. Graduation rates, college acceptance, stuff like that. Some of the countries with the best education systems in the world don't have these kinds of standardized tests. Also, nobody's suggesting moving the goal posts to explain away bad outcomes. If every school suddenly loses three points on average from one year to another on what is supposed to be the same test, the only common variable there is the test itself. The test HAS to be different from year to year in order not to repeat questions, but it HAS to remain a comparable difficulty.
  - mulmen 9y ago
    Graduation rate strikes me as a dangerous metric because it could easily be maximized by lowering the bar. We want to graduate as many students as possible but we should not compromise quality to do so. It seems to me we need metrics on both. I do admit to not knowing what I am talking about so maybe there is no danger in using the graduation rate metric.
    - swivelmaster 9y ago
      Well the best metric is going to be something like annual pre-tax income a decade after graduation but that will never happen so we'll just need to figure out some kind of composite metric that avoids perverse incentives. edit: This is not an entirely serious suggestion. See later comment below.
      - mulmen 9y ago
        Should income really be the final metric? What about less tangible (measurable?) factors like happiness? If we program all of our students to maximize income do we really make the world a better place?
        swivelmaster 9y ago
        It's not a metric for the students though, it's for the school. If you average it among all students, you get a reasonable metric for the how the school is doing. Of course, this would likely fail if you applied it to arts schools... Anyway, it wasn't a completely serious suggestion. It's got some problems. Happiness is a good thing to measure, but you'd want to still do some long term tracking of other metrics. Incarceration rates, income, college graduation rates, higher degrees and certifications, etc. edit: I make the comment about arts schools having received a degree in film and taken a haphazard path into startups and programming that ultimately worked out pretty well. I absolutely encourage people to go to school for the arts if that's what you want to be doing.
        humanrebar 9y ago
- dragonwriter 9y ago
  The existing test is privately developed, opaque, and the responsible policy makers in the states using it, including California, can't get information on the validation from the maker. And, regardless of previous trends, which varied, every state using it saw a drop in 2017. This is all mentioned in the article. Either there was a common national effect that dropped real outcomes, all states using the test independently made simultaneously missteps that dropped real outcomes, or the 2017 test was changed in some way which produced the downward movement. If it weren't for the opacity, it would be easy to eliminate the last possibility.
- criddell 9y ago
  > So what's your recommended alternative metric? AFAIK, good test scores correlate well with college success, but don't say much about quality of life beyond that. There might not be a good metric. For a lot of people, one or two great teachers have more of an impact than anything that can be tested for.
ransom1538 9y ago
"This is the most important sentence in the article. It points to a problem in the test itself - California's students didn't just get magically stupider." Yes. Having gone through public education in California the tests were treated as a total joke. They had no weight to your grades and there was zero incentive to do well. Often you wouldn't even get the results. In elementary school it was common for a teacher to say "When you have finished with this test, you can go to lunch" -- well kids would just start filling in bubbles. These tests were also accompanied by a sheet to see if you have tried drugs or sex all anonymous. My entire 3rd grade class did heroin.
- swivelmaster 9y ago
  When was this? I graduated high school in 2002 and felt like the tests were taken fairly seriously. I went to school in Marin, though, so I don't know how that compares to anywhere else.
  - Pigo 9y ago
    I agree, it always seemed to me that most of us didn't want people to think we're stupid. It seems sad that you have to worry that a kid will just fill in whatever so they can go to lunch. That seems like a self-esteem or discipline problem, not a fault of test taking. For all the complaining, I'm curious what a better solution would be for finding out how much kids are learning.
  - Redoubts 9y ago
    I’m only a couple years younger than you, but I definitely experienced a significant “trying is for white people” attitude when I was in school in LA.
- labster 9y ago
  Easy solution: make the tests count for a significant chunk of the students' grades. Or at certain years, require a passing grade on the standardized tests to be promoted to the next grade.
  - 9y ago
WillPostForFood 9y ago
I think this is more important: The numbers are half a percentage-point different from 2016 — down in English and up in math. Scores didn't really do down, they stayed the same. Why would one expect them to go up? Kids didn't get magically smarter, schools didn't change. It's more suspicious that the schools got better every year under the previous test.
- Waterluvian 9y ago
  Optimizing test taking took some years, maybe?
  - WillPostForFood 9y ago
    Sure, that's possible, and may suggest the current test is better, not flawed.
rtpg 9y ago
I get the sentiment, but it's possible that schools have in fact changed a bit since last year. It's not a stretch to imagine the introduction of new math textbooks causing a change of a couple points, if the books are in fact better/worse.
trollopTheJope 9y ago
I think the implication is that Californian students (along with students in at least 12 other states) got stupider "unmagically".
alexasmyths 9y ago
"Standardized testing isn't a particularly good way to gauge schools' ability" Maybe not the best, but it's still a measure, however crude, and that it's going down is a legitimate cause for concern. It's not magic - basic reading and math comprehension, are probably fairly strongly correlated with all sorts of life outcomes, though as you point out maybe not as direct as we would like to think - it still matters. As messed up and politicized as the teaching industry may be - at the core level, again, it's not rocket science: decent teachers, students who are somewhat motivated and want to learn, a classroom with a blackboard and some books etc. etc..
kazinator 9y ago
-> magically get stupider Anyway, it's actually more than magic at work; kids in grades 3-8 and 11 in 2016 are not the same set of people as kids in grades 3-8 and 11 in 2017.
- kazinator 9y ago
  One thing I'm getting at here is that even if we discount things like demographic shifts, there is the factor that the kids are one year older and are getting the harder tests. The new grade 3 kids being tested are last year's grade 2; the 4's are last years 3rd graders and so on. If you don't teach the kids anything all year, push them onto the next grade and then test them at the next level, they will do worse. Based on this idea, a year-over-year performance should be able to detect a real, instantaneous problem.
  - swivelmaster 9y ago
    Are you suggesting that the entire state education system can suddenly get worse for everyone everywhere all at once over the course of a year? That seems... unlikely.

bitxbitxbitcoin 9y ago

It's almost as if the standard of what is standard changes drastically as the world does.

swivelmaster 9y ago
Or the process by which they design and score the test isn't very good.

lr4444lr 9y ago

So what's your recommended alternative metric? I don't doubt there is a good case to improve certain standardized tests, but that doesn't mean there isn't a valuable correlation even in a flawed one, nor that it's better to keep moving the goal posts into more subjective territory to explain away bad outcomes.

swivelmaster 9y ago
They're mentioned in the article. Graduation rates, college acceptance, stuff like that. Some of the countries with the best education systems in the world don't have these kinds of standardized tests. Also, nobody's suggesting moving the goal posts to explain away bad outcomes. If every school suddenly loses three points on average from one year to another on what is supposed to be the same test, the only common variable there is the test itself. The test HAS to be different from year to year in order not to repeat questions, but it HAS to remain a comparable difficulty.
- mulmen 9y ago
  Graduation rate strikes me as a dangerous metric because it could easily be maximized by lowering the bar. We want to graduate as many students as possible but we should not compromise quality to do so. It seems to me we need metrics on both. I do admit to not knowing what I am talking about so maybe there is no danger in using the graduation rate metric.
  - swivelmaster 9y ago
    Well the best metric is going to be something like annual pre-tax income a decade after graduation but that will never happen so we'll just need to figure out some kind of composite metric that avoids perverse incentives. edit: This is not an entirely serious suggestion. See later comment below.
    - mulmen 9y ago
      Should income really be the final metric? What about less tangible (measurable?) factors like happiness? If we program all of our students to maximize income do we really make the world a better place?
      - swivelmaster 9y ago
        It's not a metric for the students though, it's for the school. If you average it among all students, you get a reasonable metric for the how the school is doing. Of course, this would likely fail if you applied it to arts schools... Anyway, it wasn't a completely serious suggestion. It's got some problems. Happiness is a good thing to measure, but you'd want to still do some long term tracking of other metrics. Incarceration rates, income, college graduation rates, higher degrees and certifications, etc. edit: I make the comment about arts schools having received a degree in film and taken a haphazard path into startups and programming that ultimately worked out pretty well. I absolutely encourage people to go to school for the arts if that's what you want to be doing.
      - humanrebar 9y ago
dragonwriter 9y ago
The existing test is privately developed, opaque, and the responsible policy makers in the states using it, including California, can't get information on the validation from the maker. And, regardless of previous trends, which varied, every state using it saw a drop in 2017. This is all mentioned in the article. Either there was a common national effect that dropped real outcomes, all states using the test independently made simultaneously missteps that dropped real outcomes, or the 2017 test was changed in some way which produced the downward movement. If it weren't for the opacity, it would be easy to eliminate the last possibility.
criddell 9y ago
> So what's your recommended alternative metric? AFAIK, good test scores correlate well with college success, but don't say much about quality of life beyond that. There might not be a good metric. For a lot of people, one or two great teachers have more of an impact than anything that can be tested for.

ransom1538 9y ago

"This is the most important sentence in the article. It points to a problem in the test itself - California's students didn't just get magically stupider." Yes. Having gone through public education in California the tests were treated as a total joke. They had no weight to your grades and there was zero incentive to do well. Often you wouldn't even get the results. In elementary school it was common for a teacher to say "When you have finished with this test, you can go to lunch" -- well kids would just start filling in bubbles. These tests were also accompanied by a sheet to see if you have tried drugs or sex all anonymous. My entire 3rd grade class did heroin.

swivelmaster 9y ago
When was this? I graduated high school in 2002 and felt like the tests were taken fairly seriously. I went to school in Marin, though, so I don't know how that compares to anywhere else.
- Pigo 9y ago
  I agree, it always seemed to me that most of us didn't want people to think we're stupid. It seems sad that you have to worry that a kid will just fill in whatever so they can go to lunch. That seems like a self-esteem or discipline problem, not a fault of test taking. For all the complaining, I'm curious what a better solution would be for finding out how much kids are learning.
- Redoubts 9y ago
  I’m only a couple years younger than you, but I definitely experienced a significant “trying is for white people” attitude when I was in school in LA.
labster 9y ago
Easy solution: make the tests count for a significant chunk of the students' grades. Or at certain years, require a passing grade on the standardized tests to be promoted to the next grade.
- 9y ago

WillPostForFood 9y ago

I think this is more important: The numbers are half a percentage-point different from 2016 — down in English and up in math. Scores didn't really do down, they stayed the same. Why would one expect them to go up? Kids didn't get magically smarter, schools didn't change. It's more suspicious that the schools got better every year under the previous test.

Waterluvian 9y ago
Optimizing test taking took some years, maybe?
- WillPostForFood 9y ago
  Sure, that's possible, and may suggest the current test is better, not flawed.

rtpg 9y ago

I get the sentiment, but it's possible that schools have in fact changed a bit since last year. It's not a stretch to imagine the introduction of new math textbooks causing a change of a couple points, if the books are in fact better/worse.

trollopTheJope 9y ago

I think the implication is that Californian students (along with students in at least 12 other states) got stupider "unmagically".

alexasmyths 9y ago

"Standardized testing isn't a particularly good way to gauge schools' ability" Maybe not the best, but it's still a measure, however crude, and that it's going down is a legitimate cause for concern. It's not magic - basic reading and math comprehension, are probably fairly strongly correlated with all sorts of life outcomes, though as you point out maybe not as direct as we would like to think - it still matters. As messed up and politicized as the teaching industry may be - at the core level, again, it's not rocket science: decent teachers, students who are somewhat motivated and want to learn, a classroom with a blackboard and some books etc. etc..

kazinator 9y ago

-> magically get stupider Anyway, it's actually more than magic at work; kids in grades 3-8 and 11 in 2016 are not the same set of people as kids in grades 3-8 and 11 in 2017.

kazinator 9y ago
One thing I'm getting at here is that even if we discount things like demographic shifts, there is the factor that the kids are one year older and are getting the harder tests. The new grade 3 kids being tested are last year's grade 2; the 4's are last years 3rd graders and so on. If you don't teach the kids anything all year, push them onto the next grade and then test them at the next level, they will do worse. Based on this idea, a year-over-year performance should be able to detect a real, instantaneous problem.
- swivelmaster 9y ago
  Are you suggesting that the entire state education system can suddenly get worse for everyone everywhere all at once over the course of a year? That seems... unlikely.