7 ms·

I recall reading/seeing that a lot of standardized testing in the US is gamed by schools, which focus on “teaching the test”. Could the decline in test scores

by ern 3y ago

I recall reading/seeing that a lot of standardized testing in the US is gamed by schools, which focus on “teaching the test”. Could the decline in test scores be related to a shift im how schools approach these tests?

taeric 3y ago
I also seem to recall that most concerns over "teaching to the test" were themselves misguided. Is like being worried that you are training to the sport, for kids that are getting into sports. Not that there isn't systemic cheating that can be done by some places. There can also be neglect of topics. That said, for the most part, if you can do the test, you have learned more of the topic than if you can't.
- jewayne 3y ago
  This. If you're not "teaching to the test", what are you teaching, exactly?
  - karaterobot 3y ago
    It'd be a less flexible approach, with more lessons oriented around test problems than the underlying concepts they're meant to represent? So, an equivalent would be learning leetcode programming puzzles to pass an interview, vs. learning to be a well-rounded programmer.
  - detaro 3y ago
    Related skills that are useful outside the limited form the standardized test takes. To take a practical example, if you wanted to get a group of kids to get best high marks in my high-school CS final exams, it'd been very effective to have them drill a a few standard sorting algorithms and a pile of formal definitions, so they are certain to be able to perfectly reproduce them from memory when prompted. But I'm fairly sure the few extra weeks of messing around with graph algorithms, despite that not being material relevant for the exams, probably were more useful for everyone in the class despite someone typoing a heapsort or not remembering how UTF-8 bit encoding works under pressure and loosing some points on that.
  - aylmao 3y ago
    There's a clue later on in the article, when it describes reading for fun almost every day is also lower than previous years. The goal of school is not for students to learn what The Pearl by Hemingway is about. The goal is for students to learn "to read"— ie, learn what to skim, what to pay attention to, what's being said between lines, how literary devices are used, how to piece different layers of ideas from a stream of words, etc. And importantly, to enjoy it at least to some degree, since it's a healthy habit Teaching the test in this context would be similar to teaching students to write some "top 20" algorithms, and recite their time complexities, knowing at least some of them will be on the test. It's definitely easier than teaching students the parts that make up a programming language, what big-O notation actually means, and the sort of generalized knowledge that will not only allow them to write and analyze _any_ program, but to see the beauty in programming and maybe even inspire a few to program for fun. This view of school as some sort of "training camp" for a test, is a terrible manifestation of focusing on the metric to the detriment of focusing on the goal.
    - brycewray 3y ago
      Hemingway?
      - jewayne 3y ago
        Was it over when the Germans bombed Pearl Harbor?
        brycewray 3y ago
        Germans? Forget it. He's rolling.
    - jewayne 3y ago
      Answers like this worry me, because it implies you believe that general principles can be taught without teaching basic facts or learning specific examples. Correct me if I'm wrong, but I think it's generally agreed upon that it's easiest to learn a specific example first, then use that as a jumping off point to take on the more daunting task of learning general principles. Programming languages are a great example of this -- in an introductory programming class, you're going to learn to create programs in a specific language, using all the constructs necessary to do so. And after you have some experience and mastery with that language, you have something to compare and contrast other languages against. And it's that mastery that actually inspires (more than) a few to program for fun.
  - gms7777 3y ago
    The problem is that there are plenty of valuable skills that are difficult to test in a standardized way. Most standardized test questions focus on Bloom levels 1-3 (remember, understand, apply) and struggle to test higher order skills (analyze, evaluate, create). Those questions are harder to write, usually take much longer for students to answer, and it's hard to format them such that they can be graded in a standardized manner (e.g., how do you test the skills ability to "create", to generate new ideas, to plan, to design, using a multiple choice question?) Placing a heavy emphasis on teaching to standardized tests means that teachers spend less time fostering higher order thinking skills.
    - no_wizard 3y ago
      >how do you test the skills ability to "create", to generate new ideas, to plan, to design, using a multiple choice question? The real answer to this is "don't use multiple choice questions", but that would require significant investment in actually properly grading the outputs. One of course could say that then you're going to teach to criteria (IE, what are the evaluators looking for?) but that may or may not be a bad thing, depending on how its handled.
    - scarmig 3y ago
      But you need to master the lower order skills before you can master the higher order skills. If someone is unable to evaluate a multi-digit sum, they'll be unable to devise creative solutions that use multi-digit sums.
      - gms7777 3y ago
        Of course, I'm not saying that the lower order skills are not important -- as you say, they're foundational. And I'm not even saying that it's a bad idea to test lower order skills. But they're not the only important thing, and "teaching to the test" tends to treat them like they are and prioritize them over critical thinking, synthesis, and creative skills.
  - xeromal 3y ago
    I have a buddy who managed to pass his engineer PE without studying and is generally very good at all math and logic based problems without really having to "learn the test". What he excels at though is breaking down a problem and deriving the solution rather than pattern matching. It isn't a very good skill for a timed test, but it is a very good skill in real life in engineering and pretty much everything else. Teaching the test takes away the learning of this valuable skill to have.
    - taeric 3y ago
      If they passed the PE Exam, they are good at testing. They may also be good at other things, but that is the measuring problem, in a nutshell. How do you build a measuring mechanism that measures what you care about, and not only what it is good at measuring? The best example of this is a car's speedometer. It doesn't actually measure how fast the car is going. Mainly because that is a surprisingly hard thing to do. Instead, it measures something that is easy to measure and converts to what we care about. Specifically, how fast are the tires rotating? Which is all to say, hard problem space. I have no doubt there are degenerate cases out there. My assertion is that is largely in the noise, at the moment. Happy (well, mayhap that isn't the correct word, but interested?) to be shown data that shows otherwise.
- jimbob45 3y ago
  I think the original criticism surrounding "teaching to the test" was that students were being taught useless shortcuts to answers rather than actually learning the material. I've never personally felt that the teachers I had ever did it though. One contrived example might be, "If you see "states' rights" anywhere in the question, then the answer is most likely going to be Daniel Webster, John Calhoun, or Henry Clay."
  - taeric 3y ago
    Yeah, but again, this isn't as damning as it sounds. Teaching kids to play sports, you definitely start with the "stand out there and watch the ball in motion." Ideally, you want them to engage with the sport, but just getting them there with the gear is a big part of it.
    - mlyle 3y ago
      I think it's pretty damning, actually. Spending most of your time circumventing the test and rote repetition undoes any learning of the real subject. I taught AP Microeconomics this past year. About a third of my time was spent teaching about different ways that the AP grading standards are picky and ways to exploit the test, because this is what is most important for the outcome they're worried about-- scoring a 5 on the exam. In turn, it didn't leave a lot of time for deeper discussions, simulations, etc, engaging with microeconomics.
      - alephnerd 3y ago
        How my AP Econ (Macro+Micro) teacher tackled that was spend the majority of the the time teaching the Samuelson and Mankiw books and then spend 1 month having everyone practice College Board style tests. Tbf, in our HS AP Econ was gatekept for 12th graders, and most of us had at least 6-7 AP tests done already so we had experience with the exams already.
        mlyle 3y ago
        I had mostly seniors with lots of AP experience too. AP Macro+Micro can work a lot better, because there's a few weeks of overlapping material. So it's the difference between fitting 45 hours of instructional material into 50 hours vs. fitting 80-85 into 100. The tight fit annoyed me, as did the strictly graded nature of the course. Two areas of particular annoyance: - I had a cohort of students who had entirely been through AP Calc, but fitting into the explanations expected for HS meant making somewhat bogus algebra based explanations (the slope of this line is double the other one, just because!!). A student who gave a calc-based explanation on the test would probably not get the points (frankly, a lot of the readers would not understand it). - About a third of the students had founded a "Game Theory" club on their own and were off in the weeds exploring game theory. I'd have loved to have taken a couple extra classes on the topic since they were already so interested and engaged, but I had no margin. I was forced to give the very simple 2 person payoff matrix form and leave the topic there. Again, I spent more time encouraging my students to not think too fancy.
sw104 3y ago
I think teaching the test is common everywhere. I remember the same thing for my GCSEs and A levels (UK pre-university qualifications) growing up, and my Russian partner says it was the same for them in school. For us, the teachers would almost tell us with 100% certainty what was going to be on the test, and what keywords/phrases to include to boost your marks. They also ran us through past papers from previous years, and lo and behold, we had the exact same questions, just worded slightly differently.

by ern 3y ago

taeric 3y ago

I also seem to recall that most concerns over "teaching to the test" were themselves misguided. Is like being worried that you are training to the sport, for kids that are getting into sports. Not that there isn't systemic cheating that can be done by some places. There can also be neglect of topics. That said, for the most part, if you can do the test, you have learned more of the topic than if you can't.

jewayne 3y ago
This. If you're not "teaching to the test", what are you teaching, exactly?
- karaterobot 3y ago
  It'd be a less flexible approach, with more lessons oriented around test problems than the underlying concepts they're meant to represent? So, an equivalent would be learning leetcode programming puzzles to pass an interview, vs. learning to be a well-rounded programmer.
- detaro 3y ago
  Related skills that are useful outside the limited form the standardized test takes. To take a practical example, if you wanted to get a group of kids to get best high marks in my high-school CS final exams, it'd been very effective to have them drill a a few standard sorting algorithms and a pile of formal definitions, so they are certain to be able to perfectly reproduce them from memory when prompted. But I'm fairly sure the few extra weeks of messing around with graph algorithms, despite that not being material relevant for the exams, probably were more useful for everyone in the class despite someone typoing a heapsort or not remembering how UTF-8 bit encoding works under pressure and loosing some points on that.
- aylmao 3y ago
  There's a clue later on in the article, when it describes reading for fun almost every day is also lower than previous years. The goal of school is not for students to learn what The Pearl by Hemingway is about. The goal is for students to learn "to read"— ie, learn what to skim, what to pay attention to, what's being said between lines, how literary devices are used, how to piece different layers of ideas from a stream of words, etc. And importantly, to enjoy it at least to some degree, since it's a healthy habit Teaching the test in this context would be similar to teaching students to write some "top 20" algorithms, and recite their time complexities, knowing at least some of them will be on the test. It's definitely easier than teaching students the parts that make up a programming language, what big-O notation actually means, and the sort of generalized knowledge that will not only allow them to write and analyze _any_ program, but to see the beauty in programming and maybe even inspire a few to program for fun. This view of school as some sort of "training camp" for a test, is a terrible manifestation of focusing on the metric to the detriment of focusing on the goal.
  - brycewray 3y ago
    Hemingway?
    - jewayne 3y ago
      Was it over when the Germans bombed Pearl Harbor?
      - brycewray 3y ago
        Germans? Forget it. He's rolling.
  - jewayne 3y ago
    Answers like this worry me, because it implies you believe that general principles can be taught without teaching basic facts or learning specific examples. Correct me if I'm wrong, but I think it's generally agreed upon that it's easiest to learn a specific example first, then use that as a jumping off point to take on the more daunting task of learning general principles. Programming languages are a great example of this -- in an introductory programming class, you're going to learn to create programs in a specific language, using all the constructs necessary to do so. And after you have some experience and mastery with that language, you have something to compare and contrast other languages against. And it's that mastery that actually inspires (more than) a few to program for fun.
- gms7777 3y ago
  The problem is that there are plenty of valuable skills that are difficult to test in a standardized way. Most standardized test questions focus on Bloom levels 1-3 (remember, understand, apply) and struggle to test higher order skills (analyze, evaluate, create). Those questions are harder to write, usually take much longer for students to answer, and it's hard to format them such that they can be graded in a standardized manner (e.g., how do you test the skills ability to "create", to generate new ideas, to plan, to design, using a multiple choice question?) Placing a heavy emphasis on teaching to standardized tests means that teachers spend less time fostering higher order thinking skills.
  - no_wizard 3y ago
    >how do you test the skills ability to "create", to generate new ideas, to plan, to design, using a multiple choice question? The real answer to this is "don't use multiple choice questions", but that would require significant investment in actually properly grading the outputs. One of course could say that then you're going to teach to criteria (IE, what are the evaluators looking for?) but that may or may not be a bad thing, depending on how its handled.
  - scarmig 3y ago
    But you need to master the lower order skills before you can master the higher order skills. If someone is unable to evaluate a multi-digit sum, they'll be unable to devise creative solutions that use multi-digit sums.
    - gms7777 3y ago
      Of course, I'm not saying that the lower order skills are not important -- as you say, they're foundational. And I'm not even saying that it's a bad idea to test lower order skills. But they're not the only important thing, and "teaching to the test" tends to treat them like they are and prioritize them over critical thinking, synthesis, and creative skills.
- xeromal 3y ago
  I have a buddy who managed to pass his engineer PE without studying and is generally very good at all math and logic based problems without really having to "learn the test". What he excels at though is breaking down a problem and deriving the solution rather than pattern matching. It isn't a very good skill for a timed test, but it is a very good skill in real life in engineering and pretty much everything else. Teaching the test takes away the learning of this valuable skill to have.
  - taeric 3y ago
    If they passed the PE Exam, they are good at testing. They may also be good at other things, but that is the measuring problem, in a nutshell. How do you build a measuring mechanism that measures what you care about, and not only what it is good at measuring? The best example of this is a car's speedometer. It doesn't actually measure how fast the car is going. Mainly because that is a surprisingly hard thing to do. Instead, it measures something that is easy to measure and converts to what we care about. Specifically, how fast are the tires rotating? Which is all to say, hard problem space. I have no doubt there are degenerate cases out there. My assertion is that is largely in the noise, at the moment. Happy (well, mayhap that isn't the correct word, but interested?) to be shown data that shows otherwise.
jimbob45 3y ago
I think the original criticism surrounding "teaching to the test" was that students were being taught useless shortcuts to answers rather than actually learning the material. I've never personally felt that the teachers I had ever did it though. One contrived example might be, "If you see "states' rights" anywhere in the question, then the answer is most likely going to be Daniel Webster, John Calhoun, or Henry Clay."
- taeric 3y ago
  Yeah, but again, this isn't as damning as it sounds. Teaching kids to play sports, you definitely start with the "stand out there and watch the ball in motion." Ideally, you want them to engage with the sport, but just getting them there with the gear is a big part of it.
  - mlyle 3y ago
    I think it's pretty damning, actually. Spending most of your time circumventing the test and rote repetition undoes any learning of the real subject. I taught AP Microeconomics this past year. About a third of my time was spent teaching about different ways that the AP grading standards are picky and ways to exploit the test, because this is what is most important for the outcome they're worried about-- scoring a 5 on the exam. In turn, it didn't leave a lot of time for deeper discussions, simulations, etc, engaging with microeconomics.
    - alephnerd 3y ago
      How my AP Econ (Macro+Micro) teacher tackled that was spend the majority of the the time teaching the Samuelson and Mankiw books and then spend 1 month having everyone practice College Board style tests. Tbf, in our HS AP Econ was gatekept for 12th graders, and most of us had at least 6-7 AP tests done already so we had experience with the exams already.
      - mlyle 3y ago
        I had mostly seniors with lots of AP experience too. AP Macro+Micro can work a lot better, because there's a few weeks of overlapping material. So it's the difference between fitting 45 hours of instructional material into 50 hours vs. fitting 80-85 into 100. The tight fit annoyed me, as did the strictly graded nature of the course. Two areas of particular annoyance: - I had a cohort of students who had entirely been through AP Calc, but fitting into the explanations expected for HS meant making somewhat bogus algebra based explanations (the slope of this line is double the other one, just because!!). A student who gave a calc-based explanation on the test would probably not get the points (frankly, a lot of the readers would not understand it). - About a third of the students had founded a "Game Theory" club on their own and were off in the weeds exploring game theory. I'd have loved to have taken a couple extra classes on the topic since they were already so interested and engaged, but I had no margin. I was forced to give the very simple 2 person payoff matrix form and leave the topic there. Again, I spent more time encouraging my students to not think too fancy.

sw104 3y ago

I think teaching the test is common everywhere. I remember the same thing for my GCSEs and A levels (UK pre-university qualifications) growing up, and my Russian partner says it was the same for them in school. For us, the teachers would almost tell us with 100% certainty what was going to be on the test, and what keywords/phrases to include to boost your marks. They also ran us through past papers from previous years, and lo and behold, we had the exact same questions, just worded slightly differently.