15 ms·

NCLB tried to fix those "ineffectual, corrupt money pits" by giving enormous, overriding importance to a small set of easily gamed statistics. Creating a singl

• frozenport 12y ago
  >>easily gamed statistics. Nope. If they were easily gamed Park would have survived? The failed at teaching to the test and had the cheat the old fashioned way.
• yummyfajitas 12y ago
  Please be concrete. How do you game standardized tests? Sample tests can be found here, so make sure your proposed method would actually work. https://www.georgiastandards.org/resources/Pages/Tools/TestingResources.aspx https://www.georgiastandards.org/resources/Pages/Tools/Testi...
  • samdk 12y ago
    Did you read the article? The entire thing is about a school that for years did things like have teachers change test answers after the fact in order to increase their overall test scores. That's what I mean by gaming statistics. edit: clearly I should have chosen a different word. I think you're all focusing on one specific word I used, and not on the actual argument I'm making, which is that if your model of assessment depends on a single Big Important Number, people will find ways to make that number what they need it to be.
    • twoodfin 12y ago
      That's not gaming; it's cheating. No objective evaluation system could be immune to that.
    • yummyfajitas 12y ago
      If "game" simply means "cheat", the solution is simple - third party test administration. I thought you meant they do something legal other than education to make the scores go up.
    • deleted 12y ago
      [deleted]
    • rmc 12y ago
      That's outright cheating and fraud. Any test can be cheated if you look at the answers beforehand. "Gaming" means figuring out flaws in the way it's tested and then doing lots of that. In your lines-of-code-added metric, "gaming" means adding and removing lots of code. "cheating" means hacking the database with results and changing the values.
  • JoachimSchipper 12y ago
    Most basically, a school that frequently teaches useful-but-not-tested material will, all else being equal, produce lower test scores than a school that sticks rigidly to the tested material. I'm not sure this is "gaming", proper; but if this focus on the tested material prevents a teacher from enthusiastically expounding on something (s)he thinks is especially fascinating, I'd count that as a loss. Because having enthusiastic teachers is great. (Of course, standardized testing has benefits, too - I'm not sufficiently informed on American education to have an overall opinion. I just wanted to point out that there are ways to test better that don't improve education.)
    • yummyfajitas 12y ago
      If a teacher enthusiastically expounds upon football, string theory or creationism, and fails to teach reading comprehension, they will indeed suffer on the tests. This doesn't seem like a bad thing. Deciding what needs to be taught is the job of the political system, not the teachers.
      • octo_t 12y ago
        If a maths teacher wants to teach category theory, or algorithmic analysis (things which can be both interesting and relevant to a someone growing up), they can't. Is that a better use case for you?
        • yummyfajitas 12y ago
          The state legislature and federal government have decided that arithmetic is more useful than category theory. I agree with them completely, and I'm one of those rare people who is extremely sympathetic to category theory. So yes, I think tests are doing the right thing here - making sure the teacher does his job before he goes off on random tangents.
      • zarify 12y ago
        Ignoring the obvious hyperbole of your first sentence, sure politics essentially sets the curriculum (for better or for worse). That doesn't excuse the political system also getting in the way of teaching it. Mercifully in Australia we don't have standardized testing every year (yet), but enough importance is tied to the tests that do take place that a disproportionate amount of time is spent preparing kids for the style of tests administered, regardless of any gaming of the system, reducing the time spent on the rest of the curriculum. I know we've argued education before and so I'm not expecting a hallelujah moment, but I figured it was a point that should also be made.
  • dalke 12y ago
    I'll be concrete and define "game standardized tests" as "aspects which influence the test results other than proficiency in the subject matter." On the teacher side, one way to game them is via test prep. Quoting from http://curmudgucation.blogspot.com/2014/04/what-test-prep-is-not.html http://curmudgucation.blogspot.com/2014/04/what-test-prep-is... : "learning how to perform the specific cockamamie tasks favored by the designers of the various state-level assessments" That includes: > We have covered "How To Spot the Fake Answers Put There To Fool You." We've discussed "Questions About Context Clues Mean You Must Ignore What You Think You Know." We've discussed how open-ended questions require counting skills (the answer to any question that includes "Give three reasons that..." just requires a full three reasons of anything at all, but give three). For lower-function students, we covered such basics as "Read All Four Answers Before You Pick One." > We have pushed aside old literary forms like "short stories" and "novels" in favor of "reading selections"-- one-page-sized chunks of boring contextless pablum which nobody reads in real life, but everybody reads on standardized tests. We have taught them to always use big words like "plethora" on their essay answers, and to always fill up the whole essay page, no matter what repetitive gibberish is requires. We have taught them to always rewrite the prompt as their topic sentence. In PA, we have taught them what sort of crazy possible meaning the test-writers might have assigned to the words "tone" and "mood." This is gaming the system because had the test givers used a completely different approach, say, of using open-ended questions instead of multiple-choice answers, or "give persuasive reasons that..." instead of the more easily tested "three", then this test prep would not work. Not that that alternate testing form will happen in standardized tests, because it's a lot more expensive to grade free-form tests. But teachers themselves can ask these sorts of questions, to help gauge proficiency. Another way to game the test is on the test giver's side - they get to define what the pass/fail thresholds are. For example, "a test question was considered "hard" or "easy" not because it required a particular skill; its difficulty was determined based on how many students got it correct." Quote from http://jerseyjazzman.blogspot.com/2013/08/scoring-ny-tests-with-triple-lindy.html http://jerseyjazzman.blogspot.com/2013/08/scoring-ny-tests-w... . Since the thresholds can be determined after the fact, this means it's not really judging proficiency but instead is making the test give a pre-defined pass/fail curve.
    • yummyfajitas 12y ago
  • cafard 12y ago
    You can cut the Gordian knot by simply having the teachers correct enough errors on enough tests--that has been done in the Washington, DC, area. Or you can get a sneak preview of the test and drill the kids on that for a few days.
  • rbehrends 12y ago
    By necessity, any test can realistically test only a subset of a student's knowledge. If the patterns that the tested subset follows are easily predictable or have a simpler structure than the actual knowledge, you can get better performance by teaching those patterns than by teaching the actual skills. Multiple choice tests are particularly egregious examples, as for them you only have to be able to verify an answer instead of having to derive the answer. Standardized tests are rife with multiple choice tests and questions that follow predictable patterns.
    • danielweber 12y ago
      Good test design is hard, especially for an amateur. But it seems the remedy ought to be "have better tests" instead of "don't test." There are good tests out there; the AP tests have a significant multiple-choice component that resists simple trickery.
      • rbehrends 12y ago
        > But it seems the remedy ought to be "have better tests" instead of "don't test." Nobody is saying "don't test". The only place that I know of that doesn't test in the traditional sense are Waldorf schools, and even they still evaluate their students. Concerns are being raised about standardized testing in a NCLB-influenced environment, both in how that limits test design, tests for the wrong things, and how this kind of testing creates the wrong incentives. > the AP tests have a significant multiple-choice component that resists simple trickery. It isn't just about trickery. I remember encountering a pumping lemma question during the GRE that was simply easier to answer in multiple choice form because I didn't have to prove the correctness of the right answer, but only the incorrectness of the wrong ones (and even there I didn't have to work out the fiddly details, I just needed a modest amount of confidence).
  • jfoutz 12y ago
    Arrange to get a few minutes alone with the tests, correct a few answers on your worst performing students tests, or just swap out their answer sheets with improved answers. Freakanomics talked about this in regard to Chicago public schools merit pay system.
• danielweber 12y ago
  Managing by numbers is bad. I once had a boss look over the commit count to the source code repo to measure effectiveness. However, software bosses have another avenue: they can just hire and fire based on how well they think people are performing. They don't need to set up spreadsheets or anything to "prove" who isn't doing their job. And the engineer who has been told to move on can move on, because there isn't just one employer in town, even outside of the top-5 cities. And the manager has a reason to care about properly evaluating his reports. This attempt to measure educational performance by numbers, for all that it is screwed up, has risen as a desperate attempt for the people paying the bills to feel like they have some kind of accountability into the system they are paying for.
• gwern 12y ago
  A corrupt incompetent cronyistic mess is going to be a mess whether or not you use a small set of gamed statistics or nothing at all. The reporter doesn't get it when they critically remark that the reform model "ignored less quantifiable signs of intellectual development" - no, what would happen if it didn't is just that the people who are willing to make up test scores en masse will also make up all those 'less quantifiable signs' except you won't have any way of knowing about their bullshit. The NCLB gaming of standardized tests, on the other hand, forces the incompetence out into the open in the form of unmistakable undefensible clear outright fraud. So, which is better? To have all your metrics gamed and to not know it, or have them gamed and a chance of detecting the fraud?
  • dragonwriter 12y ago
    > The NCLB gaming of standardized tests, on the other hand, forces the incompetence out into the open in the form of unmistakable undefensible clear outright fraud. The outright fraud isn't the only thing it forces (and, really, its sort of the extreme fringe); the more routine thing it forces is reluctance of schools to promote students based on academic ability since holding advanced students back in grade level improves metrics of students that are function at or above grade level. (Of course, its an unsustainable optimization, which also increases the rate at which advanced students with involved parents defect from the public school system entirely, but that's an effect that takes longer to materialize than the short-term effect on metrics.)