8 ms·

"How to spot the fake answers put there to fool you" == "how to see when an answer isn't even in the ballpark". That's a useful skill. That "context clues" thin

"How to spot the fake answers put there to fool you" == "how to see when an answer isn't even in the ballpark". That's a useful skill. That "context clues" thing suggests teaching students how to solve the problem in front of them, not the easy problem their mind wants to substitute for it [1]. From what this guy describes, "test prep" sounds like "educating students". I agree that grading of essays is a disaster. It's not specific to standardized tests, however - that's how all my essays were graded from grade 1 all the way to college. Another way to game the test is on the test giver's side - they get to define what the pass/fail thresholds are. This is why tests are standardized, not left up to the schools or teachers. [1] A hard problem: "What is the optimal incarceration time to dissuade people from pedophilia." An easy problem: "How angry do pedophiles make me feel?" When most people hear the first question, which is hard, their mind substitutes the second much easier question for it. See Kahneman's book "Thinking Fast and Slow" for more on this. http://www.amazon.com/gp/product/0374533555/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0374533555&linkCode=as2&tag=christuc-20&linkId=KWB55KZLPEI4OQCE http://www.amazon.com/gp/product/0374533555/ref=as_li_tl?ie=...

rando289 12y ago
> From what this guy describes, "test prep" sounds like "educating students". It's educating them in the wrong things. For example, goal of class: better thinker by learning to do multiplication. Because of test, they only learned memorizing multiplication tables.
- yummyfajitas 12y ago
  The tests ask for multiplying 3 digit numbers. That's a lot of memorizing.
  - octo_t 12y ago
    that was a simple example. A better example might be memorizing quotes from a book to regurgitate in the test* * (I had to do this as part of GCSE English in the UK - I expect a similar situation in the US)
    - yummyfajitas 12y ago
      Why don't you look at the real tests and see if such tricks are possible?
dalke 12y ago
I defined what it meant to game a test, and gave definitions which fit the example. You have rejected my definition, without giving an alternative. "That's a useful skill" is not a useful educational criterion. Knowing how to change a tire on a car is a useful skill, but it's not appropriate for a math course. "Educating students" is also a useless criterion. Education is a never-ending process. I'm still learning things now. Schools by necessity must restrict themselves to certain topics. A Spanish teacher cannot simply use "I'm educating students" as an excuse to spend four weeks on Canadian politics in the 1970s. If test prep is so important, why isn't it its own course, where the teachers are trained for it, and where there are specific curriculum goals? "This is why tests are standardized, not left up to the schools or teachers" - are you willfully ignoring the point? Someone defines the standards. The page I linked to suggests that the standards for this New York test were defined by people who want the public school systems to fail, as part of the general effort to privatize public school. As http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/08/07/what-big-drop-in-new-standardized-test-scores-really-means/ http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/08/... points out: > The bottom line is that there are tremendous financial interests driving the agenda about our schools — from test makers, to publishers, to data management corporations — all making tremendous profits from the chaotic change. When the scores drop, they prosper. When the tests change, they prosper. When schools scramble to buy materials to raise scores, they prosper. There are curriculum developers earning millions to created scripted lessons to turn teachers into deliverers of modules in alignment with the Common Core (or to replace teachers with computer software carefully designed for such alignment). This is all to be enforced by their principals, who must attend “calibration events” run by “network teams.” You even used the passive "that's why tests are standardized" - who standardized the tests, what political and financial goals influence them, and how transparent is the standardization process? When the schools and teachers define the tests, then these issues are much clearer, and any failures are limited to just the school or teacher, and not systemic to the entire state. Hence why it's possible, and likely, that some test standardizers have gamed the test results, under my concrete definition of "gaming."
- yummyfajitas 12y ago
  I agree that if you want to define education in the course material as "gaming", the tests can indeed be gamed. Knowing how to reject an obviously false answer (e.g., "23.6 x 10.9 = ? a) 1,000,000, b) 0.000000001, c) 257.24 d) 527.24) is part of learning math. As for who defines the standards, the answer is our politicians or whoever they delegate to. And if you define gaming as "defining a standard", then yes, the test creators also game the system. So far, you haven't actually pointed to any part of the standard that you object to. Nor have you pointed out any sort of gaming other than "teaching the material on the tests, including how to ballpark answers".
zb 12y ago
Ironically, standardised testing is a prime example of a system choosing to solve an easy problem instead of the problem it has. Hard question: are our schools meeting the educational needs of their pupils and society? Easy question: did enough pupils fill in enough of the right circles on this test for us all to not get fired next year? If you exchanged test scores for production figures and teachers for farmers, this article could be a story directly out of Soviet agriculture in the collectivisation era. I'm finding it highly entertaining to watch you, of all people, defending this system so vehemently.
- sp332 12y ago
  If you don't have standardized testing, how do you know if the students are being educated? More specifically, how do you identify schools that need help and focus resources effectively?
  - amputect 12y ago
    The problem is that standardized testing doesn't tell you if students are being educated, it tells you if students are passing standardized tests, which is a different problem that, at best, only partially overlaps with the actual question. If you over-optimize for standardized testing, you end up under-optimizing for actual education. I had to take the WASL as part of my graduation from high school, back in 2003. Forcing this requirement on schools did not improve our education, it degraded it. Instead of actually reading two or three good books in our english class, we studied cliff notes of 15 or so works, so we could write shallow summaries of "key themes" for the test, for whichever books actually appeared on it. Instead of going forward with trigonometry in math, we went over estimation and hammered on very basic geometry problems. My government class stopped covering its subject matter entirely, and we studied analogies and reading comprehension, because government wasn't on the WASL (at the time; it might be now). The net effect was that as a class we did really well on that test, but our actual education suffered. That's what standardized testing gets you.
    - sp332 12y ago
      The problem is that standardized testing doesn't tell you if students are being educated But what will?
Liesmith 12y ago
>"How to spot the fake answers put there to fool you" == "how to see when an answer isn't even in the ballpark". That's a useful skill. That "context clues" thing suggests teaching students how to solve the problem in front of them, not the easy problem their mind wants to substitute for it[1]. From what this guy describes, "test prep" sounds like "educating students". OK, sure, it does sound like educating students, IF you accept that teaching people to spot inconsistencies and trap questions in a ritualized multiple choice test is a skill that transfers to other situations. Unfortunately this is really not the case at all. It's an education in how to navigate specific public-school bureaucracy. It doesn't even teach people to navigate other shitty bureaucracies.
sophacles 12y ago
The thing you are blatantly ignoring is this: standardized tests also teach children the lesson that there will be exactly one correct answer in all of life's situations, and that there will be exactly one OK method for every possible challenge faced. No exceptions. Which is sometimes true sure. But it is frustrating working with academic "stars" who internalize this once they come to the "real world". They are focused so much on getting the "right answer" within some narrowly defined context of right, that they can't see the better solution by reinterpreting the problem or recasting the assumptions in a slightly different order. They are unable to combine bits of knowledge from different buckets, because the test questions are all neatly siloed. For example, I've had this argument with fresh grads many, many times: me: you need to limit your UDP packets to 512 bytes (or 8K depending on the situation). them: but my teachers told me UDP packet size is a 16 bit integer. me: yeah, but many stacks cut off shorter, because there is a different standard that says routers can drop packets bigger than their preferred size, the only minimum is 512 bytes. them: my teacher told me that the packet size is a 16 bit field. Why are you talking about routers? me: because you need to combine information to actually solve a problem? them: whatever, I need to figure out what the bug in my code is causing these packets to be dropped. Or - me: hey $intern, let's figure out a few ways to solve approach this problem. expounds on the problem, lays out a few things that might work. The goal here is to try a few different techniques so we can work them into the bigger design. Any questions? intern: no. a few days later intern: Hey I think i solved the problem, is this one solution right? me: it's one way. It has some good stuff and bad stuff, but we want to try a few solutions to determine how to think about this. intern: looks like a lost puppy but is it right? Following conversation about multiple solutions and exploring solutions resembles "who's on first" The biggest problem with standardized testing is there is no room for the idea that outside of school, it isn't always about doing the rote thing, the simple siloed task in front of you, but rather incorporating various bits of knowledge, about applying the bits of knowledge in ways that allow task completion for tasks that aren't extremely well defined with a pre-arranged solution. In fact - the lack of a pre arranged solution is what defines most work outside of menial jobs. The idea that there is more than one approach or solution to something is antithetical to the core of standardized testing. (keep in mind - that for the statistics to be meaningful, the tests can't allow for grading criterial other than "one strictly correct answer" or you end up with issues in the numbers as the result of graders being different.)
6d0debc071 12y ago
> "How to spot the fake answers put there to fool you" == "how to see when an answer isn't even in the ballpark". That's a useful skill. It's a probabilistic skill, and in so far as the test is designed to give a measure of what someone knows, and not how lucky they've been, it's gaming the system. Is it a useful skill? Well, yes. To an extent. So is knowing your addition table, but we expect students to have moved somewhat beyond that by the time they're in secondary education. Just as we'd expect someone educated for five years, six or seven hours a week, 28-40 weeks a year, to have advanced somewhat beyond the need for discarding comedy answers as a viable test strategy. By secondary level, we'd expect them to know, (or be capable of running the calculation,) to decide among those answers that are actually in the ball park. Approaches for which the cost is more or less constant regardless of how many answers are on the page: You trust your calculation or memory to have given you the answer and discard all others by default.

rando289 12y ago

> From what this guy describes, "test prep" sounds like "educating students". It's educating them in the wrong things. For example, goal of class: better thinker by learning to do multiplication. Because of test, they only learned memorizing multiplication tables.

yummyfajitas 12y ago
The tests ask for multiplying 3 digit numbers. That's a lot of memorizing.
- octo_t 12y ago
  that was a simple example. A better example might be memorizing quotes from a book to regurgitate in the test* * (I had to do this as part of GCSE English in the UK - I expect a similar situation in the US)
  - yummyfajitas 12y ago
    Why don't you look at the real tests and see if such tricks are possible?

dalke 12y ago

I defined what it meant to game a test, and gave definitions which fit the example. You have rejected my definition, without giving an alternative. "That's a useful skill" is not a useful educational criterion. Knowing how to change a tire on a car is a useful skill, but it's not appropriate for a math course. "Educating students" is also a useless criterion. Education is a never-ending process. I'm still learning things now. Schools by necessity must restrict themselves to certain topics. A Spanish teacher cannot simply use "I'm educating students" as an excuse to spend four weeks on Canadian politics in the 1970s. If test prep is so important, why isn't it its own course, where the teachers are trained for it, and where there are specific curriculum goals? "This is why tests are standardized, not left up to the schools or teachers" - are you willfully ignoring the point? Someone defines the standards. The page I linked to suggests that the standards for this New York test were defined by people who want the public school systems to fail, as part of the general effort to privatize public school. As http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/08/07/what-big-drop-in-new-standardized-test-scores-really-means/ http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/08/... points out: > The bottom line is that there are tremendous financial interests driving the agenda about our schools — from test makers, to publishers, to data management corporations — all making tremendous profits from the chaotic change. When the scores drop, they prosper. When the tests change, they prosper. When schools scramble to buy materials to raise scores, they prosper. There are curriculum developers earning millions to created scripted lessons to turn teachers into deliverers of modules in alignment with the Common Core (or to replace teachers with computer software carefully designed for such alignment). This is all to be enforced by their principals, who must attend “calibration events” run by “network teams.” You even used the passive "that's why tests are standardized" - who standardized the tests, what political and financial goals influence them, and how transparent is the standardization process? When the schools and teachers define the tests, then these issues are much clearer, and any failures are limited to just the school or teacher, and not systemic to the entire state. Hence why it's possible, and likely, that some test standardizers have gamed the test results, under my concrete definition of "gaming."

yummyfajitas 12y ago
I agree that if you want to define education in the course material as "gaming", the tests can indeed be gamed. Knowing how to reject an obviously false answer (e.g., "23.6 x 10.9 = ? a) 1,000,000, b) 0.000000001, c) 257.24 d) 527.24) is part of learning math. As for who defines the standards, the answer is our politicians or whoever they delegate to. And if you define gaming as "defining a standard", then yes, the test creators also game the system. So far, you haven't actually pointed to any part of the standard that you object to. Nor have you pointed out any sort of gaming other than "teaching the material on the tests, including how to ballpark answers".

zb 12y ago

Ironically, standardised testing is a prime example of a system choosing to solve an easy problem instead of the problem it has. Hard question: are our schools meeting the educational needs of their pupils and society? Easy question: did enough pupils fill in enough of the right circles on this test for us all to not get fired next year? If you exchanged test scores for production figures and teachers for farmers, this article could be a story directly out of Soviet agriculture in the collectivisation era. I'm finding it highly entertaining to watch you, of all people, defending this system so vehemently.

sp332 12y ago
If you don't have standardized testing, how do you know if the students are being educated? More specifically, how do you identify schools that need help and focus resources effectively?
- amputect 12y ago
  The problem is that standardized testing doesn't tell you if students are being educated, it tells you if students are passing standardized tests, which is a different problem that, at best, only partially overlaps with the actual question. If you over-optimize for standardized testing, you end up under-optimizing for actual education. I had to take the WASL as part of my graduation from high school, back in 2003. Forcing this requirement on schools did not improve our education, it degraded it. Instead of actually reading two or three good books in our english class, we studied cliff notes of 15 or so works, so we could write shallow summaries of "key themes" for the test, for whichever books actually appeared on it. Instead of going forward with trigonometry in math, we went over estimation and hammered on very basic geometry problems. My government class stopped covering its subject matter entirely, and we studied analogies and reading comprehension, because government wasn't on the WASL (at the time; it might be now). The net effect was that as a class we did really well on that test, but our actual education suffered. That's what standardized testing gets you.
  - sp332 12y ago
    The problem is that standardized testing doesn't tell you if students are being educated But what will?

Liesmith 12y ago

>"How to spot the fake answers put there to fool you" == "how to see when an answer isn't even in the ballpark". That's a useful skill. That "context clues" thing suggests teaching students how to solve the problem in front of them, not the easy problem their mind wants to substitute for it[1]. From what this guy describes, "test prep" sounds like "educating students". OK, sure, it does sound like educating students, IF you accept that teaching people to spot inconsistencies and trap questions in a ritualized multiple choice test is a skill that transfers to other situations. Unfortunately this is really not the case at all. It's an education in how to navigate specific public-school bureaucracy. It doesn't even teach people to navigate other shitty bureaucracies.

sophacles 12y ago

The thing you are blatantly ignoring is this: standardized tests also teach children the lesson that there will be exactly one correct answer in all of life's situations, and that there will be exactly one OK method for every possible challenge faced. No exceptions. Which is sometimes true sure. But it is frustrating working with academic "stars" who internalize this once they come to the "real world". They are focused so much on getting the "right answer" within some narrowly defined context of right, that they can't see the better solution by reinterpreting the problem or recasting the assumptions in a slightly different order. They are unable to combine bits of knowledge from different buckets, because the test questions are all neatly siloed. For example, I've had this argument with fresh grads many, many times: me: you need to limit your UDP packets to 512 bytes (or 8K depending on the situation). them: but my teachers told me UDP packet size is a 16 bit integer. me: yeah, but many stacks cut off shorter, because there is a different standard that says routers can drop packets bigger than their preferred size, the only minimum is 512 bytes. them: my teacher told me that the packet size is a 16 bit field. Why are you talking about routers? me: because you need to combine information to actually solve a problem? them: whatever, I need to figure out what the bug in my code is causing these packets to be dropped. Or - me: hey $intern, let's figure out a few ways to solve approach this problem. expounds on the problem, lays out a few things that might work. The goal here is to try a few different techniques so we can work them into the bigger design. Any questions? intern: no. a few days later intern: Hey I think i solved the problem, is this one solution right? me: it's one way. It has some good stuff and bad stuff, but we want to try a few solutions to determine how to think about this. intern: looks like a lost puppy but is it right? Following conversation about multiple solutions and exploring solutions resembles "who's on first" The biggest problem with standardized testing is there is no room for the idea that outside of school, it isn't always about doing the rote thing, the simple siloed task in front of you, but rather incorporating various bits of knowledge, about applying the bits of knowledge in ways that allow task completion for tasks that aren't extremely well defined with a pre-arranged solution. In fact - the lack of a pre arranged solution is what defines most work outside of menial jobs. The idea that there is more than one approach or solution to something is antithetical to the core of standardized testing. (keep in mind - that for the statistics to be meaningful, the tests can't allow for grading criterial other than "one strictly correct answer" or you end up with issues in the numbers as the result of graders being different.)

6d0debc071 12y ago

> "How to spot the fake answers put there to fool you" == "how to see when an answer isn't even in the ballpark". That's a useful skill. It's a probabilistic skill, and in so far as the test is designed to give a measure of what someone knows, and not how lucky they've been, it's gaming the system. Is it a useful skill? Well, yes. To an extent. So is knowing your addition table, but we expect students to have moved somewhat beyond that by the time they're in secondary education. Just as we'd expect someone educated for five years, six or seven hours a week, 28-40 weeks a year, to have advanced somewhat beyond the need for discarding comedy answers as a viable test strategy. By secondary level, we'd expect them to know, (or be capable of running the calculation,) to decide among those answers that are actually in the ball park. Approaches for which the cost is more or less constant regardless of how many answers are on the page: You trust your calculation or memory to have given you the answer and discard all others by default.