3 ms·

> The only way to avoid it is to make the test subjective and non-standard. How so? You can make it multiple-choice (objective) difficult, non-algorithmic math

by aianus 11y ago

> The only way to avoid it is to make the test subjective and non-standard. How so? You can make it multiple-choice (objective) difficult, non-algorithmic math problems comprising multiple concepts each and then grade the whole test on a curve. Impossible to game except to actually teach the students a broad range of math skills and how to problem-solve, while still objective and standard (in the sense that everyone gets the same test and is graded the same).

iheartmemcache 11y ago
Ooh yay, a topic I can speak fairly well-informed on. >> You can make it multiple-choice (objective) difficult, non-algorithmic math problems comprising multiple concepts each and then grade the whole test on a curve. You do realize that that's what Olympiads are right? https://en.wikipedia.org/wiki/International_Mathematical_Olympiad https://en.wikipedia.org/wiki/International_Mathematical_Oly... You can be fairly confident the Chinese "teach for the test" to a few handpicked kids who can perform well on those problems (in the same way they select children with an aptitude for gymnastics at an early age). Nearly 20 wins, not one Fields Medal. You can 'teach to the test' at any level up to the 400 levels, at which point you start creating new ways to approach the test, solving problems which weren't solved for a couple hundred years, or founding problems which won't be solved for another couple hundred.
mikekchar 11y ago
As a former English as a Foreign Language teacher, I can assure you that there are always ways to game exams in English. Because I had to help some of my students get into university, I took a look at the entrance exams. Even though I am a native English speaker, I found that I could not finish the Todai (top school in Japan) entrance exam for English in the required time. It required reading and understanding very large amounts of text and then answering difficult multiple choice questions at the end. Sometimes the questions were difficult to understand and required that you were really careful about concentrating on the wording. So instead, I decided to try again (with a different year's test) and instead of reading the material ahead of time, I used the questions to direct my efforts. I would look at all the answers and then scan the text for areas that talked about those things. Having identified those, I then looked at the question and ruled out the answers that didn't seem related. Finally I settled on the answers -- mainly by applying grammar rules mechanically (i.e., a wrong answer would be in the wrong tense compared to the supplied text). I was able to complete the exam in time and get full marks. I had absolutely no idea what the text said. In fact, I didn't even know what half of the questions asked. Now for my dilemma as a teacher. If my student gets accepted to Todai, it will basically make their life. You are almost guaranteed to get a good job that will give you the kind of life that most Japanese people are looking for. If I ask the student to try to do the exam properly, by reading the text, they will fail. Even I would fail. I can teach them to pass it in a different way. What should I do? The test is too difficult. There are always going to be ways to game it and the successful students do. Therefore the test gets more difficult and the students are forced to game it even more. As a teacher, should I tell my student that they can't go to Todai, or should I teach them how to game the exam? As far as I was concerned, I decided that teaching English and passing entrance exams were 2 different things. I taught English for those students who were interested. I gave up a lot of my free time (a few hours every day) to help them do that. When exam time came, though, I taught no English at all. I taught them how to do what they needed to do to be successful. Every one of my colleagues did the same thing. Probably every teacher in the country does the same thing. Standardized tests that grade you and decide your future are insane.
- aianus 11y ago
  It sounds like it's gameable because it's predicatable. Imagine the next year's Todai entrance exam was totally different and unpredictable (say, transcribing an Eminem song or something similarly off-the-wall), then all the people who studied ways to game last year's exam would be SOL and only the people who actually know English would get in. That's the kind of system I'm advocating for: get a bunch of smart test writers together with the express intent of coming up with novel ways to destroy the people who game the exams.
  - mikekchar 11y ago
    The reason they can't reasonably make ungameable tests is that they have tens of thousands of applicants. The need for an easy to mark test is more important than the need for an ungameable test. Otherwise you could simply interview applicants in English. (Todai actually considered this approach, but I understand that they abandoned it as being impractical). In my more cynical periods, I suspect that they don't mind people gaming the exams. After all, most Todai students don't actually need to speak English (or know crazy details of history, or write thousands of kanji, or whatever). They are more interested in clever students who are willing to study like insane people and practice boring stuff over and over again. That's what you need to do to game the exams. In a twisted way, it works...
pps43 11y ago
Making test questions more difficult won't help. More people will make simple stupid mistakes, so it's going to be more like a lottery. If you forgive simple stupid mistakes, the test is becoming subjective again. Here's an example of a bad question: what is the area of the right triangle with hypotenuse 10 and height 6 (height is measured from the vertex opposite to the hypotenuse)? It does not test understanding. The optimal algorithm for the student is to parse the question looking for keywords - "triangle", "area", "height", recalling the formula, and plugging the numbers into it to get the answer 30. The question is from the real test, and by the way, this answer is incorrect. Why it is incorrect is a better question. Here's an example of a good question: prove the Pythagorean Theorem. You can't answer it by picking the right answer from the multiple choice list. You have to explain your line of reasoning to the examiner and be able to answer question like "why you say these two triangles are congruent". Such interactive process is much better at determining student's skill level, and you can't prepare for it by simply memorizing stuff without understanding. The student might not like the proof in the textbook and prefer another proof. It take a human to understand whether this other proof holds water.

by aianus 11y ago

iheartmemcache 11y ago

Ooh yay, a topic I can speak fairly well-informed on. >> You can make it multiple-choice (objective) difficult, non-algorithmic math problems comprising multiple concepts each and then grade the whole test on a curve. You do realize that that's what Olympiads are right? https://en.wikipedia.org/wiki/International_Mathematical_Olympiad https://en.wikipedia.org/wiki/International_Mathematical_Oly... You can be fairly confident the Chinese "teach for the test" to a few handpicked kids who can perform well on those problems (in the same way they select children with an aptitude for gymnastics at an early age). Nearly 20 wins, not one Fields Medal. You can 'teach to the test' at any level up to the 400 levels, at which point you start creating new ways to approach the test, solving problems which weren't solved for a couple hundred years, or founding problems which won't be solved for another couple hundred.

mikekchar 11y ago

As a former English as a Foreign Language teacher, I can assure you that there are always ways to game exams in English. Because I had to help some of my students get into university, I took a look at the entrance exams. Even though I am a native English speaker, I found that I could not finish the Todai (top school in Japan) entrance exam for English in the required time. It required reading and understanding very large amounts of text and then answering difficult multiple choice questions at the end. Sometimes the questions were difficult to understand and required that you were really careful about concentrating on the wording. So instead, I decided to try again (with a different year's test) and instead of reading the material ahead of time, I used the questions to direct my efforts. I would look at all the answers and then scan the text for areas that talked about those things. Having identified those, I then looked at the question and ruled out the answers that didn't seem related. Finally I settled on the answers -- mainly by applying grammar rules mechanically (i.e., a wrong answer would be in the wrong tense compared to the supplied text). I was able to complete the exam in time and get full marks. I had absolutely no idea what the text said. In fact, I didn't even know what half of the questions asked. Now for my dilemma as a teacher. If my student gets accepted to Todai, it will basically make their life. You are almost guaranteed to get a good job that will give you the kind of life that most Japanese people are looking for. If I ask the student to try to do the exam properly, by reading the text, they will fail. Even I would fail. I can teach them to pass it in a different way. What should I do? The test is too difficult. There are always going to be ways to game it and the successful students do. Therefore the test gets more difficult and the students are forced to game it even more. As a teacher, should I tell my student that they can't go to Todai, or should I teach them how to game the exam? As far as I was concerned, I decided that teaching English and passing entrance exams were 2 different things. I taught English for those students who were interested. I gave up a lot of my free time (a few hours every day) to help them do that. When exam time came, though, I taught no English at all. I taught them how to do what they needed to do to be successful. Every one of my colleagues did the same thing. Probably every teacher in the country does the same thing. Standardized tests that grade you and decide your future are insane.

aianus 11y ago
It sounds like it's gameable because it's predicatable. Imagine the next year's Todai entrance exam was totally different and unpredictable (say, transcribing an Eminem song or something similarly off-the-wall), then all the people who studied ways to game last year's exam would be SOL and only the people who actually know English would get in. That's the kind of system I'm advocating for: get a bunch of smart test writers together with the express intent of coming up with novel ways to destroy the people who game the exams.
- mikekchar 11y ago
  The reason they can't reasonably make ungameable tests is that they have tens of thousands of applicants. The need for an easy to mark test is more important than the need for an ungameable test. Otherwise you could simply interview applicants in English. (Todai actually considered this approach, but I understand that they abandoned it as being impractical). In my more cynical periods, I suspect that they don't mind people gaming the exams. After all, most Todai students don't actually need to speak English (or know crazy details of history, or write thousands of kanji, or whatever). They are more interested in clever students who are willing to study like insane people and practice boring stuff over and over again. That's what you need to do to game the exams. In a twisted way, it works...

pps43 11y ago

Making test questions more difficult won't help. More people will make simple stupid mistakes, so it's going to be more like a lottery. If you forgive simple stupid mistakes, the test is becoming subjective again. Here's an example of a bad question: what is the area of the right triangle with hypotenuse 10 and height 6 (height is measured from the vertex opposite to the hypotenuse)? It does not test understanding. The optimal algorithm for the student is to parse the question looking for keywords - "triangle", "area", "height", recalling the formula, and plugging the numbers into it to get the answer 30. The question is from the real test, and by the way, this answer is incorrect. Why it is incorrect is a better question. Here's an example of a good question: prove the Pythagorean Theorem. You can't answer it by picking the right answer from the multiple choice list. You have to explain your line of reasoning to the examiner and be able to answer question like "why you say these two triangles are congruent". Such interactive process is much better at determining student's skill level, and you can't prepare for it by simply memorizing stuff without understanding. The student might not like the proof in the textbook and prefer another proof. It take a human to understand whether this other proof holds water.