7 ms·

Why I Let My Students Cheat On Their Exam

267 points by SeanDav 14y ago

danpalmer 14y ago
This reminds me of a slightly, although not entirely, related paper that I read a while ago titled "Embracing the Kobayashi Maru – Why You Should Teach Your Students to Cheat". I've linked to it in this item: https://news.ycombinator.com/item?id=5609346 https://news.ycombinator.com/item?id=5609346 because I think it is different enough to warrant a separate discussion.
guimarin 14y ago
This would have been more interesting to me as a study in game theory, if some of the 'mob' had given false or incomplete answers to the rest of the group on a minority of the questions, saving the 'fully-correct' answers for themselves. That way, if there was a curve they would benefit totally from the 'mob' and still come out ahead of it.
- SEMW 14y ago
  Surely one of the main points of the exercise was that, precisely because there wasn't a curve, whole-class cooperation can result in everyone in the class getting a higher score (20% higher) than they usually would. Grading on a curve wouldn't be the same game just 'more interesting', it'd be a completely different game, and one that's lost most of the "To win at some games, cooperation is better than competition" aspect.
songzme 14y ago
It should be fair to note that experiments like this can only work in small groups. In much larger class sizes chaos will ensue. It will also be an interesting experiment to take a large group of students, split them into groups, and then give out exams to each group. By adding a factor of competitiveness test scores might even go higher. Competition is everywhere in nature.
- OnyeaboAduba 14y ago
  I spent some of my highschool years in a differnt country in boarding school and grades and class rank where constantly posted for everyone to see It brought out a real competitive spirit in me and in the whole class and maybe a little shame for the the people at the bottom of the list . I thought It was a very effective tool in keeping the class as whole engaged but that type of thing would never be allowed in the US for multitude of reasons.
  - songzme 14y ago
    The reason that class ranking is not prevalent in US is because of publicly shaming the students who are not doing as well as the other. This can be mitigated if students are working in groups, and group scores are posted instead of individual scores. This way, a group who is scoring low can help and inspire each other to do well and share the pain (shame)
    - rdouble 14y ago
      In American public schools, class ranking leads to public shaming of the students who are doing well...
      - aspensmonster 14y ago
        Depends on the school. At the public high school I attended --several years on various top 100 lists and top 10 in the state-- class ranking was most definitely a shaming of the "weird" minority of kids who were taking "normal" classes. These normal classes were on a completely different scoring tier that ensured the students taking them could not even break the 50th percentile. And coming from Texas, where getting into the top 10 percent of your graduating class is an automatic ticket to any state university, the competition was most definitely fierce.
  - mpyne 14y ago
    That's what we do when training the Navy nukes here in the U.S.
guimarin 14y ago
I am constantly surprised that we have a 'moral' aversion to cheating. IMO school teaches two strategies for success. 1.) work hard within the rules and 2.) cheat but don't get caught. Each is equally valid in society, even without a 'moral' component.
- icegreentea 14y ago
  The 'moral' aversion to cheating I believe comes out the very game theory and evolutionary behavior that this post talks about. When society sets up rules and systems, there is a maximum amount of cheating/parasitism that it can withstand before the system breaks down, and everyone (at least the majority of the non-cheaters) loses out (assuming a 'good' system). I think we all know/feel this at a very deep level, and understand that if we want to set up and maintain cooperative systems, cheating and parasitism must be kept to a sufficiently low level. Cheaters and 'goodies' all know this. That our western civilization morality tends to align with the interests of large-scale cooperative systems is probably no accident. I limit this to western cause I have no clue about any other cultures and I don't want to make an ass of myself. Of course, this in itself says nothing about the goodness of the system. Cheating a bad system is always an interesting thought experiment. Breaking bad systems and replacing with a new, better one often requires a great degree of 'cheating'. All sorts of interesting things here.
  - guimarin 14y ago
    I think this is a complex and naunced issue. I have a number of thoughts on your arguments. Society doesn't set up rules and systems, they seem to emerge as a consequence of beneficial cooperative behavior towards a 'goal'. When it's no longer beneficial to cooperate towards a goal, over say other strategies like cheating, the rules are overlooked, passed, or changed as quickly as they emerged. The prevalence of War as a way to 'rebalance' resources comes to mind immediately, but so too does the reality that only 2% of people pay use taxes. To your point about parasitism. Human societies are always in a state of change with respect to sociological and technological concerns, and the success of a society in achieving its 'goals' seems largely dependent on how well it adapts to this change. The parasitism you've described seems to be more a reality of economics than society in general, and though your point seems valid on its face, You wouldn't say that the success companies like Uber, which are cheating parasites by your definition, is really a net negative for the society. to your point about morality. I believe morality is really orthogonal to this issue and should be considered from different viewpoints in different contexts because humans can hold contradictory points of view and still remain 'functioning'. History holds many examples of individuals who challenge what is 'moral' in a society through cheating, and they are sometimes lauded, sometimes not. I think of the narrative surrounding Rosa Parks on one side ( in support of cheating ) and the story of Cincinnatus ( in support of not cheating ) when considering morality in the context of the cooperation cheating dichotomy. and you're right, this is a very interesting topic, definitely outside the scope of this small classroom experiment.
auctiontheory 14y ago
If you enjoy this kind of thinking, you'll love Dan Ariely's Behavioral Economics class when it next comes to Coursera.
- SilasX 14y ago
  Because we'll enjoy cheating on it so much more, or something?
cm2012 14y ago
It's not cheating if everyone is following the same rules. Cheating is almost by definition an unfair advantage, when this was pretty close to a level playing field.
- tkahn6 14y ago
  And how about the students in different sections not taught by this teacher who have a closed-book no-collaboration final exam?
deleted 14y ago
[deleted]
- crosvenir 14y ago
  I was thinking that what the students did during the test here was what my classmates and I did for almost every homework assignment. We would work through the engineering problems (sometimes individually/sometimes as a group), go over them, ask/answer questions, and move on. I learned so much doing this (mostly that there is more than one way to perform analysis to arrive at an answer and that engineers love looking up the answer in the back of the book and working backwards). I wonder if this would be more aligned with how the grading system works as a method by which periodic quizzes are given. We had lone wolves/busy students who couldn't participate, but I think everyone would have benefited from this as a class group activity.
btilly 14y ago
I would be curious what the students thought of the process. I am sure that the essay was good. I am sure that some learned a lot. But how many just road coattails and didn't learn much other than how to copy? In a game like this where everyone stands to gain, and nobody loses, the incentives are to let everyone participate because the cost of riding is free. How many took advantage of that? Also on a task like this, individual personalities of key figures matter a lot. My guess is that just a few individuals were responsible for a lot of the dynamics. The whole exercise would have been more interesting - and more competitive - if he had started with a grading scale at the start, same rules, but said that he would rank answers against each other, with multi-way ties being ranked at the median of the group. (So if 5 people tie for 3rd, they will all be scored as 5th.) This would push cooperation/competition to a new high.
- CurtMonash 14y ago
  That's a good approach for a generic class on game theory. It's not so good for the specific case of animal behavior, in which the whole herd will indeed often prosper or suffer at once.
Siecje 14y ago
You should have had more questions to have more strategy. That was a pretty easy test... You also gave them an easy way out if it was too hard...
keerthiko 14y ago
I saw this on a Naruto episode first (http://www.hulu.com/watch/36407#i0,p20,s1,d0 http://www.hulu.com/watch/36407#i0,p20,s1,d0), about 6 months before I started college. For those who don't want to watch the episode, it's a written exam ninja trainees (genin) need to take to be promoted to ninja squad leader (chuunin). Any team (of 3) that has a single member fail will be failed as a group. If any ninja gets caught cheating they will be given a warning, and after three warnings they will be failed. SPOILER (no really, the episode is great in this context even if you don't like/watch anime much) The intelligent (but not book-smart) notice the leniency of the anti-cheating-system and the carefully worded instructions to "not get caught", and then find ways to obtain answers from "sources", qualified chuunin who were intentionally planted in the exam hall who knew the answers, or to collaborate with teammates. End plot summary I think it's important to teach kids not just the answers to problems posed, but also ways to find them. If it's too easy for them to find answers without rigid rules, it probably means that he problems we are posing them are not hard or thought-provoking enough in the first place. Edited for a missed phrase
- nsxwolf 14y ago
  Interestingly this is the only episode of Naruto I've ever seen. I remember being quite impressed at how cerebral it was, as I was expecting to see a bunch of ninjas running around slicing off each other's heads.
lt 14y ago
Instead, they were changing their goal in the Education Game from “Get a higher grade than my classmates” to “Get to the best answer.” The goal shouldn't be “Get a higher grade than my classmates”, it should be “Get the highest grade possible”. And the goal for the teacher should be “Translate each student knowledge data into a numeric value”, which is somewhat orthogonal to the experiment.
- aspensmonster 14y ago
  "Translate each student knowledge data into a numeric value" Which isn't at all easy. Grades certainly have some correlation to knowledge (certainly not "one" though), but in a perfect world we'd be putting everyone through oral examination and putting their projects through peer review on a constant basis. This is expensive though, in time and money and man power.
- willismichael 14y ago
  Shouldn't the goal be to "Learn the most possible"? I know, it's not quantifiable, so maybe it doesn't qualify as a goal at all, but it concerns me when people act like the purpose of education is to get grades.
  - cmstoken 14y ago
    You'd be surprised to learn how many college students goals aren't actually to "Learn the most possible."
    - tkahn6 14y ago
      I'd be more surprised if even 1% of college students had to goal to learn as much as possible as opposed to get the highest grades possible.
    - benblodgett 14y ago
      Schools emphasize the numerical values that represent the 'amount learned', not the actual acquisition of knowledge. Any college student with the sole goal of learning how to learn & think unfortunately doesn't survive the education system.
Millennium 14y ago
So really, this was was a class project involving a scenario that superficially resembled cheating on an exam, rather than actually cheating on an exam.
geebee 14y ago
I loved this approach until I read the exam question. After that, I only liked it. It would have been more interesting to see an exam with question like "is the population of x in hardy-weinberg equilibrium", where x is something students should know something (but not everything) about if they've been paying attention. It would be possible, but not easy, for a very well prepared student to do the research and figure it out (or at least write an intelligent answer) in an hour. Some bright students who haven't studied much probably would be able to research quickly and come up with a good answer, especially if they split up the task. Some people will clearly be vastly more valuable to teams than others. In short, I think the professor here picked a question that tilts too heavily toward collaboration. If this approach is really going to work, you need to induce some reluctance among the very well prepared, who may realize that their friends who spent more time preparing for the non-collaborative physics exam will be getting a relatively free ride. I guess I just don't think this quite brought out the nastiness that would truly arise in a state of nature - something of course I understand he'd rather avoid!
forgotAgain 14y ago
So is the article title another experiment in Behavioral Ecology?
droithomme 14y ago
1. This obviously wasn't cheating since it was within the prescribed parameters for the exam. Some exams are open book, some are open note, some are even take home with no limits. 2. The highest score and thus most insightful and highest quality work, according to the professor's own evaluation, came not from collaboration but from one of the 3 lone wolves.
- pixelcort 14y ago
  The average was higher than that of prior classes. Also, it is implied that the lone wolves may have had access to some of the work the mob was producing, enabling one of them to get ahead by simply correcting a few mistakes without giving them back to the mob.
  - cschep 14y ago
    This immediately reminded me of open source software. Interesting.
  - scott_s 14y ago
    She explicitly stated so: Although the Wolves listened and contributed to discussions, they preferred their individual variants over the Mob’s joint answer.
  - probably_wrong 14y ago
    I don't think they necessarily "corrected a few mistakes without giving them back to the mob". There is also a chance that they indeed proposed their answers to the mob, but they didn't like them and rejected them. It wouldn't be the first time that a great idea is shot down because it's too far from everyone's comfort zone. But I'm just guessing.
- steveklabnik 14y ago
  1: From TFA: > Although by conventional test-taking rules, the students were cheating, they actually weren’t in this case. 2: The mob still contributed to the work of the lone wolves, it was absolutely still 'collaboration': > Although the Wolves listened and contributed to discussions, they preferred their individual variants over the Mob’s joint answer.
tenpoundhammer 14y ago
The most accurate measurement of what a person has learned over a period of time is their ability at which they are generally able to discuss a subject. The best exam would be a personal exam in which the teacher orally administers a series of generalized questions and explorers the subject matter with each student to understand their actual level of understanding. While it would take a long time to administer these tests for every student it would provide a lot of value for many subjects, while a written test would still be important to determine particular skill sets for certain subjects like math. Any average student can jam a notecard full of likely facts or cram facts into their head the night before an exam, without learning much. Likewise working as a team to generate the best answer can allow people to receive a high grade without knowing much. There is also the fact that even a well read student with a good understanding of the subject matter can a fail a test because they don't have a the particular points asked on the test.
- lifesavers 14y ago
  I like this approach as well (in theory) and only have some "anecdotal" experience - if you can call it that. My family originated from East Europe and as such most of my family elders have an old school Soviet experience in education. Consequently they have little knowledge and no experience when it comes to modern-day Western education infrastructure. For instance, my parents were genuinely surprised to hear that I wrote written exams and submitted them only to be reviewed at face value - which is the experience of nearly all undergrads I expect. Their experience was that, even in a "straight-forward" subject like Mathematics, they had to first submit a written response to a problem set in an examination type settings - and then upon completion they would have to explain their solution and approach one-on-one with a faculty member who would only then submit a grade. Obviously this was done to weed out the cheaters who couldn't explain why they had to correct answers to all the questions. But the result can't be ignored that one-on-one oral examinations, while not exactly pragmatic, are very effective!
nollidge 14y ago
And yet again, here's 25 comments from the Hacker News Pedant Squad to remind you that it's not REALLY cheating if the teacher lets you, in case you only recently disembarked from an 18-wheeler full of turnips.
- calibraxis 14y ago
  What's unfortunate about such comments is that such departures from the norm actually are considered bad ("cheating") according to the ideology of mainstream education. The article's author wasn't the only teacher with the bright idea to have collaborative tests, where he was willing to answer whatever the students asked. I know of at least one teacher who was forced to stop. Education could look very different, and more diverse. Universities have certain functions in society. If they don't fulfill those functions, they won't get government and corporate funding. Even Salman Khan explains that one important function of schools is to subjugate, fashioning people into obedient cogs. (In his "The One World School House", in the chapter "The Prussian Model".)
dice 14y ago
I had a professor who used a similar concept of fostering active discussion and debate within the classroom, except he did it for every lecture. He would present a concept, then ask us a series of questions about it. Students would first think on the question and vote on their own, then the question was opened to discussion and debate. After the topic had been hashed out we would re-vote. Often times the difference between the first and second votes were significant, sometimes resolving a split opinion and other times the majority would change their minds. It was a lot of fun, and easily the best and most informative experience of my educational career.
- nostrademons 14y ago
  Most of my high school education was like this. It's fairly common in "alternative" schools like Montessori, Waldorf, Essential, or Sudbury schools.
avalaunch 14y ago
I wonder if this was being graded on a curve. If so, an interesting strategy would have been to sabotage the group effort while secretly preparing a "lone wolf" answer. The saboteur could continually steer the conversation in the wrong direction and possibly even alter the group answer to something absurd before it was turned in. Or if he wanted to concentrate on his own work, he could bring some headphones to tune out the horrendous noise coming from outside the classroom where his friends happen to be practicing for their upcoming rock gig.
sriram_sun 14y ago
Here is what I would like the professor to do. Follow up with a "normal" test and see how the students perform. If the class average was greater that prior year performances, the co-operation actually helped in drilling the concepts in.
carlob 14y ago
When I was in high school my classmates and I found a document from a previous year that amounted to a superset of all the questions my philosophy teacher was going to put in test. Since some of the questions were pretty hard we ended up studying really hard together to try and crack them before tests. Though what we did was somehow against the rules I feel I learned a lot more by studying the questions collaboratively than by studying on my own. I later found this form of collaboration extremely rewarding and effective during university. We spent many afternoon trying to crack hard problem sets and explaining them to one another.
- tassl 14y ago
  It has always puzzled me why a professor would be willing to repeat the answers from previous years. Some account that people that work "harder" would be solving previous exams, but this is given that the exams are available to all the class (which is a prerequisite for this argument to be true). I've found myself in classes where a group had exams, problem sets and other information from previous years (and solved) and they hid and refused to share those.
  - Periodic 14y ago
    I've found that in my grad-school years that most professors released a set of sample exams, usually with answers. For some this would be a made-up exam. For most it was a previous year's exam. For others it was all the exams presented to the class for the last ten years with answers. It was quite productive to go over all these exams as a group and strive to understand the questions, and particularly to understand the patterns, these were the concepts that lay behind the questions and once you had those you could answer many questions. These days it isn't hard to find example problems and exams on the internet for your class or similar ones.
  - Steuard 14y ago
    It is hard to come up with entirely new problems for every exam every year. That doesn't mean that doing so isn't important, but it really is time consuming and a whole lot of work. It's also painfully easy to think that you've changed things enough for the question to be "new" compared with examples from class or homework or previous years' tests, only for some student to recognize it enough to regurgitate some memorized solution rather than actually thinking. So while I'd never say that repeating questions is a good thing, I can't be too hard on folks who do (intentionally or not).
temp453463343 14y ago
This is a completely idiotic way of testing people. I could have come into class not knowing anything, waited till the end of the test and then offered 100 dollars to anyone to be allowed to copy their whole test. And I would have passed the class? It kinda reminds me of CS-whores. The one or two CS girls in each graduation class that regularly used sex-appeal/flirtation to get CS guys to do their homework (very successfully too!). At some point your no longer testing people's knowledge and creative thinking, but their social skills and ability to manipulate others (and maybe their breast size/size of their 6 pack). I feel like the professor treated this like a sick little game and enjoyed himself while doing it. At the end of the day you are competing against your peers either through a curve, or through your raw grades because you're all going into the same fields and are competing for the same jobs with your GPAs. Making a system where cut-throat behavior is encouraged and putting down your pays off is really sad.
- MJR 14y ago
  I completely disagree. I found this to be an excellent example of real world training. I worked through college in IT and from that job on, working in the real world was exactly like this. We worked on projects with teams of people. We looked for the best possible solutions to every problem we faced. Even when working alone on something, you had your peers as resources. You weren't locked in a closet coding on a computer, offline, without access to books or other tools. That's what test taking was like for me. Some people, like me, are experiential learners. Others find their best thoughts through discussion and the evolution of ideas. I would have loved to have taken many classes like this. Fortunately, that's what the real world has been like for me - being a member or leading a team to produce the best product we can, either as a team, department or company. As you called it my "pay off" is education. I found mine through methods like this. Not through memorizing facts and methods just to pass a test.
  - temp453463343 14y ago
    While I agree that the process is more similar to the "real world" - the fundamental problem is that the grading is not. When you work as a team towards some goal whoever is evaluating you is doing so based on your contribution to the team. Here the fundamental scoring method is still the test, so an individual's motivation is to do better for himself; the teams success is secondary (and if the test is curved it may be non existent). Free riders aren't punished or penalized and all sorts of outside influences come in where people start trading on money/sex/favors/friendships/etc. Arguably those outside influences exist in the real world, but why create a system that explicitly has no mechanisms to counteract them? (ie. the prof says you can do anything that is legal and doesn't try to catch "cheaters")
CurtMonash 14y ago
I'm a big fan of unusual approaches to testing, good as I was at the conventional kind. In high school astronomy, we could bring in a few pages of notes. I once wrote out the notes in great detail, FORGOT THEM AT HOME, and got the highest score on the test. In college I learned a lot studying for tests, both multiple choice (geology) and essay (history), where the questions were highly predictable. I love the test Hilary Putnam announced the first day of every class: Part 1: Write a suitable question for this class. Part 2: Answer it. You will be graded on both parts. (Note: The recursive answer of giving him his own question back turns out to be infinitely long.) I once gave an open book test, saying exam questions would be chosen only from problems I'd assigned for homework. Students still did badly. (Ouch.)
lnanek2 14y ago
Too bad no one tried to sabotage the consensus answers. You could get responsibility for writing one of the group answers, write it wrong, then turn in your own test with it correct. Inject false data into discussions, etc.. It isn't really game theory without some interesting gains and loses due to cooperation or deception.
Brashman 14y ago
I like this idea, except that I'd be worried about money entering into the scheme. In a real world setting, it's not unreasonable to pay someone to do the work for you, but in a school/learning setting, I dislike the possibility of people paying their way out of the work.
f4stjack 14y ago
Actually this is not cheating. Cheating means copying somebody's content with or without his permission in an exam where this kind of information sharing is forbidden. In here, the lecturer specifies that information sharing is not forbidden. Therefore technically this is not "cheating". But it is wonderful to see when you remove the outside limiting factors and let the phenomena occurs in its own volition. Probably this will be one of the courses the participant students will never forget, instead of courses where you "memorize" everything until the last exam. Kudos to Mr. Nonacs.
acjohnson55 14y ago
A collaborative assignment is a really solid education tool. What's missing is an explanation of why the author thinks this led to an objective evaluation of what individual students learned before test day (if he does indeed think that). That is the point of an exam. Imagine if med students could collaborate on their boards. It would pretty well invalidate the purpose.

267 points by SeanDav 14y ago

This reminds me of a slightly, although not entirely, related paper that I read a while ago titled "Embracing the Kobayashi Maru – Why You Should Teach Your Students to Cheat". I've linked to it in this item: https://news.ycombinator.com/item?id=5609346 https://news.ycombinator.com/item?id=5609346 because I think it is different enough to warrant a separate discussion.

guimarin 14y ago

This would have been more interesting to me as a study in game theory, if some of the 'mob' had given false or incomplete answers to the rest of the group on a minority of the questions, saving the 'fully-correct' answers for themselves. That way, if there was a curve they would benefit totally from the 'mob' and still come out ahead of it.

SEMW 14y ago
Surely one of the main points of the exercise was that, precisely because there wasn't a curve, whole-class cooperation can result in everyone in the class getting a higher score (20% higher) than they usually would. Grading on a curve wouldn't be the same game just 'more interesting', it'd be a completely different game, and one that's lost most of the "To win at some games, cooperation is better than competition" aspect.

songzme 14y ago

It should be fair to note that experiments like this can only work in small groups. In much larger class sizes chaos will ensue. It will also be an interesting experiment to take a large group of students, split them into groups, and then give out exams to each group. By adding a factor of competitiveness test scores might even go higher. Competition is everywhere in nature.

OnyeaboAduba 14y ago
I spent some of my highschool years in a differnt country in boarding school and grades and class rank where constantly posted for everyone to see It brought out a real competitive spirit in me and in the whole class and maybe a little shame for the the people at the bottom of the list . I thought It was a very effective tool in keeping the class as whole engaged but that type of thing would never be allowed in the US for multitude of reasons.
- songzme 14y ago
  The reason that class ranking is not prevalent in US is because of publicly shaming the students who are not doing as well as the other. This can be mitigated if students are working in groups, and group scores are posted instead of individual scores. This way, a group who is scoring low can help and inspire each other to do well and share the pain (shame)
  - rdouble 14y ago
    In American public schools, class ranking leads to public shaming of the students who are doing well...
    - aspensmonster 14y ago
      Depends on the school. At the public high school I attended --several years on various top 100 lists and top 10 in the state-- class ranking was most definitely a shaming of the "weird" minority of kids who were taking "normal" classes. These normal classes were on a completely different scoring tier that ensured the students taking them could not even break the 50th percentile. And coming from Texas, where getting into the top 10 percent of your graduating class is an automatic ticket to any state university, the competition was most definitely fierce.
- mpyne 14y ago
  That's what we do when training the Navy nukes here in the U.S.

guimarin 14y ago

I am constantly surprised that we have a 'moral' aversion to cheating. IMO school teaches two strategies for success. 1.) work hard within the rules and 2.) cheat but don't get caught. Each is equally valid in society, even without a 'moral' component.

icegreentea 14y ago
The 'moral' aversion to cheating I believe comes out the very game theory and evolutionary behavior that this post talks about. When society sets up rules and systems, there is a maximum amount of cheating/parasitism that it can withstand before the system breaks down, and everyone (at least the majority of the non-cheaters) loses out (assuming a 'good' system). I think we all know/feel this at a very deep level, and understand that if we want to set up and maintain cooperative systems, cheating and parasitism must be kept to a sufficiently low level. Cheaters and 'goodies' all know this. That our western civilization morality tends to align with the interests of large-scale cooperative systems is probably no accident. I limit this to western cause I have no clue about any other cultures and I don't want to make an ass of myself. Of course, this in itself says nothing about the goodness of the system. Cheating a bad system is always an interesting thought experiment. Breaking bad systems and replacing with a new, better one often requires a great degree of 'cheating'. All sorts of interesting things here.
- guimarin 14y ago
  I think this is a complex and naunced issue. I have a number of thoughts on your arguments. Society doesn't set up rules and systems, they seem to emerge as a consequence of beneficial cooperative behavior towards a 'goal'. When it's no longer beneficial to cooperate towards a goal, over say other strategies like cheating, the rules are overlooked, passed, or changed as quickly as they emerged. The prevalence of War as a way to 'rebalance' resources comes to mind immediately, but so too does the reality that only 2% of people pay use taxes. To your point about parasitism. Human societies are always in a state of change with respect to sociological and technological concerns, and the success of a society in achieving its 'goals' seems largely dependent on how well it adapts to this change. The parasitism you've described seems to be more a reality of economics than society in general, and though your point seems valid on its face, You wouldn't say that the success companies like Uber, which are cheating parasites by your definition, is really a net negative for the society. to your point about morality. I believe morality is really orthogonal to this issue and should be considered from different viewpoints in different contexts because humans can hold contradictory points of view and still remain 'functioning'. History holds many examples of individuals who challenge what is 'moral' in a society through cheating, and they are sometimes lauded, sometimes not. I think of the narrative surrounding Rosa Parks on one side ( in support of cheating ) and the story of Cincinnatus ( in support of not cheating ) when considering morality in the context of the cooperation cheating dichotomy. and you're right, this is a very interesting topic, definitely outside the scope of this small classroom experiment.

auctiontheory 14y ago

If you enjoy this kind of thinking, you'll love Dan Ariely's Behavioral Economics class when it next comes to Coursera.

SilasX 14y ago
Because we'll enjoy cheating on it so much more, or something?

cm2012 14y ago

It's not cheating if everyone is following the same rules. Cheating is almost by definition an unfair advantage, when this was pretty close to a level playing field.

tkahn6 14y ago
And how about the students in different sections not taught by this teacher who have a closed-book no-collaboration final exam?

deleted 14y ago

[deleted]

crosvenir 14y ago
I was thinking that what the students did during the test here was what my classmates and I did for almost every homework assignment. We would work through the engineering problems (sometimes individually/sometimes as a group), go over them, ask/answer questions, and move on. I learned so much doing this (mostly that there is more than one way to perform analysis to arrive at an answer and that engineers love looking up the answer in the back of the book and working backwards). I wonder if this would be more aligned with how the grading system works as a method by which periodic quizzes are given. We had lone wolves/busy students who couldn't participate, but I think everyone would have benefited from this as a class group activity.

btilly 14y ago

I would be curious what the students thought of the process. I am sure that the essay was good. I am sure that some learned a lot. But how many just road coattails and didn't learn much other than how to copy? In a game like this where everyone stands to gain, and nobody loses, the incentives are to let everyone participate because the cost of riding is free. How many took advantage of that? Also on a task like this, individual personalities of key figures matter a lot. My guess is that just a few individuals were responsible for a lot of the dynamics. The whole exercise would have been more interesting - and more competitive - if he had started with a grading scale at the start, same rules, but said that he would rank answers against each other, with multi-way ties being ranked at the median of the group. (So if 5 people tie for 3rd, they will all be scored as 5th.) This would push cooperation/competition to a new high.

CurtMonash 14y ago
That's a good approach for a generic class on game theory. It's not so good for the specific case of animal behavior, in which the whole herd will indeed often prosper or suffer at once.

Siecje 14y ago

You should have had more questions to have more strategy. That was a pretty easy test... You also gave them an easy way out if it was too hard...

keerthiko 14y ago

I saw this on a Naruto episode first (http://www.hulu.com/watch/36407#i0,p20,s1,d0 http://www.hulu.com/watch/36407#i0,p20,s1,d0), about 6 months before I started college. For those who don't want to watch the episode, it's a written exam ninja trainees (genin) need to take to be promoted to ninja squad leader (chuunin). Any team (of 3) that has a single member fail will be failed as a group. If any ninja gets caught cheating they will be given a warning, and after three warnings they will be failed. SPOILER (no really, the episode is great in this context even if you don't like/watch anime much) The intelligent (but not book-smart) notice the leniency of the anti-cheating-system and the carefully worded instructions to "not get caught", and then find ways to obtain answers from "sources", qualified chuunin who were intentionally planted in the exam hall who knew the answers, or to collaborate with teammates. End plot summary I think it's important to teach kids not just the answers to problems posed, but also ways to find them. If it's too easy for them to find answers without rigid rules, it probably means that he problems we are posing them are not hard or thought-provoking enough in the first place. Edited for a missed phrase

nsxwolf 14y ago
Interestingly this is the only episode of Naruto I've ever seen. I remember being quite impressed at how cerebral it was, as I was expecting to see a bunch of ninjas running around slicing off each other's heads.

lt 14y ago

Instead, they were changing their goal in the Education Game from “Get a higher grade than my classmates” to “Get to the best answer.” The goal shouldn't be “Get a higher grade than my classmates”, it should be “Get the highest grade possible”. And the goal for the teacher should be “Translate each student knowledge data into a numeric value”, which is somewhat orthogonal to the experiment.

aspensmonster 14y ago
"Translate each student knowledge data into a numeric value" Which isn't at all easy. Grades certainly have some correlation to knowledge (certainly not "one" though), but in a perfect world we'd be putting everyone through oral examination and putting their projects through peer review on a constant basis. This is expensive though, in time and money and man power.
willismichael 14y ago
Shouldn't the goal be to "Learn the most possible"? I know, it's not quantifiable, so maybe it doesn't qualify as a goal at all, but it concerns me when people act like the purpose of education is to get grades.
- cmstoken 14y ago
  You'd be surprised to learn how many college students goals aren't actually to "Learn the most possible."
  - tkahn6 14y ago
    I'd be more surprised if even 1% of college students had to goal to learn as much as possible as opposed to get the highest grades possible.
  - benblodgett 14y ago
    Schools emphasize the numerical values that represent the 'amount learned', not the actual acquisition of knowledge. Any college student with the sole goal of learning how to learn & think unfortunately doesn't survive the education system.

Millennium 14y ago

So really, this was was a class project involving a scenario that superficially resembled cheating on an exam, rather than actually cheating on an exam.

geebee 14y ago

I loved this approach until I read the exam question. After that, I only liked it. It would have been more interesting to see an exam with question like "is the population of x in hardy-weinberg equilibrium", where x is something students should know something (but not everything) about if they've been paying attention. It would be possible, but not easy, for a very well prepared student to do the research and figure it out (or at least write an intelligent answer) in an hour. Some bright students who haven't studied much probably would be able to research quickly and come up with a good answer, especially if they split up the task. Some people will clearly be vastly more valuable to teams than others. In short, I think the professor here picked a question that tilts too heavily toward collaboration. If this approach is really going to work, you need to induce some reluctance among the very well prepared, who may realize that their friends who spent more time preparing for the non-collaborative physics exam will be getting a relatively free ride. I guess I just don't think this quite brought out the nastiness that would truly arise in a state of nature - something of course I understand he'd rather avoid!

forgotAgain 14y ago

So is the article title another experiment in Behavioral Ecology?

droithomme 14y ago

1. This obviously wasn't cheating since it was within the prescribed parameters for the exam. Some exams are open book, some are open note, some are even take home with no limits. 2. The highest score and thus most insightful and highest quality work, according to the professor's own evaluation, came not from collaboration but from one of the 3 lone wolves.

pixelcort 14y ago
The average was higher than that of prior classes. Also, it is implied that the lone wolves may have had access to some of the work the mob was producing, enabling one of them to get ahead by simply correcting a few mistakes without giving them back to the mob.
- cschep 14y ago
  This immediately reminded me of open source software. Interesting.
- scott_s 14y ago
  She explicitly stated so: Although the Wolves listened and contributed to discussions, they preferred their individual variants over the Mob’s joint answer.
- probably_wrong 14y ago
  I don't think they necessarily "corrected a few mistakes without giving them back to the mob". There is also a chance that they indeed proposed their answers to the mob, but they didn't like them and rejected them. It wouldn't be the first time that a great idea is shot down because it's too far from everyone's comfort zone. But I'm just guessing.
steveklabnik 14y ago
1: From TFA: > Although by conventional test-taking rules, the students were cheating, they actually weren’t in this case. 2: The mob still contributed to the work of the lone wolves, it was absolutely still 'collaboration': > Although the Wolves listened and contributed to discussions, they preferred their individual variants over the Mob’s joint answer.

tenpoundhammer 14y ago

The most accurate measurement of what a person has learned over a period of time is their ability at which they are generally able to discuss a subject. The best exam would be a personal exam in which the teacher orally administers a series of generalized questions and explorers the subject matter with each student to understand their actual level of understanding. While it would take a long time to administer these tests for every student it would provide a lot of value for many subjects, while a written test would still be important to determine particular skill sets for certain subjects like math. Any average student can jam a notecard full of likely facts or cram facts into their head the night before an exam, without learning much. Likewise working as a team to generate the best answer can allow people to receive a high grade without knowing much. There is also the fact that even a well read student with a good understanding of the subject matter can a fail a test because they don't have a the particular points asked on the test.

lifesavers 14y ago
I like this approach as well (in theory) and only have some "anecdotal" experience - if you can call it that. My family originated from East Europe and as such most of my family elders have an old school Soviet experience in education. Consequently they have little knowledge and no experience when it comes to modern-day Western education infrastructure. For instance, my parents were genuinely surprised to hear that I wrote written exams and submitted them only to be reviewed at face value - which is the experience of nearly all undergrads I expect. Their experience was that, even in a "straight-forward" subject like Mathematics, they had to first submit a written response to a problem set in an examination type settings - and then upon completion they would have to explain their solution and approach one-on-one with a faculty member who would only then submit a grade. Obviously this was done to weed out the cheaters who couldn't explain why they had to correct answers to all the questions. But the result can't be ignored that one-on-one oral examinations, while not exactly pragmatic, are very effective!

nollidge 14y ago

And yet again, here's 25 comments from the Hacker News Pedant Squad to remind you that it's not REALLY cheating if the teacher lets you, in case you only recently disembarked from an 18-wheeler full of turnips.

calibraxis 14y ago
What's unfortunate about such comments is that such departures from the norm actually are considered bad ("cheating") according to the ideology of mainstream education. The article's author wasn't the only teacher with the bright idea to have collaborative tests, where he was willing to answer whatever the students asked. I know of at least one teacher who was forced to stop. Education could look very different, and more diverse. Universities have certain functions in society. If they don't fulfill those functions, they won't get government and corporate funding. Even Salman Khan explains that one important function of schools is to subjugate, fashioning people into obedient cogs. (In his "The One World School House", in the chapter "The Prussian Model".)

dice 14y ago

I had a professor who used a similar concept of fostering active discussion and debate within the classroom, except he did it for every lecture. He would present a concept, then ask us a series of questions about it. Students would first think on the question and vote on their own, then the question was opened to discussion and debate. After the topic had been hashed out we would re-vote. Often times the difference between the first and second votes were significant, sometimes resolving a split opinion and other times the majority would change their minds. It was a lot of fun, and easily the best and most informative experience of my educational career.

nostrademons 14y ago
Most of my high school education was like this. It's fairly common in "alternative" schools like Montessori, Waldorf, Essential, or Sudbury schools.

avalaunch 14y ago

I wonder if this was being graded on a curve. If so, an interesting strategy would have been to sabotage the group effort while secretly preparing a "lone wolf" answer. The saboteur could continually steer the conversation in the wrong direction and possibly even alter the group answer to something absurd before it was turned in. Or if he wanted to concentrate on his own work, he could bring some headphones to tune out the horrendous noise coming from outside the classroom where his friends happen to be practicing for their upcoming rock gig.

sriram_sun 14y ago

Here is what I would like the professor to do. Follow up with a "normal" test and see how the students perform. If the class average was greater that prior year performances, the co-operation actually helped in drilling the concepts in.

carlob 14y ago

When I was in high school my classmates and I found a document from a previous year that amounted to a superset of all the questions my philosophy teacher was going to put in test. Since some of the questions were pretty hard we ended up studying really hard together to try and crack them before tests. Though what we did was somehow against the rules I feel I learned a lot more by studying the questions collaboratively than by studying on my own. I later found this form of collaboration extremely rewarding and effective during university. We spent many afternoon trying to crack hard problem sets and explaining them to one another.

tassl 14y ago
It has always puzzled me why a professor would be willing to repeat the answers from previous years. Some account that people that work "harder" would be solving previous exams, but this is given that the exams are available to all the class (which is a prerequisite for this argument to be true). I've found myself in classes where a group had exams, problem sets and other information from previous years (and solved) and they hid and refused to share those.
- Periodic 14y ago
  I've found that in my grad-school years that most professors released a set of sample exams, usually with answers. For some this would be a made-up exam. For most it was a previous year's exam. For others it was all the exams presented to the class for the last ten years with answers. It was quite productive to go over all these exams as a group and strive to understand the questions, and particularly to understand the patterns, these were the concepts that lay behind the questions and once you had those you could answer many questions. These days it isn't hard to find example problems and exams on the internet for your class or similar ones.
- Steuard 14y ago
  It is hard to come up with entirely new problems for every exam every year. That doesn't mean that doing so isn't important, but it really is time consuming and a whole lot of work. It's also painfully easy to think that you've changed things enough for the question to be "new" compared with examples from class or homework or previous years' tests, only for some student to recognize it enough to regurgitate some memorized solution rather than actually thinking. So while I'd never say that repeating questions is a good thing, I can't be too hard on folks who do (intentionally or not).

temp453463343 14y ago

This is a completely idiotic way of testing people. I could have come into class not knowing anything, waited till the end of the test and then offered 100 dollars to anyone to be allowed to copy their whole test. And I would have passed the class? It kinda reminds me of CS-whores. The one or two CS girls in each graduation class that regularly used sex-appeal/flirtation to get CS guys to do their homework (very successfully too!). At some point your no longer testing people's knowledge and creative thinking, but their social skills and ability to manipulate others (and maybe their breast size/size of their 6 pack). I feel like the professor treated this like a sick little game and enjoyed himself while doing it. At the end of the day you are competing against your peers either through a curve, or through your raw grades because you're all going into the same fields and are competing for the same jobs with your GPAs. Making a system where cut-throat behavior is encouraged and putting down your pays off is really sad.

MJR 14y ago
I completely disagree. I found this to be an excellent example of real world training. I worked through college in IT and from that job on, working in the real world was exactly like this. We worked on projects with teams of people. We looked for the best possible solutions to every problem we faced. Even when working alone on something, you had your peers as resources. You weren't locked in a closet coding on a computer, offline, without access to books or other tools. That's what test taking was like for me. Some people, like me, are experiential learners. Others find their best thoughts through discussion and the evolution of ideas. I would have loved to have taken many classes like this. Fortunately, that's what the real world has been like for me - being a member or leading a team to produce the best product we can, either as a team, department or company. As you called it my "pay off" is education. I found mine through methods like this. Not through memorizing facts and methods just to pass a test.
- temp453463343 14y ago
  While I agree that the process is more similar to the "real world" - the fundamental problem is that the grading is not. When you work as a team towards some goal whoever is evaluating you is doing so based on your contribution to the team. Here the fundamental scoring method is still the test, so an individual's motivation is to do better for himself; the teams success is secondary (and if the test is curved it may be non existent). Free riders aren't punished or penalized and all sorts of outside influences come in where people start trading on money/sex/favors/friendships/etc. Arguably those outside influences exist in the real world, but why create a system that explicitly has no mechanisms to counteract them? (ie. the prof says you can do anything that is legal and doesn't try to catch "cheaters")

CurtMonash 14y ago

I'm a big fan of unusual approaches to testing, good as I was at the conventional kind. In high school astronomy, we could bring in a few pages of notes. I once wrote out the notes in great detail, FORGOT THEM AT HOME, and got the highest score on the test. In college I learned a lot studying for tests, both multiple choice (geology) and essay (history), where the questions were highly predictable. I love the test Hilary Putnam announced the first day of every class: Part 1: Write a suitable question for this class. Part 2: Answer it. You will be graded on both parts. (Note: The recursive answer of giving him his own question back turns out to be infinitely long.) I once gave an open book test, saying exam questions would be chosen only from problems I'd assigned for homework. Students still did badly. (Ouch.)

lnanek2 14y ago

Too bad no one tried to sabotage the consensus answers. You could get responsibility for writing one of the group answers, write it wrong, then turn in your own test with it correct. Inject false data into discussions, etc.. It isn't really game theory without some interesting gains and loses due to cooperation or deception.

Brashman 14y ago

I like this idea, except that I'd be worried about money entering into the scheme. In a real world setting, it's not unreasonable to pay someone to do the work for you, but in a school/learning setting, I dislike the possibility of people paying their way out of the work.

f4stjack 14y ago

Actually this is not cheating. Cheating means copying somebody's content with or without his permission in an exam where this kind of information sharing is forbidden. In here, the lecturer specifies that information sharing is not forbidden. Therefore technically this is not "cheating". But it is wonderful to see when you remove the outside limiting factors and let the phenomena occurs in its own volition. Probably this will be one of the courses the participant students will never forget, instead of courses where you "memorize" everything until the last exam. Kudos to Mr. Nonacs.

acjohnson55 14y ago

A collaborative assignment is a really solid education tool. What's missing is an explanation of why the author thinks this led to an objective evaluation of what individual students learned before test day (if he does indeed think that). That is the point of an exam. Imagine if med students could collaborate on their boards. It would pretty well invalidate the purpose.