3 ms·

Professor with background in game design abandons grades for experience points

39 points by mcantelon 17y ago

naner 17y ago
This is dumb. You're going to end up with students who are great at farmville and terrible at anything remotely difficult. In other words, he is rewarding task completion and ignoring self-motivation, discipline, creativity, etc. I've seen this same theme repeated many times. "Foods engineered to be addictively tasty" or "Reality TV engineered to hook viewers", etc. You are manipulating people to obtain a specific result (sell more food cheaply, get more viewers, get higher grades), but not necessarily the one that would be most beneficial to the individual (healthy food, rewarding storytelling, mature adults capable of tackling complex problems).
- psyklic 17y ago
  It all depends on how it is implemented. After all, you could take regular grades and assign points to each grade -- and the two systems would be equivalent. The big question for me though is exactly as you point out. Engineering projects, for example, are really only valuable if they work. And it's hard to measure the progress in-between non-working and working if you don't observe the team carefully, except of course by "time spent." No matter which system is used for grading, it won't be perfect -- it will cater toward some students and not toward others. Since most students are only exposed to one dominant way, I am all for experiments such as this.
  - naner 17y ago
    > It all depends on how it is implemented. After all, you could take regular grades and assign points to each grade -- and the two systems would be equivalent. This is absolutely true. That is why people always claim you only get out what you put into schooling. You have to do more than just make the grade. This article makes it sound like he really wants to ramp things up, though. And I believe he's approaching it the wrong way.
- bd 17y ago
  Well, considering this is a game design class, it's actually quite clever. Also, details do matter: instead of Farmville it could be more like Dungeons & Dragons where professor is your dungeon master.
- thwarted 17y ago
  It would be interesting to have it based on awards/achievements, rather than experience points as experience points are just a renaming of the grade number that maps to a letter. At the beginning of the class, there is a huge number of projects or tasks, and they are combined in different ways to "unlock" achievements. Actual grades could be based on how many of different difficulties you have at the end of the class. You can't unlock more difficult achievements without unlocking other ones first (or in tandem). Students get to choose exactly which path they want to take through the assignments to get the right kind/enough achievements to pass. Consider a compiler design class, where the tasks are things like "implement a RPN calculator", "write an LALR parser", "parser that acknowledges operator precedence", "code generator for a CISC machine", "code generator for a RISC machine", "implement space-saving optimizations", "implement time-saving optimizations", "execute code by walking the AST", "generate intermediate bytecode", "parser generator using Yacc/Bison", "parser generator using lemon". These are kind of high level, they'd need to be somewhat more discrete in the ramp-up phase at the start of the class.
  - barrkel 17y ago
    I don't think XP is a simple mapping to grade numbers - see my comment elsewhere in this post. Doing less than perfectly in any single test hurts your potential grade permanently, but not necessarily so with XP.
    - thwarted 17y ago
      I'm not saying it necessarily is a simple mapping, but I think it's easier for people to misunderstand the relationship since both are effectively "point based". Can an XP based system be any different than those teachers who say upfront that each quiz or test or question or whathaveyou is worth X number of points? If you need a certain number of XP to pass the class, and you don't gain enough at a critical juncture in the course, do you get a chance to make it up? Doing away with the time component of class based instruction, where exams are administered at given, set times, and you just need to prove that you know the material come the last day of class, would be an XP based system. I've had teachers do that before. Really, expecting students to make specific progress at a specific rate over the semester, as codified in regular exams, is a way to avoid having to deal with teaching different things at different times to a group of students, in the name of efficiency.
- derefr 17y ago
  It's important to point out that there are still certain standards you have to adhere to in order to earn the achievements/XP. In fact I wouldn't be surprised if, in order to gain the experience for completing an assignment, you actually have to complete it to a 100% rating (which, of course, makes sense in programming assignments, but not as much in English papers.) No bugs, no errors, no unclear thinking—you have to demonstrate, through that assignment and that assignment alone, that you understand every line and every word that went into its construction, and all the topics it covered. There are many students who scrape by with a 60-70% grade in classes—this means that, after completing the course, there may be 40% of its content that they have no idea about, yet still managed to pass, and therefore are considered to know for having "completed" the course. This would be impossible an experience-based system—you'd do each assignment, over and over, until you got it right, and then you'd be done that topic. You wouldn't have to later do a test, or an assignment covering the same topics, because you had already proven you were capable of doing everything required. It's almost as if right now we're passing or failing kids using fuzzy logic like a spam filter, when there's a simple set of discrete rules (for some subjects) that would give correct judgements 100% of the time, instead of 80%.
- barrkel 17y ago
  Not all manipulation is evil - on the contrary, all sorts of everyday manipulation is entirely benign and positive in nature. See e.g. http://www.predictablyirrational.com/?p=235 http://www.predictablyirrational.com/?p=235
henning 17y ago
"LFM Sheldon final PST with GPA, transcript checks at library PST"
scott_s 17y ago
This is no different than how many (most?) courses are structured. Many teachers assign points to various projects, homeworks, test and quizzes. The overall grade in the course then depends on total points accumulation (in relation to other students if there is a curve).
- barrkel 17y ago
  Only if you look at things at a high enough level of abstraction - at which point all sorts of different things start seeming similar. I would expect course scoring structured along game design lines to look significantly different to current course scoring structures, which are usually based around projects and tests at pretty widely spaced intervals. But games don't work that way: you get positive feedback almost immediately after successfully doing something. The feedback is in smaller but more frequent packets. Lee Sheldon frames it as XP. But XP doesn't work like test grading. Every time you do less than perfectly on a test, you have permanently lost the potential to make that up. If a project is worth 20%, and you score 15/20, the best you can do out of the whole course is 95%. But with XP, you could do something extra to make up the gap - the equivalent of XP grinding. Perhaps an extra project, an extra presentation, etc.
glhaynes 17y ago
Instead of "F", your failing report says "You have died."
- psyklic 17y ago
  "You have died of dysentery. Play again?"
johnwatson11218 17y ago
One other thing I would like to see from games is the ability to go very fast. I wish I could find a statistics course that has the feel of working through a long campaign in a video game. I really feel like certain subjects could have that repetition concept blended with the idea of getting somewhere. For languages there are flashcards but with computers we could make that so much more general. For me the main thing is the speed and the idea that I have an extra 15 minutes here and I want to sit down and be so immersed in my subject of study that I actually learn something.
zitterbewegung 17y ago
I am graded on a points system already. I earn points and they are scaled due to how well I do on a given quiz, test or project. These points are then translated to a letter grade and curved. So there really isn't much of a difference.

39 points by mcantelon 17y ago

naner 17y ago

This is dumb. You're going to end up with students who are great at farmville and terrible at anything remotely difficult. In other words, he is rewarding task completion and ignoring self-motivation, discipline, creativity, etc. I've seen this same theme repeated many times. "Foods engineered to be addictively tasty" or "Reality TV engineered to hook viewers", etc. You are manipulating people to obtain a specific result (sell more food cheaply, get more viewers, get higher grades), but not necessarily the one that would be most beneficial to the individual (healthy food, rewarding storytelling, mature adults capable of tackling complex problems).

psyklic 17y ago
It all depends on how it is implemented. After all, you could take regular grades and assign points to each grade -- and the two systems would be equivalent. The big question for me though is exactly as you point out. Engineering projects, for example, are really only valuable if they work. And it's hard to measure the progress in-between non-working and working if you don't observe the team carefully, except of course by "time spent." No matter which system is used for grading, it won't be perfect -- it will cater toward some students and not toward others. Since most students are only exposed to one dominant way, I am all for experiments such as this.
- naner 17y ago
  > It all depends on how it is implemented. After all, you could take regular grades and assign points to each grade -- and the two systems would be equivalent. This is absolutely true. That is why people always claim you only get out what you put into schooling. You have to do more than just make the grade. This article makes it sound like he really wants to ramp things up, though. And I believe he's approaching it the wrong way.
bd 17y ago
Well, considering this is a game design class, it's actually quite clever. Also, details do matter: instead of Farmville it could be more like Dungeons & Dragons where professor is your dungeon master.
thwarted 17y ago
It would be interesting to have it based on awards/achievements, rather than experience points as experience points are just a renaming of the grade number that maps to a letter. At the beginning of the class, there is a huge number of projects or tasks, and they are combined in different ways to "unlock" achievements. Actual grades could be based on how many of different difficulties you have at the end of the class. You can't unlock more difficult achievements without unlocking other ones first (or in tandem). Students get to choose exactly which path they want to take through the assignments to get the right kind/enough achievements to pass. Consider a compiler design class, where the tasks are things like "implement a RPN calculator", "write an LALR parser", "parser that acknowledges operator precedence", "code generator for a CISC machine", "code generator for a RISC machine", "implement space-saving optimizations", "implement time-saving optimizations", "execute code by walking the AST", "generate intermediate bytecode", "parser generator using Yacc/Bison", "parser generator using lemon". These are kind of high level, they'd need to be somewhat more discrete in the ramp-up phase at the start of the class.
- barrkel 17y ago
  I don't think XP is a simple mapping to grade numbers - see my comment elsewhere in this post. Doing less than perfectly in any single test hurts your potential grade permanently, but not necessarily so with XP.
  - thwarted 17y ago
    I'm not saying it necessarily is a simple mapping, but I think it's easier for people to misunderstand the relationship since both are effectively "point based". Can an XP based system be any different than those teachers who say upfront that each quiz or test or question or whathaveyou is worth X number of points? If you need a certain number of XP to pass the class, and you don't gain enough at a critical juncture in the course, do you get a chance to make it up? Doing away with the time component of class based instruction, where exams are administered at given, set times, and you just need to prove that you know the material come the last day of class, would be an XP based system. I've had teachers do that before. Really, expecting students to make specific progress at a specific rate over the semester, as codified in regular exams, is a way to avoid having to deal with teaching different things at different times to a group of students, in the name of efficiency.
derefr 17y ago
It's important to point out that there are still certain standards you have to adhere to in order to earn the achievements/XP. In fact I wouldn't be surprised if, in order to gain the experience for completing an assignment, you actually have to complete it to a 100% rating (which, of course, makes sense in programming assignments, but not as much in English papers.) No bugs, no errors, no unclear thinking—you have to demonstrate, through that assignment and that assignment alone, that you understand every line and every word that went into its construction, and all the topics it covered. There are many students who scrape by with a 60-70% grade in classes—this means that, after completing the course, there may be 40% of its content that they have no idea about, yet still managed to pass, and therefore are considered to know for having "completed" the course. This would be impossible an experience-based system—you'd do each assignment, over and over, until you got it right, and then you'd be done that topic. You wouldn't have to later do a test, or an assignment covering the same topics, because you had already proven you were capable of doing everything required. It's almost as if right now we're passing or failing kids using fuzzy logic like a spam filter, when there's a simple set of discrete rules (for some subjects) that would give correct judgements 100% of the time, instead of 80%.
barrkel 17y ago
Not all manipulation is evil - on the contrary, all sorts of everyday manipulation is entirely benign and positive in nature. See e.g. http://www.predictablyirrational.com/?p=235 http://www.predictablyirrational.com/?p=235

henning 17y ago

"LFM Sheldon final PST with GPA, transcript checks at library PST"

scott_s 17y ago

This is no different than how many (most?) courses are structured. Many teachers assign points to various projects, homeworks, test and quizzes. The overall grade in the course then depends on total points accumulation (in relation to other students if there is a curve).

barrkel 17y ago
Only if you look at things at a high enough level of abstraction - at which point all sorts of different things start seeming similar. I would expect course scoring structured along game design lines to look significantly different to current course scoring structures, which are usually based around projects and tests at pretty widely spaced intervals. But games don't work that way: you get positive feedback almost immediately after successfully doing something. The feedback is in smaller but more frequent packets. Lee Sheldon frames it as XP. But XP doesn't work like test grading. Every time you do less than perfectly on a test, you have permanently lost the potential to make that up. If a project is worth 20%, and you score 15/20, the best you can do out of the whole course is 95%. But with XP, you could do something extra to make up the gap - the equivalent of XP grinding. Perhaps an extra project, an extra presentation, etc.

glhaynes 17y ago

Instead of "F", your failing report says "You have died."

psyklic 17y ago
"You have died of dysentery. Play again?"

johnwatson11218 17y ago

One other thing I would like to see from games is the ability to go very fast. I wish I could find a statistics course that has the feel of working through a long campaign in a video game. I really feel like certain subjects could have that repetition concept blended with the idea of getting somewhere. For languages there are flashcards but with computers we could make that so much more general. For me the main thing is the speed and the idea that I have an extra 15 minutes here and I want to sit down and be so immersed in my subject of study that I actually learn something.

zitterbewegung 17y ago

I am graded on a points system already. I earn points and they are scaled due to how well I do on a given quiz, test or project. These points are then translated to a letter grade and curved. So there really isn't much of a difference.