4 ms·
> Software has no emotional biases, either So what sorts of biases does it have? Remember the ol' adage about programmers figuring out any sort of metric, and
by pavel_lishin 15y ago
> Software has no emotional biases, either
So what sorts of biases does it have? Remember the ol' adage about programmers figuring out any sort of metric, and exploiting it? I wonder if a group of students could find a way to game the grading system.
- eru 15y agoI know of some automatic grading system they tried for psychology students. It used some clever techniques to see how far a student's free-form answer was away from the model answer. The system gave essentially the same grades as manual grading. But, the automated grading system did not take the order of words into account at all. If that was more than a test, it would have been easily gamed, or at least incited responses that having nothing to do with a human readable text. You would still have to learn the `passwords' for each question.
- _delirium 15y agoThe essay-grading software I've seen described in enough detail to figure out how it works has a heavy bias towards enforcing specific structural rules, while mostly missing "quality of argument" type questions. So a student writing to exploit its bias would want to worry more about formal structure: essay organization, correct grammar, use of certain "must-have" keywords, etc.; but worry less about things like avoiding logical fallacies in their argument, because the software won't notice those.
- pavel_lishin 15y agoIt would be amusing if true AI rose out of our attempts to catch cheating students.
- eru 15y agoThat would be fun, though I believe that the problem of catching students doesn't provide enough data.
- marklabedz 15y agoThis makes sense as it is a lot like how one is taught to answer essay questions on standardized tests.