5 ms·

Accused of Cheating by an Algorithm, and a Professor She Had Never Met

15 points by notmysql_ 4y ago

poulpy123 4y ago
Everything in the article is insane, and not only being flagged as a cheater because she looked down regularly
irl_chad 4y ago
Student 94871! Drink your verification can of Mtn Dew (tm) to begin the exam!
charcircuit 4y ago
Clickbait. She wasn't accused by an algorithm. A human being accused her of cheating.
- xnorswap 4y ago
  It's not clickbait, an algorithm flagged the behaviour as indicative of cheating. Saying that's not an accusation and it's the people in the loop who accuse is just weaselling. People will just do what the algorithms tell them, they don't want to make their own judgements for fear of getting it wrong and treat the computer as gospel.
- piva00 4y ago
  The article touches on that exact point, where it argues that schools buying this product might be taking the word of AI as "the word of God" and just rubber stamping what's been decided by AI. The argument is presented there, you just need to read it...
bakugo 4y ago
- tzmudzin 4y ago
  > light is apparently racist for not reflecting equally off of different skin colors. Excuse me? https://www.theverge.com/2019/1/25/18197137/amazon-rekognition-facial-recognition-bias-race-gender https://www.theverge.com/2019/1/25/18197137/amazon-rekogniti...
- mcphage 4y ago
  > light is apparently racist for not reflecting equally off of different skin colors The light does what the light does, but if your software doesn’t recognize and account for the fact that it doesn’t reflect equally off different skin colors, and instead pretends that it reflects the same off of everyone, and thus classifies it’s own error as offending behavior—then yeah, your software is racist.
mizzao 4y ago
Issues line these aren't out of the question as the baseline for cheating is pretty high, see this recent thread: https://news.ycombinator.com/item?id=31544634 https://news.ycombinator.com/item?id=31544634
yk 4y ago
There was a reddit thread on these anti-cheating software the other day, the threat mostly reminded me that I used to stare at the ceiling during exams. And a bit more abstractly, they are trying to temporarily establish control over a space, so that their ml vodoo can flag suspicious behavior. A poster is actually a nice example, that they have problems with teenagers having a poster of their favorite pop star in their bedroom is an obvious, but there are probably also posters that get identified as a person with a probability of 10^-3 or so and subsequently flagged as another person entered the room during the exam. That situation is not improved by them not controlling the lighting of the room. And of course, a institution paying a company to judge students is a recipe to ignore the students interests. We can assume that everybody involved thinks that minimizing false positives is important, in the abstract, however concretely this software is sold with a nice presentation to administrators and a nice UI for professors, minimizing the hoops students have to jump through, or their privacy, is not an immediate pain point for either.

15 points by notmysql_ 4y ago

poulpy123 4y ago

Everything in the article is insane, and not only being flagged as a cheater because she looked down regularly

irl_chad 4y ago

Student 94871! Drink your verification can of Mtn Dew (tm) to begin the exam!

charcircuit 4y ago

Clickbait. She wasn't accused by an algorithm. A human being accused her of cheating.

xnorswap 4y ago
It's not clickbait, an algorithm flagged the behaviour as indicative of cheating. Saying that's not an accusation and it's the people in the loop who accuse is just weaselling. People will just do what the algorithms tell them, they don't want to make their own judgements for fear of getting it wrong and treat the computer as gospel.
piva00 4y ago
The article touches on that exact point, where it argues that schools buying this product might be taking the word of AI as "the word of God" and just rubber stamping what's been decided by AI. The argument is presented there, you just need to read it...

bakugo 4y ago

tzmudzin 4y ago
> light is apparently racist for not reflecting equally off of different skin colors. Excuse me? https://www.theverge.com/2019/1/25/18197137/amazon-rekognition-facial-recognition-bias-race-gender https://www.theverge.com/2019/1/25/18197137/amazon-rekogniti...
mcphage 4y ago
> light is apparently racist for not reflecting equally off of different skin colors The light does what the light does, but if your software doesn’t recognize and account for the fact that it doesn’t reflect equally off different skin colors, and instead pretends that it reflects the same off of everyone, and thus classifies it’s own error as offending behavior—then yeah, your software is racist.

mizzao 4y ago

Issues line these aren't out of the question as the baseline for cheating is pretty high, see this recent thread: https://news.ycombinator.com/item?id=31544634 https://news.ycombinator.com/item?id=31544634

yk 4y ago

There was a reddit thread on these anti-cheating software the other day, the threat mostly reminded me that I used to stare at the ceiling during exams. And a bit more abstractly, they are trying to temporarily establish control over a space, so that their ml vodoo can flag suspicious behavior. A poster is actually a nice example, that they have problems with teenagers having a poster of their favorite pop star in their bedroom is an obvious, but there are probably also posters that get identified as a person with a probability of 10^-3 or so and subsequently flagged as another person entered the room during the exam. That situation is not improved by them not controlling the lighting of the room. And of course, a institution paying a company to judge students is a recipe to ignore the students interests. We can assume that everybody involved thinks that minimizing false positives is important, in the abstract, however concretely this software is sold with a nice presentation to administrators and a nice UI for professors, minimizing the hoops students have to jump through, or their privacy, is not an immediate pain point for either.