4 ms·

https://www.voiceofsandiego.org/topics/education/college-students-are-learning-hard-lessons-about-anti-cheating-software/ https://www.voiceofsandiego.org/topics

by pdkl95 5y ago

https://www.voiceofsandiego.org/topics/education/college-students-are-learning-hard-lessons-about-anti-cheating-software/ https://www.voiceofsandiego.org/topics/education/college-stu... > At the self-described “heart” of the company’s monitoring software is Monitor AI, a “powerful artificial intelligence engine” that collects facial detection data [...] to identify “patterns and anomalies associated with cheating.” > "... people who have some sort of facial disfigurement have special challenges; they might get flagged because their face has an unexpected geometry.” This is literally phrenology with a bunch of linear algebra instead of calipers.

unishark 5y ago
My guess is they're trying to classify behaviors like glancing off in some direction repeatedly to look at a cheat sheet or a phone, the same things human proctors watch for. Presumably the system can fail to accurately estimate the gaze direction for some people because they are too different from the design assumptions made. i don't really think that's the same as phrenology. It's more akin to BMI being a poor estimate of obesity for bodybuilders.
ineedasername 5y ago
Wow that's awful. There might be some very general common characteristics of body language and facial movements the very roughly correlate to certain general situations. (Panic comes to mind) but those are still going to vary incredibly widely over the population based on so many factors, not the least of which are culture, peer groups, and general personal quirks. A person with a history of panic attacks, for example, may have learned to cope well enough not to completely lose it in a meeting at work. The list of other examples would be endless. And how do you get a reliable data set for this to begin with? Most schools allow for a formal appeal or inquiry process precisely because anything but very blatant cheating can be a murky area. Even if you had videos and a tagged set of the outcomes of inquiries like that, you still wouldn't have intercoder ratings for reliability and therefore an no verified data set. I know some methods that can be used against untagged data, but at the very least you actually have to know if there was cheating or not. And talk about black box-- this isn't even something that human review of specific incidents could validate. If a human couldn't reliably come to the correct conclusion based on available data, an AI using pattern matching with a questionable data set sure won't. When the article mentioned companies using it I figured it was just early stages of testing it. Or used for some low stakes marketing/advertisement targeting. That would still be bad, but this is world's worse. Life ruining worse.

by pdkl95 5y ago

unishark 5y ago

My guess is they're trying to classify behaviors like glancing off in some direction repeatedly to look at a cheat sheet or a phone, the same things human proctors watch for. Presumably the system can fail to accurately estimate the gaze direction for some people because they are too different from the design assumptions made. i don't really think that's the same as phrenology. It's more akin to BMI being a poor estimate of obesity for bodybuilders.

ineedasername 5y ago

Wow that's awful. There might be some very general common characteristics of body language and facial movements the very roughly correlate to certain general situations. (Panic comes to mind) but those are still going to vary incredibly widely over the population based on so many factors, not the least of which are culture, peer groups, and general personal quirks. A person with a history of panic attacks, for example, may have learned to cope well enough not to completely lose it in a meeting at work. The list of other examples would be endless. And how do you get a reliable data set for this to begin with? Most schools allow for a formal appeal or inquiry process precisely because anything but very blatant cheating can be a murky area. Even if you had videos and a tagged set of the outcomes of inquiries like that, you still wouldn't have intercoder ratings for reliability and therefore an no verified data set. I know some methods that can be used against untagged data, but at the very least you actually have to know if there was cheating or not. And talk about black box-- this isn't even something that human review of specific incidents could validate. If a human couldn't reliably come to the correct conclusion based on available data, an AI using pattern matching with a questionable data set sure won't. When the article mentioned companies using it I figured it was just early stages of testing it. Or used for some low stakes marketing/advertisement targeting. That would still be bad, but this is world's worse. Life ruining worse.