4 ms·

> A federally-funded study found that even the best facial recognition algorithms fail to work on Black and brown people, trans and non-binary people, as well a

> A federally-funded study found that even the best facial recognition algorithms fail to work on Black and brown people, trans and non-binary people, as well as children and women in general. I assume it is black and brown trans and non-binary people. I don't see an algorithm fail because someone identifies as anything. However, why even mention it here, in this context? It makes the people behind this sound ridiculous. Aside from the technical difficulties with image recognition on BPoC which make this sort of tech a non-starter, I think universities and schools need to (finally) find a way to assess people that does not involve anything that can be cheated by having a book open next do the computer. Projects and papers and weekly problem sets to solve are a way better way to assess people.

MattGaiser 5y ago
It isn't so much a book that is a problem. It is divide and conquer by groups of students.
spoonjim 5y ago
Everyone wants a tie-in to the hot button issue of the day. You don’t need to get black trans kids involved to point out all of the flaws in Eproctoring.
- wizzwizz4 5y ago
  To point out enough flaws to disqualify e-proctoring, no. To point out all of them, yes.
fcatalan 5y ago
In our initial trials involving about 5 people the software systematically failed to recognise one woman just because her (pretty standard) hairstyle. It also failed to detect my kids coming up on camera a few times.
- dna_polymerase 5y ago
  Sounds like these solutions are just outright garbage. I wonder if ML/CV will ever work outside of GAFAM.
jarcane 5y ago
Recognition algorithms often make assumptions about the way people "should" be shaped according to their recorded gender or the way the machine attempts to "guess" said gender, and a clash in expectations gets flagged and generates false negatives because things aren't in the "right" places it thinks they should be. For a really simple example if this problem, the TSA body scanner machines flag on trans people constantly. It detects as an "unknown mass" if gendered body parts (ie. breasts/genitals) don't match its binary expectations for what gender is marked on their passport. Often they get it coming and going: a trans femme still marked M will get flagged for having breasts, but if they've changed it to F, they get flagged for their genitals instead. Now you get a full grope patdown from a suspicious TSA officer, because a machine was coded to make binary assumptions about bodies.
- goatxi 5y ago
  It just means that the body scanner is programmed to recognize biological realities (that transwomen are men presenting as women, and transmen are women presenting as men), rather than questionable self-beliefs.
- donkarma 5y ago
  So we put sex on passports? What is the issue?
  - wizzwizz4 5y ago
    > Often they get it coming and going: a trans femme still marked M will get flagged for having breasts, but if they've changed it to F, they get flagged for their genitals instead. An F sex marker isn't going to stop this happening.
  - goatxi 5y ago
    One problem is that in some countries (such as the UK), you can legally change your birth certificate to have it record your supposed 'gender identity' in the sex field instead. Your actual sex is erased.
- AussieWog93 5y ago
goatxi 5y ago
Reminds me of that AI model for sex classification that would - quite accurately - recognize trans people as their birth sex rather than their 'gender identity'. People were complaining that it was transphobic or whatever, but it was just saying what it saw. Like what most of us see, even if we politely acquiesce to the fantasy.

MattGaiser 5y ago

It isn't so much a book that is a problem. It is divide and conquer by groups of students.

spoonjim 5y ago

Everyone wants a tie-in to the hot button issue of the day. You don’t need to get black trans kids involved to point out all of the flaws in Eproctoring.

wizzwizz4 5y ago
To point out enough flaws to disqualify e-proctoring, no. To point out all of them, yes.

fcatalan 5y ago

In our initial trials involving about 5 people the software systematically failed to recognise one woman just because her (pretty standard) hairstyle. It also failed to detect my kids coming up on camera a few times.

dna_polymerase 5y ago
Sounds like these solutions are just outright garbage. I wonder if ML/CV will ever work outside of GAFAM.

jarcane 5y ago

Recognition algorithms often make assumptions about the way people "should" be shaped according to their recorded gender or the way the machine attempts to "guess" said gender, and a clash in expectations gets flagged and generates false negatives because things aren't in the "right" places it thinks they should be. For a really simple example if this problem, the TSA body scanner machines flag on trans people constantly. It detects as an "unknown mass" if gendered body parts (ie. breasts/genitals) don't match its binary expectations for what gender is marked on their passport. Often they get it coming and going: a trans femme still marked M will get flagged for having breasts, but if they've changed it to F, they get flagged for their genitals instead. Now you get a full grope patdown from a suspicious TSA officer, because a machine was coded to make binary assumptions about bodies.

goatxi 5y ago
It just means that the body scanner is programmed to recognize biological realities (that transwomen are men presenting as women, and transmen are women presenting as men), rather than questionable self-beliefs.
donkarma 5y ago
So we put sex on passports? What is the issue?
- wizzwizz4 5y ago
  > Often they get it coming and going: a trans femme still marked M will get flagged for having breasts, but if they've changed it to F, they get flagged for their genitals instead. An F sex marker isn't going to stop this happening.
- goatxi 5y ago
  One problem is that in some countries (such as the UK), you can legally change your birth certificate to have it record your supposed 'gender identity' in the sex field instead. Your actual sex is erased.
AussieWog93 5y ago

goatxi 5y ago

Reminds me of that AI model for sex classification that would - quite accurately - recognize trans people as their birth sex rather than their 'gender identity'. People were complaining that it was transphobic or whatever, but it was just saying what it saw. Like what most of us see, even if we politely acquiesce to the fantasy.