5 ms·

I find it fascinating that in the middle of a hype cycle here, where really central people to AI are saying we should be worried (Altman, Hinton etc.), there's

I find it fascinating that in the middle of a hype cycle here, where really central people to AI are saying we should be worried (Altman, Hinton etc.), there's still just outright animosity towards people who are actually interested in studying these problems. It's also very far from a theoretical problem, did you know that crash test dummies are men? Literally, until extremely recently crash test dummies were all designed to be 50th percentile men[1]. Surely they test women dummies too? Oh yeah! They put a small woman dummy in the passenger seat. But still, they've presumably measured women and designed something appropriate? Nope! The small female dummy is defined as a scaled version of the male dummy. It's genuinely astonishing how short sighted engineers can be in designing technology and it's wild how angry they get when anyone tries to actually fix it. [1]: https://www.sciencedirect.com/science/article/pii/S0001457519303483#sec0015 https://www.sciencedirect.com/science/article/pii/S000145751...

mempko 3y ago
Exactly. Same problem with facial recognition which simply didn't work on anyone with dark skin. The bigger the models the less obvious the bias becomes but is still there. Gebru did good work quantifying that but google shut it down. And it's sad that many of the comments here are reactionary and hostile towards Gebru. Why is the community here so bad? I thought it's supposed to be full of smart people with computers?
- 908B64B197 3y ago
  > hostile towards Gebru Her paper aren't really high quality. Her carbon footprint calculations had all of Google's compute running from jet fuel. [0] In her "Gender Shades" paper she completely ignored the existence of the Asian race. That's a pretty big limitation for a paper named in such general terms. In another place she labels Asians as "white adjacent". She can't argue about technical subjects (here's an example [1]) because she isn't technical. [0] https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/ https://www.technologyreview.com/2020/12/04/1013294/google-a... [1] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter-amid-acrimonious-exchanges-on-ai-bias/ https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
  - mempko 3y ago
    The actual paper [0] that Gebru wrote referenced Strubell's climate impact paper [1]. None of them make the assumption that google compute runs from jet fuel. They compare training the models to flying, driving, etc. In the "Gender Shades" paper [2] as you call it, she doesn't use race but dermatologist approved Fitzpatrick Skin Type in their analysis as a proxy which of course covers all people including Asians. The kind of bullshit (a technical term) response you gave is common among reactionaries who don't read, and can't do science. [0] https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 [1] https://arxiv.org/pdf/1906.02243.pdf https://arxiv.org/pdf/1906.02243.pdf [2] http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a...
    - 908B64B197 3y ago
      > she doesn't use race but dermatologist approved Fitzpatrick Skin Type in their analysis as a proxy which of course covers all people including Asians. Erasing Asians from stats when convenient has been a staple of the CRT movement for quite some times (when, for example, their existence break down stats "proving" racism against non-whites). The term "white-adjacent" has become one of the dog whistles of the movement.
      - mempko 3y ago
        But that's not what the paper does. Asians have many skin colors from light to very dark (compare China, Korea, and Japan to Thailand, Vietnam, and India). I'm not sure what you are talking about here. I don't see how it erases Asians since Asians are a very diverse group of people in appearance. In regards to CRT, CRT has always included Asians, for example, studying the Chinese Exclusion Act in the US. Many people who criticize CRT (mostly on the right) have no idea what it is. It seems to me that the anti-CRT crowd (mostly right-wing reactionaries) are making shit up to bring down this kind of important research in AI down.
        908B64B197 3y ago
- revelio 3y ago
  > Same problem with facial recognition which simply didn't work on anyone with dark skin. That's an urban legend btw. The issue was actually that the person for whom it failed had the camera pointed straight up with a bright light behind him, so the image was totally washed out and lacked saturation. The software failed in the same way if you had a dark background and then shone a bright light into a white person's face. The algorithm needs contrast to work. Also, people on this forum really should know better than insinuate that a program working imperfectly is caused by racism. As if programmers sit around finding ways to troll black people by deliberately making face recognition not work. Even the idea of it is absurd. When people make such insinuations, it reinforces the widely held belief that accusations of racism are invariably bullshit, which can lead to a boy-crying-wolf problem.
deleted 3y ago
[deleted]
dragonwriter 3y ago
> I find it fascinating that in the middle of a hype cycle here, where really central people to AI are saying we should be worried (Altman, Hinton etc.), there's still just outright animosity towards people who are actually interested in studying these problems. Note that the abstract speculative future concerns Altman and Hinton are concerned with are very different from the immediate and present issues Gebru, her former Google colleague Margaret Mitchell (now at Huggingface), and others in the AI ethics space are raising.
- SilverBirch 3y ago
  Absolutely, I think actually her concerns are much more direct, measurable and actionable. It didn't take a genius to point out "maybe we should crash test something other than a male crash test dummy", but it was much more impactful and actionable than "let's figure out how to test self-driving cars" (to which the answer is still "let's just drive them and hope")

mempko 3y ago

Exactly. Same problem with facial recognition which simply didn't work on anyone with dark skin. The bigger the models the less obvious the bias becomes but is still there. Gebru did good work quantifying that but google shut it down. And it's sad that many of the comments here are reactionary and hostile towards Gebru. Why is the community here so bad? I thought it's supposed to be full of smart people with computers?

908B64B197 3y ago
> hostile towards Gebru Her paper aren't really high quality. Her carbon footprint calculations had all of Google's compute running from jet fuel. [0] In her "Gender Shades" paper she completely ignored the existence of the Asian race. That's a pretty big limitation for a paper named in such general terms. In another place she labels Asians as "white adjacent". She can't argue about technical subjects (here's an example [1]) because she isn't technical. [0] https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/ https://www.technologyreview.com/2020/12/04/1013294/google-a... [1] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter-amid-acrimonious-exchanges-on-ai-bias/ https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
- mempko 3y ago
  The actual paper [0] that Gebru wrote referenced Strubell's climate impact paper [1]. None of them make the assumption that google compute runs from jet fuel. They compare training the models to flying, driving, etc. In the "Gender Shades" paper [2] as you call it, she doesn't use race but dermatologist approved Fitzpatrick Skin Type in their analysis as a proxy which of course covers all people including Asians. The kind of bullshit (a technical term) response you gave is common among reactionaries who don't read, and can't do science. [0] https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 [1] https://arxiv.org/pdf/1906.02243.pdf https://arxiv.org/pdf/1906.02243.pdf [2] http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a...
  - 908B64B197 3y ago
    > she doesn't use race but dermatologist approved Fitzpatrick Skin Type in their analysis as a proxy which of course covers all people including Asians. Erasing Asians from stats when convenient has been a staple of the CRT movement for quite some times (when, for example, their existence break down stats "proving" racism against non-whites). The term "white-adjacent" has become one of the dog whistles of the movement.
    - mempko 3y ago
      But that's not what the paper does. Asians have many skin colors from light to very dark (compare China, Korea, and Japan to Thailand, Vietnam, and India). I'm not sure what you are talking about here. I don't see how it erases Asians since Asians are a very diverse group of people in appearance. In regards to CRT, CRT has always included Asians, for example, studying the Chinese Exclusion Act in the US. Many people who criticize CRT (mostly on the right) have no idea what it is. It seems to me that the anti-CRT crowd (mostly right-wing reactionaries) are making shit up to bring down this kind of important research in AI down.
      - 908B64B197 3y ago
revelio 3y ago
> Same problem with facial recognition which simply didn't work on anyone with dark skin. That's an urban legend btw. The issue was actually that the person for whom it failed had the camera pointed straight up with a bright light behind him, so the image was totally washed out and lacked saturation. The software failed in the same way if you had a dark background and then shone a bright light into a white person's face. The algorithm needs contrast to work. Also, people on this forum really should know better than insinuate that a program working imperfectly is caused by racism. As if programmers sit around finding ways to troll black people by deliberately making face recognition not work. Even the idea of it is absurd. When people make such insinuations, it reinforces the widely held belief that accusations of racism are invariably bullshit, which can lead to a boy-crying-wolf problem.

deleted 3y ago

[deleted]

dragonwriter 3y ago

> I find it fascinating that in the middle of a hype cycle here, where really central people to AI are saying we should be worried (Altman, Hinton etc.), there's still just outright animosity towards people who are actually interested in studying these problems. Note that the abstract speculative future concerns Altman and Hinton are concerned with are very different from the immediate and present issues Gebru, her former Google colleague Margaret Mitchell (now at Huggingface), and others in the AI ethics space are raising.

SilverBirch 3y ago
Absolutely, I think actually her concerns are much more direct, measurable and actionable. It didn't take a genius to point out "maybe we should crash test something other than a male crash test dummy", but it was much more impactful and actionable than "let's figure out how to test self-driving cars" (to which the answer is still "let's just drive them and hope")