4 ms·

Are you scared yet? Meet Norman, the psychopathic AI

13 points by hdivider 8y ago

eboyjr 8y ago
> One study showed that software trained on Google News became sexist as a result of the data it was learning from. When asked to complete the statement, "Man is to computer programmer as woman is to X", the software replied 'homemaker". > Dr Joanna Bryson, from the University of Bath's department of computer science said that the issue of sexist AI could be down to the fact that a lot of machines are programmed by "white, single guys from California" and can be addressed, at least partially, by diversifying the workforce. Just curious but wouldn't this more likely be a result of a bias in the training data rather than the diversity of the programmers?
- saghm 8y ago
  Yes, but who picks the training data?
  - eboyjr 8y ago
    If it was trained on Google News it would just make more sense to me that it is a reflection of our journalists and possibly whatever algorithm Google uses to identify popularity, presumably a reflection of our society.
    - stochastic_monk 8y ago
      Right. This is simply a matter of what was used as training data.
    - lozenge 8y ago
      Yes, but the intention is to demonstrate one of the issues with using ML. For example, translating into a language which uses genders in grammar. A human translator will read a passage to determine a doctor's gender. A machine translation will know that doctor is usually translated as a male doctor and apply that to the translation. If the bias in the training is strong enough it can override actual signs in the source content that the doctor is a woman. If a person reads enough translated content, this just passes on the bias from the training data. https://mashable.com/2017/11/30/google-translate-sexism/?europe=true https://mashable.com/2017/11/30/google-translate-sexism/?eur...
- acct1771 8y ago
  > When asked to complete the statement, "Man is to computer programmer as woman is to X", the software replied 'homemaker". Seems they're putting a lot of weight on an unqualified question...the AI just fed back the easiest, most popular, stereotypical answer it came up with. A tee-ball response. It didn't stop and think about who'd be offended, first.
  - deleted 8y ago
    [deleted]

13 points by hdivider 8y ago

> One study showed that software trained on Google News became sexist as a result of the data it was learning from. When asked to complete the statement, "Man is to computer programmer as woman is to X", the software replied 'homemaker". > Dr Joanna Bryson, from the University of Bath's department of computer science said that the issue of sexist AI could be down to the fact that a lot of machines are programmed by "white, single guys from California" and can be addressed, at least partially, by diversifying the workforce. Just curious but wouldn't this more likely be a result of a bias in the training data rather than the diversity of the programmers?

saghm 8y ago
Yes, but who picks the training data?
- eboyjr 8y ago
  If it was trained on Google News it would just make more sense to me that it is a reflection of our journalists and possibly whatever algorithm Google uses to identify popularity, presumably a reflection of our society.
  - stochastic_monk 8y ago
    Right. This is simply a matter of what was used as training data.
  - lozenge 8y ago
    Yes, but the intention is to demonstrate one of the issues with using ML. For example, translating into a language which uses genders in grammar. A human translator will read a passage to determine a doctor's gender. A machine translation will know that doctor is usually translated as a male doctor and apply that to the translation. If the bias in the training is strong enough it can override actual signs in the source content that the doctor is a woman. If a person reads enough translated content, this just passes on the bias from the training data. https://mashable.com/2017/11/30/google-translate-sexism/?europe=true https://mashable.com/2017/11/30/google-translate-sexism/?eur...
acct1771 8y ago
> When asked to complete the statement, "Man is to computer programmer as woman is to X", the software replied 'homemaker". Seems they're putting a lot of weight on an unqualified question...the AI just fed back the easiest, most popular, stereotypical answer it came up with. A tee-ball response. It didn't stop and think about who'd be offended, first.
- deleted 8y ago
  [deleted]