5 ms·
One big problem is there are biases we're supposed to ignore, but we're not good at (able to?) ensuring this deliberate blindness in ML. For example, we're gen
by jdashg 7y ago
One big problem is there are biases we're supposed to ignore, but we're not good at (able to?) ensuring this deliberate blindness in ML.
For example, we're generally not allowed to discriminate based on gender, but even without a gender field, ML will happily imply a gender out of its firehose of other data, and discriminate based on it anyway.
"AI finds that employees are less likely to stay at the company if their names end in 'a' and they score higher on 'cultivation' rubrics" This same AI then recommends hiring applicant 2 over applicant 1. Is this allowable?
- lotu 7y agoAre we suggesting that humans don’t do the same thing?
- nitwit005 7y agoA bit of an aside, but humans do what you describe and have name letter preferences: https://en.m.wikipedia.org/wiki/Name-letter_effect https://en.m.wikipedia.org/wiki/Name-letter_effect
- dmix 7y agoWe have lots of inherent biological biases like that.
- repolfx 7y agoWhy would a name letter bias be biological caused? Or do you mean bias about biological factors? Bear in mind "bias" is just a perjorative for "generalisation" or alternatively "lesson learned". AI algorithms are good at detecting patterns in data and bad at being politically correct. This is not a flaw of the algorithms, it's a flaw in people who can't accept measured reality and go into denial. So an AI trying to hire programmers discriminates against female sounding names, because it's learned that this is correlated with success? Apply it to hiring nurses or primary school teachers and it'll probably do the opposite. This is only "bias" if you start from ideologically driven blank slate assumptions. Otherwise it's just common sense.
- deleted 7y ago[deleted]
- dmix 7y agoBiologically in the sense that we have affinities towards our own names or initials which studies have shown. Just a consequence of how our brains organize information. Instead of someone drilling into your head that your initials = good, and others = bad via your environment, it happens through natural processes. Most biases are just pattern matching which is necessary for efficient memory references and power quick on-the-spot judgements/decisions we need to make. People make too much of a big deal out of stereotypes like it's evil to hold them even though it's a basic function of the brain. It's entirely possible without constant vigilance to make misjudgements based on that but it will still happen to everyone, even the most socially aware people.