4 ms·
I've often heard fans of AI[1] claim that a benefit of machine-based evaluations is the removal of human bias. The machine applies the rules consistently, while
by pdkl95 11y ago
I've often heard fans of AI[1] claim that a benefit of machine-based evaluations is the removal of human bias. The machine applies the rules consistently, while humans allow race, sex, religion, politics, eye-color[2], etc to bias their decisions, often without knowing the bias even exists.
This sounds great at first, but it's really just moving the problem around. Instead of many individuals making their own biased decisions, we get a uniform application of the biases of whoever designed the AI. They still may not realize how their biases influence the design process, so it's easy to think that the AI is somehow "neutral".
This is particularly bad because it removes locality from the decision process. While the fallible humans sometime inject their own racism/sexism/etc, the freedom to make those mistakes also gave them the power to fix problems that were never included in the "official policy".
Decision by remote algorithm is really just another type of central planning, and I expect it to have the same inability to respond to local conditions that cause a lot of the problems in the traditional communist states.
[1] usually combined with a new type of data collection (surveillance-as-a-business-model)
[2] biases that are often taught socially: http://www.pbs.org/wgbh/frontline/film/class-divided/ http://www.pbs.org/wgbh/frontline/film/class-divided/
- cwyers 11y agoMachine learning systems are only as biased as their training data and their construction. It doesn't even have to be overt. Like when Google Photos tagged photos of African-American users as gorillas[1] -- sure, the machine learning algorithm wasn't racist in any recognizable sense, but it reflected the bias of the training data, which didn't have enough African-American people in it to let the algorithm learn about them. 1) http://www.theverge.com/2015/7/1/8880363/google-apologizes-photos-app-tags-two-black-people-gorillas http://www.theverge.com/2015/7/1/8880363/google-apologizes-p...
- pdkl95 11y ago> training data and their construction That's what I'm talking about. Machine learning systems - and everything else that is the product of human decisions - are a reflection of the biases, intentions, and other qualities of the designer. Even the initial choice to use a machine learning system is influenced by fallible humans. Why use machine learning? What kind, and why? What is the underlying model that is being optimized? What data was it trained with? Is the design of the system a proxy for something else, intentionally or unintentionally? The real world is complicated and messy, and any model of it is always gong to be incomplete.
- eli_gottlieb 11y ago>Machine learning systems are only as biased as their training data and their construction. The Bias-Variance Tradeoff means that all machine learning involves bias. Without it, you couldn't learn.
- edanm 11y agoI've often heard that benefit mentioned, but not at all in the context of AI. Rather, it's usually in the context of applying any kind of statistics or data analytics to a field previously devoid of it (e.g. Moneyball). I don't understand why this would be a benefit of AI.
- pdkl95 11y agohttps://media.ccc.de/v/32c3-7457-predicting_crime_in_a_big_data_world https://media.ccc.de/v/32c3-7457-predicting_crime_in_a_big_d... If you're still interested, this talk on "predicting crime" specifically cites supposedly "unbiased" policing as a benefit of their new machine learning products. It all sounds like snake-oil, subconscious prejudices made into models, and confirmation bias. /sigh/