5 ms·
The critic here, Timnit Gebru, is not doing a good job of articulating a well-reasoned argument about why dataset bias is not the only problem with ML. Allow me
by ponker 6y ago
The critic here, Timnit Gebru, is not doing a good job of articulating a well-reasoned argument about why dataset bias is not the only problem with ML. Allow me to attempt to do her job:
Machine learning approaches are powerful because they can take a “small” set of labeled data and use that to make billions of new inferences at low cost. The problem is that even when these data sets are fully representative of the “real world,” the “real world” is rife with biases and unfairness and machine learning takes these unfairnesses and gives them a huge Tensorflow-powered megaphone.
For example, let’s say you wanted to use machine learning to create a system for employers to make sure that they aren’t hiring likely drug dealers. If you get a data set of every single American drug dealer’s mug shot, you’d get a disproportionately black population because white kids selling ecstasy to their high school classmates rarely get arrested and booked. The data set is not “biased” —- it is complete —- but the enforcement of the law is biased. But now, rather than using the development of this software as a moment to remove unfair bias, a machine learning model will amplify it, causing lots of black men to be denied legitimate employment opportunities, which will actually increase the chance that they sell illegal drugs to support themselves.
This machine learning doesn’t just amplify the biases in the data set, it amplifies all of the biases and other imperfections in all of the events that fed into the creation of that data set.
- azepoi 6y agoI feel like this is really unhelpful nitpicking on the definition of dataset bias. In your example the dataset you want is the one of drug dealers so the dataset from the convictions IS biased and uncomplete.
- ponker 6y agoWell this is a contrived example. But the fact is that all datasets contain the full biases of the world that created them. So there is no "unbiased" dataset... meaning that powerful AI amplifies the world as it is, rather than a designed point of view of how the world should be. That's fine in some cases (e.g. tumor detection in X Rays -- the combined judgments of thousands of surgeons might truly reflect how we want to treat a patient) but may not be better in other cases (the combined judgments of thousands of police officers may NOT be how we want to treat a suspect).
- azepoi 6y agoAnd imo LeCun would agree with this, they were talking past each other and I don't get the behavior of people lashing out and forcibly getting everyone to bend to their opinions.
- natch 6y agoWell said, all of that. >it amplifies all of the biases and other imperfections in all of the events that fed into the creation of that data set. A common retort is "well then, fix the data set, or correct for the bias." But a further problem, to riff off what you said, is it's not just this one data set, it's all data sets. And while a researcher or engineer may be able to make choices about their own data sets, they won't be able to control all data sets. This problem is also embedded amongst other problems such as the tendency for people to "trust the computer," which sounds naive, but is sometimes restated, amongst smug people who think they aren't naive, as "trust the algorithm."
- kutorio 6y agoThis is a much better articulation of the problem. From this my takeaway is that the definition of the word "bias" is overloaded here: 1) biased as in "the sample does not represent the population" 2) biased as in "the world is far from ideal" And mixing the two definitions together is confusing: "Even if the data-set is not biased, biases still exist"
- gold_mustard 6y agoIf this hypothetical system's goal is to prevent convicted drug dealers from getting hired, this seems like a perfectly complete and unbiased data set.