3 ms·
> I'm still not entirely convinced that Google's early models (as used in Images and Photos), and their infamous inability to tell black people apart from goril
by AnthonyMouse 3y ago
> I'm still not entirely convinced that Google's early models (as used in Images and Photos), and their infamous inability to tell black people apart from gorillas, was entirely an accidental occurrence. Clearly, such an association would not have been the company's intent, and a properly-produced model would not have presented it.
Nah, you're just misunderstanding the kind of bias that produced it.
Humans and gorillas are both primates. They look quite similar to each other and certainly more similar to each other than either of them are to a lizard or a bowling ball or a tree. It's the kind of mistake you'd expect it to be likely to make in general.
Now suppose you have a training data set with both humans and gorillas, with a human racial composition that reflects the makeup of the population of the first world, i.e. the majority have light skin. You ask it to classify a human with light skin, it's not completely sure if it's a human or a gorilla, but most of the humans it was trained on have light skin and basically all of the gorillas have dark skin, so it skews human. You show it a human with dark skin, it skews gorilla.
Models make that kind of error all the time -- it might usually be able to guess right for a billiard ball but its guess for the black 8-ball is a bowling ball. But some errors have political significance because if a human did that we would assume they meant something by it, and then people want to ascribe that intent to the model.
- underlipton 3y agoThis is a rationalization that fairly easy to debunk (in two ways, even): 1) Gorillas do not have dark skin the way that humans have dark skin. In fact, many features differentiate gorillas from all humans, including dark-skinned ones. The bias present in the model has to overcome these differences to categorize dark-skinned humans with gorillas rather than light-skinned humans. It's reasonable to question how this could happen accidentally. 2) The contention is that the data was poisoned. How, exactly, do you tell apart a case where not enough black people were included in the training data on purpose, versus as a result of institutional racism? In fact, does it matter? One could hold that the data poisoning simply happened at a different part of the process. Also, this: >with a human racial composition that reflects the makeup of the population of the first world, i.e. the majority have light skin. is arguable. The "first world" (within the US/Western political sphere of influence) includes billions of people who would be considered brown/black, including black, indigenous, and mestizo South and North Americans, black Africans, inhabitants of select West Asian countries, East Asians in Japan, and indigenous inhabitants of Oceania. What you mean to say is the "a human racial composition that reflects the makeup of populations Google engineers deem important," which is again damning in its own way.