4 ms·

> I’m always amazed how often the “surprise” that digital technology is in fact not unbiased and robotic in nature keeps getting written about. Algorithms that

by Hyva 9y ago

> I’m always amazed how often the “surprise” that digital technology is in fact not unbiased and robotic in nature keeps getting written about. Algorithms that optimize for click through rate for ads and search relevancy will inevitably be a reflection of ourselves and all of the shitty things various people believe. What you just observed and described is that technology _is_ unbiased and robotic. The bias is in the humans. You can't remove it with a software bandage; the software is working as intended.

Asdfbla 9y ago
> You can't remove it with a software bandage Though some people might argue that the biases software reproduces will in turn reinforce biases held by the users at large. Conversely, you may reduce human bias if you algorithmically reduce the bias in the data (which is possible). Not saying whether Google should, that's a very delicate debate, but it's thinkable. I wouldn't want to be in the place of the person at Google trying to make guidelines for that, but it's definitely doable. For instance, you could probably easily argue that the phrase "three black teenagers" (an example from the article) is pretty neutral in itself and definitely shouldn't be interpreted as a call to produce arrest pictures. In general the term "three [ethnicity] teenagers" could probably be normalized with regards to the setting the pictures show, for instance.
dmreedy 9y ago
Indeed so. I'm always reminded of Charles Babbage's self-reported interactions on the subject of his Difference Engine: 'On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.' But I think the problem is more subtle than you give it credit for. The software may be working as written, but 'intention' rarely maps cleanly to execution. The behavior of a complex system is a multidimensional shape; stretching it one way or another along some axis according to your intention for its behavior can easily have unanticipated, and often unobservable effects along other axes that you were not minding. So it may be an intention that the system optimize for click-through rates for ads, while at the same time not consciously intended to be biased against, for example, race, but also not deliberately crafted to avoid this behavior. This may functionally be the same thing as intending it to be racist, but it has different implications for who's guilty and how things get fixed.
- rspeer 9y ago
  The quip by Babbage is clever, but Babbage was too early and didn't know what we know. He could not anticipate the digital abstraction, error-correcting codes, compressed sensing, and machine learning. Modern computing systems have components that produce output that's "better" than their input, at every level. Because we have information theory now, we know it is possible for a machine to produce good output even when some of its input is bad. Google, incidentally, has aimed to do that for its entire existence. From the first time the Google search engine was deployed, its job was to take in the messy Web, full of misdirection and spam, and show you the useful pages you were looking for. And we can keep hoping for Google to keep doing this as the threats to information become more insidious.
jancsika 9y ago
> What you just observed and described is that technology _is_ unbiased and robotic. Just to get calibrated: what is an example-- real or imagined-- of technology that is biased?
- deleted 9y ago
  [deleted]
- PeterisP 9y ago
  For example, a search algorithm with an override that removes all links of sexual nature for certain hardcoded queries (say, "black girls") could be an example of a biased technology. If I'd have to make a definition, then it would probably be a technology that somehow encodes an assumption that doesn't match reality. In this context, saying that "black girls" is an innocent search term and users searching for it don't expect to get sexual results is such an assumption - it would be polite, politically correct and possibly socially desirable; but it seems likely this is simply not true in the reality we live in.
  - sokoloff 9y ago
    I had a person on my team preparing a presentation and for whatever (reasonable) reason was looking for a stock image of a rabbit in winter with frozen precipitation around. Just as our (very straight laced) CIO walked up behind, employee hit image search on "snow bunny". Hilarity ensued...
rspeer 9y ago
> You can't remove it with a software bandage Turns out you can. It just takes effort. And you should, because amplifying shitty things about people is not the ideal state of software.

by Hyva 9y ago

Asdfbla 9y ago

> You can't remove it with a software bandage Though some people might argue that the biases software reproduces will in turn reinforce biases held by the users at large. Conversely, you may reduce human bias if you algorithmically reduce the bias in the data (which is possible). Not saying whether Google should, that's a very delicate debate, but it's thinkable. I wouldn't want to be in the place of the person at Google trying to make guidelines for that, but it's definitely doable. For instance, you could probably easily argue that the phrase "three black teenagers" (an example from the article) is pretty neutral in itself and definitely shouldn't be interpreted as a call to produce arrest pictures. In general the term "three [ethnicity] teenagers" could probably be normalized with regards to the setting the pictures show, for instance.

dmreedy 9y ago

Indeed so. I'm always reminded of Charles Babbage's self-reported interactions on the subject of his Difference Engine: 'On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.' But I think the problem is more subtle than you give it credit for. The software may be working as written, but 'intention' rarely maps cleanly to execution. The behavior of a complex system is a multidimensional shape; stretching it one way or another along some axis according to your intention for its behavior can easily have unanticipated, and often unobservable effects along other axes that you were not minding. So it may be an intention that the system optimize for click-through rates for ads, while at the same time not consciously intended to be biased against, for example, race, but also not deliberately crafted to avoid this behavior. This may functionally be the same thing as intending it to be racist, but it has different implications for who's guilty and how things get fixed.

rspeer 9y ago
The quip by Babbage is clever, but Babbage was too early and didn't know what we know. He could not anticipate the digital abstraction, error-correcting codes, compressed sensing, and machine learning. Modern computing systems have components that produce output that's "better" than their input, at every level. Because we have information theory now, we know it is possible for a machine to produce good output even when some of its input is bad. Google, incidentally, has aimed to do that for its entire existence. From the first time the Google search engine was deployed, its job was to take in the messy Web, full of misdirection and spam, and show you the useful pages you were looking for. And we can keep hoping for Google to keep doing this as the threats to information become more insidious.

jancsika 9y ago

> What you just observed and described is that technology _is_ unbiased and robotic. Just to get calibrated: what is an example-- real or imagined-- of technology that is biased?

deleted 9y ago
[deleted]
PeterisP 9y ago
For example, a search algorithm with an override that removes all links of sexual nature for certain hardcoded queries (say, "black girls") could be an example of a biased technology. If I'd have to make a definition, then it would probably be a technology that somehow encodes an assumption that doesn't match reality. In this context, saying that "black girls" is an innocent search term and users searching for it don't expect to get sexual results is such an assumption - it would be polite, politically correct and possibly socially desirable; but it seems likely this is simply not true in the reality we live in.
- sokoloff 9y ago
  I had a person on my team preparing a presentation and for whatever (reasonable) reason was looking for a stock image of a rabbit in winter with frozen precipitation around. Just as our (very straight laced) CIO walked up behind, employee hit image search on "snow bunny". Hilarity ensued...

rspeer 9y ago

> You can't remove it with a software bandage Turns out you can. It just takes effort. And you should, because amplifying shitty things about people is not the ideal state of software.