5 ms·

Indeed so. I'm always reminded of Charles Babbage's self-reported interactions on the subject of his Difference Engine: 'On two occasions I have been asked, —

Indeed so. I'm always reminded of Charles Babbage's self-reported interactions on the subject of his Difference Engine: 'On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.' But I think the problem is more subtle than you give it credit for. The software may be working as written, but 'intention' rarely maps cleanly to execution. The behavior of a complex system is a multidimensional shape; stretching it one way or another along some axis according to your intention for its behavior can easily have unanticipated, and often unobservable effects along other axes that you were not minding. So it may be an intention that the system optimize for click-through rates for ads, while at the same time not consciously intended to be biased against, for example, race, but also not deliberately crafted to avoid this behavior. This may functionally be the same thing as intending it to be racist, but it has different implications for who's guilty and how things get fixed.

rspeer 9y ago
The quip by Babbage is clever, but Babbage was too early and didn't know what we know. He could not anticipate the digital abstraction, error-correcting codes, compressed sensing, and machine learning. Modern computing systems have components that produce output that's "better" than their input, at every level. Because we have information theory now, we know it is possible for a machine to produce good output even when some of its input is bad. Google, incidentally, has aimed to do that for its entire existence. From the first time the Google search engine was deployed, its job was to take in the messy Web, full of misdirection and spam, and show you the useful pages you were looking for. And we can keep hoping for Google to keep doing this as the threats to information become more insidious.

rspeer 9y ago

The quip by Babbage is clever, but Babbage was too early and didn't know what we know. He could not anticipate the digital abstraction, error-correcting codes, compressed sensing, and machine learning. Modern computing systems have components that produce output that's "better" than their input, at every level. Because we have information theory now, we know it is possible for a machine to produce good output even when some of its input is bad. Google, incidentally, has aimed to do that for its entire existence. From the first time the Google search engine was deployed, its job was to take in the messy Web, full of misdirection and spam, and show you the useful pages you were looking for. And we can keep hoping for Google to keep doing this as the threats to information become more insidious.