4 ms·

This brushes over the very real prospect of automated discrimination that we increasingly face as AI techniques replace simple statistical methods. As a societ

by shubb 9y ago

This brushes over the very real prospect of automated discrimination that we increasingly face as AI techniques replace simple statistical methods. As a society, we would want to avoid a situation where someone is turned down for a job for being female or black. There are various laws in place to try and prevent this. In the olden days, a way of pricing insurance policies would be to pick a bunch of features, identify how each one contributes to risk, and use that for pricing. Some characteristics are protected and you can't use them for pricing. If you are using them in your CV sifting algorithm you are heading for a lawsuit. It's fairly easy to spot if ethnicity is in your feature set. Sometimes, you'd have a feature that was a proxy for a protected characteristic. Supposing that women are safer drivers, perhaps a medical history of using contraceptive pills would be predictive of safe driving, because it is in fact a proxy for gender. So statisticians would examine the dominant factors in their pricing model and show that whey were not proxies. Now maybe we could conceivably throw a bunch of data at a neural network type thing, and get an opaque pricing model. It might find proxies, even if you don't include the protected data items. Maybe, if (according to the article) black people with certain characteristics are more likely to default on loans, it could be using common black names or media preferences as a proxy for ethnicity. An applicant who had few negative characteristics might get a high quote simply because the system figured out he was black. And no one would really know. It would be very hard to look at a complex bunch of weights in layers, and figure out that not only did liking certain TV shows effect your claim, but also that certain specific TV shows had a high impact and were correlated with race or gender or whatever. You'd just see a bunch of weights trained on clickstream data or something. That's worth thinking about as a society I think?

carlmr 9y ago
You mean if we model an AI after our biological brains it will be prone to stereotyping? So our artificial brains do in fact do what we set out to do. They act a bit too human.
- shubb 9y ago
  No, that isn't what I meant. Older statistical methods were not complex and could be analyzed by a human to ensure they complied with legal and moral obligations. Newer approaches such as but not limited to neural networks create a model that is so complex that it is opaque. You cannot necessarily verify that it is complying with legal and moral obligations. This is fine when you are doing song recommendations. It is more of a problem when you are making decisions that could be seen as discriminatory. There are probably best practices that can mitigate some of these problems, and we can use our human understanding of the features to try and reduce risk. Indeed, this is the same problem as 'How do we build a safety critical system with AI, and deterministically show its safe like we could with a simpler one'. Here is a writeup of some of those issues [1]. But not everyone will - because sometimes discrimination is profitable if you can get away with it. If a car insurance company can hide gender discrimination behind a complex model then they are incentivized to do that. So, we possibly have a social problem. The problem is not that newer techniques are 'like a brain and susceptible bias in the same way' because to some extent that isn't true - they work on the real data available unless you feed them movie scripts. The issue is if they intentionally or accidentally become a way of getting around laws society made for good reason. [1] https://intelligence.org/2013/08/25/transparency-in-safety-critical-systems/ https://intelligence.org/2013/08/25/transparency-in-safety-c...

by shubb 9y ago

carlmr 9y ago

You mean if we model an AI after our biological brains it will be prone to stereotyping? So our artificial brains do in fact do what we set out to do. They act a bit too human.

shubb 9y ago
No, that isn't what I meant. Older statistical methods were not complex and could be analyzed by a human to ensure they complied with legal and moral obligations. Newer approaches such as but not limited to neural networks create a model that is so complex that it is opaque. You cannot necessarily verify that it is complying with legal and moral obligations. This is fine when you are doing song recommendations. It is more of a problem when you are making decisions that could be seen as discriminatory. There are probably best practices that can mitigate some of these problems, and we can use our human understanding of the features to try and reduce risk. Indeed, this is the same problem as 'How do we build a safety critical system with AI, and deterministically show its safe like we could with a simpler one'. Here is a writeup of some of those issues [1]. But not everyone will - because sometimes discrimination is profitable if you can get away with it. If a car insurance company can hide gender discrimination behind a complex model then they are incentivized to do that. So, we possibly have a social problem. The problem is not that newer techniques are 'like a brain and susceptible bias in the same way' because to some extent that isn't true - they work on the real data available unless you feed them movie scripts. The issue is if they intentionally or accidentally become a way of getting around laws society made for good reason. [1] https://intelligence.org/2013/08/25/transparency-in-safety-critical-systems/ https://intelligence.org/2013/08/25/transparency-in-safety-c...