7 ms·

>This is the belief that A.I. codes are recipes for automating ethics itself; and that once a broad consensus around such codes has been achieved, the problem o

>This is the belief that A.I. codes are recipes for automating ethics itself; and that once a broad consensus around such codes has been achieved, the problem of determining an ethically positive future direction for computer code will have begun to be solved. I can think of literally nobody who believes this. I don't think anyone working in ethical AI would pretend that ethics can be "solved."

willberman 6y ago
For any point of view on AI, I can probably find a decent number of non technical people who hold that POV. Technical illiteracy is a significant problem imo.
curiousgal 6y ago
There always seems to be a disconnect between tech philosophers and the technology they write about. Maybe they should try to actually work on 'AI' before preaching about it. To confirm your point a simple example is when working on an automated classifier for loan applications. One way to make the system more ethical, that is less descriminating based on certain features like race or gender, is to often lower the threshold of acceptance for the marginalized classes and raise it for the other ones. Anyone who had to do that sees how it poses a dilemma of equality vs fairness or even fairness vs cost. So indeed, ethics cannot be solved.
- msla 6y ago
  > One way to make the system more ethical, that is less descriminating based on certain features like race or gender, is to often lower the threshold of acceptance for the marginalized classes and raise it for the other ones. Or don't input those values at all. Is it even legal for a human to take those things into consideration? Regardless, the AI morally shouldn't, so there's no reason for the data to even be in the input.
  - cmendel 6y ago
    Right; however, the problem is that ML is generally pretty good at finding underlying patterns in data, so even if you make an algorithm race-blind it can still construct a dummy variable that is effectively race. Unfortunately, it does this by codifying pre-existant social outcomes which are often not free of bias.
    - RobPfeifer 6y ago
      This is why disparate impact backtesting and go-forward monitoring should be done on all credit model releases
  - hectormalot 6y ago
    As someone who works in this field: the challenge is more nuanced. For example: Even if you don’t input ethnicity as a value, you might input ZIP codes. ZIP codes however can be very predictive of someone’s ethnic background, and you might still end up unfairly discriminating against certain groups of people. The Amazon recruiting AI failure is another great example of this, I think it didn’t take gender into account, but discriminated against activities like ‘head of women’s association’ etc. Finally, what’s ethical is often up for discussion. Depending on context, people will optimize for equality (equal % of credits approved for both genders) vs helping those behind (Higher % approved for those historically disadvantaged) vs helping those that ‘deserve’ it of groups were unknown (finance such that default rates of both groups are expected to be equal, likely benefiting those with already good credit scores). I don’t think any of these answers is wrong, but they are mutually exclusive. One challenge I don’t see discussed that often is that the current system of many humans evaluating loan applications leads to ‘noise‘ in outcomes so that we (on average) don’t see that much bias. (Eg a few people of certain ethnicity not getting loans because a few people are biased against them) A single AI system scoring all applications however is consistent in its bias against a group, leading to more obvious effects (eg most people in a certain area not receiving loans because the algorithm is biased against it) In my opinion, an important part is the right on human review of meaningful automated decisions. Not mentioned that often, but it’s also part of the GDPR.
- billjings 6y ago
  This seems to be a limitation of the tool, yes?
- RobPfeifer 6y ago
  How would you do that without including a variable that raises a flag with the regulators? There are ways to improve access to credit for marginalized classes but this is generally done by adding variables that are independent of those related to protected classes.
- behnamoh 6y ago
  A few points: 1. I mean I'm all in favor of ethics, but it's like saying "I believe in God/god". Everyone seems to have their own interpretation of it. When people are talking about ethics/God/etc., how can they be sure that they're talking about the same thing? Centuries of debating over what is "ethical" has got us no where. Kant got close, but then there are caveats to his definition of ethics, too. 2. Who says AI should behave according to "our" ethics (whatever those are)? Sooner or later, with the emergence of ASI (artificial super intelligence) we'll have to accept the superior form of consciousness the ASI is. Expecting it to behave according to our rules is like monkeys asking us not to take away all the bananas because for them, it's unethical. 3. Even without AI, people have all sorts of biases and prejudices against certain ethnicities/races/colors/etc. You're not happy with the results that AI provides? Fine, hire some people instead and see where that leads you. The only thing you lose is the higher performance of AI. 4. Expecting AI systems to have no bias (while employees replaced by the AI certainly were prone to bias) is equivalent to expecting technology to change our world for the better. That doesn't always happen. You think smartphones are necessarily a good thing? Think about where they produce those gadgets and how they treat the workers. 5. At the end of the day, companies will stick with whatever "works" for them, regardless of what label we put on it (good/bad/ethical/fair/...). If it adds value, then most of the time it's enough. If being ethical (according to some definition) doesn't bring in more cash, companies are unlikely to invest in it. At least, with explainable AI, there is benefits for the firm to improve the accuracy of the AI in the long term. But pushing companies to adopt some ethical/unbiased/fair AI that doesn't add value for them is itself unethical (in my book).
  - kijin 6y ago
    > we'll have to accept the superior form of consciousness the ASI is. On what definition of "have to"? Large numbers of people will not only refuse to accept the ASI's interpretation of ethics, they will also refuse to acknowledge that the purported ASI is any better than ourselves/themselves when it comes to ethics. If you try to settle the matter by giving the ASI the force of law, there will be armed revolts and violent crackdowns.
    - behnamoh 6y ago
- yamrzou 6y ago
  What if it turns out there is reason people discriminate using those features (for example that people of race X tend to fail repaying their loans), and the classifier just learned that reason from the data. Why should we be expected to "fix" the reality for the classifier?
  - stanmancan 6y ago
    Because now people start to get rejected for loans because of their race, and that’s wrong for many obvious reasons.
    - yamrzou 6y ago
      I agree with you that it is generally wrong, i.e. that other factors should come first. But what if the data suggests it? For example if is known that if you lend to someone from race X, there is 80% chance he would not pay back. Would you not incorporate that factor into your decision?
      - stanmancan 6y ago
        No. Find the other, non-racist data points that correlate.
        matheusmoreira 6y ago
        Why should correlations with ethnicity be dismissed? Epidemiological studies tell us some diseases disproportionally affect certain groups. Should doctors dismiss this knowledge as racist and refrain from screening patients in the affected groups because they might come off as racist?
        lidHanteyk 6y ago
        Yes. We know, more specifically, that genes are to blame for diseases like hemochromatosis [0], sickle cell anemia [1], or Tay-Sachs [2]. We also know, from pedigree collapse [3], that humans broadly form one single race. Therefore we know that correlations with any definition of ethnicity or race are spurious, because those definitions must be socially constructed, because the gene pool simply does not have the shape that race realists claim that it does. Think in terms of contraposition. Sure, if race were real, then maybe it might make sense to talk about racial demographics. However, since race clearly is not real, any demographic correlations must be bogus. There is a much simpler explanation for why some skin colors seem socioeconomically advantaged: Because our society itself has bigoted opinions about skin colors, and has practices like redlining [4] which systematically oppress folks. [0] https://en.wikipedia.org/wiki/Hereditary_haemochromatosis https://en.wikipedia.org/wiki/Hereditary_haemochromatosis [1] https://en.wikipedia.org/wiki/Sickle_cell_disease https://en.wikipedia.org/wiki/Sickle_cell_disease [2] https://en.wikipedia.org/wiki/Tay%E2%80%93Sachs_disease https://en.wikipedia.org/wiki/Tay%E2%80%93Sachs_disease [3] https://en.wikipedia.org/wiki/Pedigree_collapse https://en.wikipedia.org/wiki/Pedigree_collapse [4] https://en.wikipedia.org/wiki/Redlining https://en.wikipedia.org/wiki/Redlining
joe_the_user 6y ago
No one would say that they believe in automating ethics if you pin them down and ask them honestly. But lots of institutional speak still talks as that's the goal. Some portion of Nick Bostrom's book effectively argues for something like that, for example.
coddle-hark 6y ago
Not sure what the author meant there but we do have a problem of offloading the ethical burden of our systems onto the algorithms themselves and it should be taken more seriously. Like when Google shipped an AI for moderating hate speech which turned out to be biased against minorities [1]. Google would have been sued if that bias had been coded in procedurally, but because they used "AI" their response was basically "we'll take another look at the training data". The damage caused is the same in both cases though, and it shouldn't take a team of independent researchers to prevent Google from shipping racist products. [1] https://fortune.com/2019/08/16/google-jigsaw-perspective-racial-bias/ https://fortune.com/2019/08/16/google-jigsaw-perspective-rac...
- ekianjo 6y ago
  > The damage caused is the same in both cases though, and it shouldn't take a team of independent researchers to prevent Google from shipping racist products. Your comment reads like "company X should not be shipping products with issues affecting users" - as long as it was not intentional and they take corrective actions, things are working as well as they can.
  - coddle-hark 6y ago
    With that logic we shouldn't need clinical trials for drugs or food additives either. Just ship it, if someone gets hurt we'll sort it out later! At this point it's well known that unfair biases can easily creep into these kind of systems. I don't think it's unreasonable to require products that could potentially cause much harm (e.g. online censoring, scoring CVs, vetting loan applications) to be tested for fairness before letting them loose on the public.
- smegma2 6y ago
  Forgive me for only reading the abstract. > The results show evidence of systematic racial bias in all datasets, as classifiers trained on them tend to predict that tweets written in African-American English are abusive at substantially higher rates. Wouldn't we need to know the true rate of abusive tweets by race in order to conclude this? Is this addressed at all, e.g. with human labelled data, or am I mistaken in thinking this is necessary?

willberman 6y ago

For any point of view on AI, I can probably find a decent number of non technical people who hold that POV. Technical illiteracy is a significant problem imo.

curiousgal 6y ago

There always seems to be a disconnect between tech philosophers and the technology they write about. Maybe they should try to actually work on 'AI' before preaching about it. To confirm your point a simple example is when working on an automated classifier for loan applications. One way to make the system more ethical, that is less descriminating based on certain features like race or gender, is to often lower the threshold of acceptance for the marginalized classes and raise it for the other ones. Anyone who had to do that sees how it poses a dilemma of equality vs fairness or even fairness vs cost. So indeed, ethics cannot be solved.

msla 6y ago
> One way to make the system more ethical, that is less descriminating based on certain features like race or gender, is to often lower the threshold of acceptance for the marginalized classes and raise it for the other ones. Or don't input those values at all. Is it even legal for a human to take those things into consideration? Regardless, the AI morally shouldn't, so there's no reason for the data to even be in the input.
- cmendel 6y ago
  Right; however, the problem is that ML is generally pretty good at finding underlying patterns in data, so even if you make an algorithm race-blind it can still construct a dummy variable that is effectively race. Unfortunately, it does this by codifying pre-existant social outcomes which are often not free of bias.
  - RobPfeifer 6y ago
    This is why disparate impact backtesting and go-forward monitoring should be done on all credit model releases
- hectormalot 6y ago
  As someone who works in this field: the challenge is more nuanced. For example: Even if you don’t input ethnicity as a value, you might input ZIP codes. ZIP codes however can be very predictive of someone’s ethnic background, and you might still end up unfairly discriminating against certain groups of people. The Amazon recruiting AI failure is another great example of this, I think it didn’t take gender into account, but discriminated against activities like ‘head of women’s association’ etc. Finally, what’s ethical is often up for discussion. Depending on context, people will optimize for equality (equal % of credits approved for both genders) vs helping those behind (Higher % approved for those historically disadvantaged) vs helping those that ‘deserve’ it of groups were unknown (finance such that default rates of both groups are expected to be equal, likely benefiting those with already good credit scores). I don’t think any of these answers is wrong, but they are mutually exclusive. One challenge I don’t see discussed that often is that the current system of many humans evaluating loan applications leads to ‘noise‘ in outcomes so that we (on average) don’t see that much bias. (Eg a few people of certain ethnicity not getting loans because a few people are biased against them) A single AI system scoring all applications however is consistent in its bias against a group, leading to more obvious effects (eg most people in a certain area not receiving loans because the algorithm is biased against it) In my opinion, an important part is the right on human review of meaningful automated decisions. Not mentioned that often, but it’s also part of the GDPR.
billjings 6y ago
This seems to be a limitation of the tool, yes?
RobPfeifer 6y ago
How would you do that without including a variable that raises a flag with the regulators? There are ways to improve access to credit for marginalized classes but this is generally done by adding variables that are independent of those related to protected classes.
behnamoh 6y ago
A few points: 1. I mean I'm all in favor of ethics, but it's like saying "I believe in God/god". Everyone seems to have their own interpretation of it. When people are talking about ethics/God/etc., how can they be sure that they're talking about the same thing? Centuries of debating over what is "ethical" has got us no where. Kant got close, but then there are caveats to his definition of ethics, too. 2. Who says AI should behave according to "our" ethics (whatever those are)? Sooner or later, with the emergence of ASI (artificial super intelligence) we'll have to accept the superior form of consciousness the ASI is. Expecting it to behave according to our rules is like monkeys asking us not to take away all the bananas because for them, it's unethical. 3. Even without AI, people have all sorts of biases and prejudices against certain ethnicities/races/colors/etc. You're not happy with the results that AI provides? Fine, hire some people instead and see where that leads you. The only thing you lose is the higher performance of AI. 4. Expecting AI systems to have no bias (while employees replaced by the AI certainly were prone to bias) is equivalent to expecting technology to change our world for the better. That doesn't always happen. You think smartphones are necessarily a good thing? Think about where they produce those gadgets and how they treat the workers. 5. At the end of the day, companies will stick with whatever "works" for them, regardless of what label we put on it (good/bad/ethical/fair/...). If it adds value, then most of the time it's enough. If being ethical (according to some definition) doesn't bring in more cash, companies are unlikely to invest in it. At least, with explainable AI, there is benefits for the firm to improve the accuracy of the AI in the long term. But pushing companies to adopt some ethical/unbiased/fair AI that doesn't add value for them is itself unethical (in my book).
- kijin 6y ago
  > we'll have to accept the superior form of consciousness the ASI is. On what definition of "have to"? Large numbers of people will not only refuse to accept the ASI's interpretation of ethics, they will also refuse to acknowledge that the purported ASI is any better than ourselves/themselves when it comes to ethics. If you try to settle the matter by giving the ASI the force of law, there will be armed revolts and violent crackdowns.
  - behnamoh 6y ago
yamrzou 6y ago
What if it turns out there is reason people discriminate using those features (for example that people of race X tend to fail repaying their loans), and the classifier just learned that reason from the data. Why should we be expected to "fix" the reality for the classifier?
- stanmancan 6y ago
  Because now people start to get rejected for loans because of their race, and that’s wrong for many obvious reasons.
  - yamrzou 6y ago
    I agree with you that it is generally wrong, i.e. that other factors should come first. But what if the data suggests it? For example if is known that if you lend to someone from race X, there is 80% chance he would not pay back. Would you not incorporate that factor into your decision?
    - stanmancan 6y ago
      No. Find the other, non-racist data points that correlate.
      - matheusmoreira 6y ago
        Why should correlations with ethnicity be dismissed? Epidemiological studies tell us some diseases disproportionally affect certain groups. Should doctors dismiss this knowledge as racist and refrain from screening patients in the affected groups because they might come off as racist?
        lidHanteyk 6y ago
        Yes. We know, more specifically, that genes are to blame for diseases like hemochromatosis [0], sickle cell anemia [1], or Tay-Sachs [2]. We also know, from pedigree collapse [3], that humans broadly form one single race. Therefore we know that correlations with any definition of ethnicity or race are spurious, because those definitions must be socially constructed, because the gene pool simply does not have the shape that race realists claim that it does. Think in terms of contraposition. Sure, if race were real, then maybe it might make sense to talk about racial demographics. However, since race clearly is not real, any demographic correlations must be bogus. There is a much simpler explanation for why some skin colors seem socioeconomically advantaged: Because our society itself has bigoted opinions about skin colors, and has practices like redlining [4] which systematically oppress folks. [0] https://en.wikipedia.org/wiki/Hereditary_haemochromatosis https://en.wikipedia.org/wiki/Hereditary_haemochromatosis [1] https://en.wikipedia.org/wiki/Sickle_cell_disease https://en.wikipedia.org/wiki/Sickle_cell_disease [2] https://en.wikipedia.org/wiki/Tay%E2%80%93Sachs_disease https://en.wikipedia.org/wiki/Tay%E2%80%93Sachs_disease [3] https://en.wikipedia.org/wiki/Pedigree_collapse https://en.wikipedia.org/wiki/Pedigree_collapse [4] https://en.wikipedia.org/wiki/Redlining https://en.wikipedia.org/wiki/Redlining

joe_the_user 6y ago

No one would say that they believe in automating ethics if you pin them down and ask them honestly. But lots of institutional speak still talks as that's the goal. Some portion of Nick Bostrom's book effectively argues for something like that, for example.

coddle-hark 6y ago

Not sure what the author meant there but we do have a problem of offloading the ethical burden of our systems onto the algorithms themselves and it should be taken more seriously. Like when Google shipped an AI for moderating hate speech which turned out to be biased against minorities [1]. Google would have been sued if that bias had been coded in procedurally, but because they used "AI" their response was basically "we'll take another look at the training data". The damage caused is the same in both cases though, and it shouldn't take a team of independent researchers to prevent Google from shipping racist products. [1] https://fortune.com/2019/08/16/google-jigsaw-perspective-racial-bias/ https://fortune.com/2019/08/16/google-jigsaw-perspective-rac...

ekianjo 6y ago
> The damage caused is the same in both cases though, and it shouldn't take a team of independent researchers to prevent Google from shipping racist products. Your comment reads like "company X should not be shipping products with issues affecting users" - as long as it was not intentional and they take corrective actions, things are working as well as they can.
- coddle-hark 6y ago
  With that logic we shouldn't need clinical trials for drugs or food additives either. Just ship it, if someone gets hurt we'll sort it out later! At this point it's well known that unfair biases can easily creep into these kind of systems. I don't think it's unreasonable to require products that could potentially cause much harm (e.g. online censoring, scoring CVs, vetting loan applications) to be tested for fairness before letting them loose on the public.
smegma2 6y ago
Forgive me for only reading the abstract. > The results show evidence of systematic racial bias in all datasets, as classifiers trained on them tend to predict that tweets written in African-American English are abusive at substantially higher rates. Wouldn't we need to know the true rate of abusive tweets by race in order to conclude this? Is this addressed at all, e.g. with human labelled data, or am I mistaken in thinking this is necessary?