10 ms·

>It's really only people where you can't tell what it does/is from the outside. Thing is, that's not even true in that it doesn't fully acknowledge the problem

>It's really only people where you can't tell what it does/is from the outside. Thing is, that's not even true in that it doesn't fully acknowledge the problem. You often CAN tell information from people's outward appearance, albeit probabilistically. Therein lies the problem: you can very easily train an algorithm to be maximally right according to your cost function, but end up biased because the underlying distributions aren't the same between groups. The issue is that as a society we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption.

um_ya 7y ago
In many cases, being logically correct is more important than being politically correct. For example: It may be politically correct to drive down a bad neighborhood in the middle of the night, but it isn't logically correct.
- seph-reed 7y ago
  True, but poorly stated. Which is to say, you aren't wrong, but you're missing the indications of nuance which are really, really important to the personal liberty of not being defined by superficial traits.
- munificent 7y ago
  It logically correct to do that when it's your neighborhood and you're driving home. But note the implicit bias in your own comment: you assume yourself and all the readers of the comment are not people who live in bad neighborhoods.
  - serf 7y ago
    >But note the implicit bias in your own comment: you assume yourself and all the readers of the comment are not people who live in bad neighborhoods. that assumption results simply from the need for 'bad neighborhood' to be a negative scoring action. if you oversimplify it to get rid of 'implicit bias' (which I don't agree exists in the example), the results turn into near-meaningless babble. "For example: It may be politically correct to do something that ignores statistical dangers in favor of the promise of human goodness, which may result in the possibility for more personal endangerment than other choices, but it isn't logically sound to ignore such statistics for the hope of a less biased personal experience." The example requires the person driving to be detached from the bad neighborhood that they have a choice to drive through. How that isn't an obvious requirement for the example to have merit is beyond me.
  - whenchamenia 7y ago
    While there may be some bias, it is as much yours. Even people who live in bad neighborhoods avoid driving through them when possible.
  - sershe 7y ago
    That is not technically even true. If you live in a bad neighborhood it's still probabilistically better to drive home thru a good neighborhood than thru another bad neighborhood.
  - wruza 7y ago
    Did you live in really bad neighborhood? I did. Out of two bus stops I always used one that required to cross two streets with no crosswalks. The other required to walk right through the middle of my lovely vicinity with 50% probability of giving something up to ‘charity’.
- tsimionescu 7y ago
  Well, it's especially in those situations that you need institutional controls to forbid the logically correct choice. After all, you don't really need a law telling companies they aren't allowed to hire infants as senior officers - it's already not in the company's interest to do so. However, when there is a logically correct but politically incorrect decision that the company could make, it is now that you need laws to prevent the company from taking that choice. Of course, as the weight of an institution's decisions goes down, so does the need to police its actions. In particular, it is rarely necessary to prevent an individual person from acting on their biases. Applying this to your example, if we had an AI that should suggest your best route home, and it avoided a short route through a bad neighborhood, that is likely ok. However, if a municipality used the same AI to decide where to prioritize changing street lights, that should be prohibited.
fao_ 7y ago
> The issue is that as a society we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption. Sorry, but it's not a decision. Science has found repeatedly that using outward characteristics does not work as a good classification measure. Society simply enforces not making bad judgements. See: Phrenology, Racism -- "appearance implies certain things about someone's character and intelligence" (https://en.wikipedia.org/wiki/Phrenology https://en.wikipedia.org/wiki/Phrenology) Sex -- "A person's gender presentation determines their gender or sex" (https://en.wikipedia.org/wiki/Transgender_history https://en.wikipedia.org/wiki/Transgender_history), "A person's genitalia determines their chromosones", "A person's chromosones determine their sex and their ability to give birth" (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2190741/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2190741/). Wealth -- "That person is wearing bad clothing/living an unassuming life therefore they must be poor" (https://www.npr.org/2018/12/29/680883772/social-worker-led-frugal-life-to-donate-nearly-11-million-to-childrens-charities https://www.npr.org/2018/12/29/680883772/social-worker-led-f...) Intelligence -- "This person scores badly on intelligence tests therefore they must not be worthwhile" (https://en.wikipedia.org/wiki/Richard_Feynman https://en.wikipedia.org/wiki/Richard_Feynman, https://www.psychologytoday.com/gb/blog/sudden-genius/201101/is-high-intelligence-necessary-be-genius https://www.psychologytoday.com/gb/blog/sudden-genius/201101...), "This person appears to be unskilled or incapable of many tasks therefore they must be totally unskilled" (http://en.wikipedia.org/wiki/Savant_syndrome http://en.wikipedia.org/wiki/Savant_syndrome)
- deogeo 7y ago
  > Science has found repeatedly that using outward characteristics does not work as a good classification measure. Science has found that, on average, it works [1]. Of course there will be many cases where it fails - that's how statistical inference works. Whether that makes it a 'good' measure, by whatever standard, is a different question. But there is no doubt information can be inferred from appearance. [1] http://www.spsp.org/news-center/blog/stereotype-accuracy-response http://www.spsp.org/news-center/blog/stereotype-accuracy-res...
- Enginerrrd 7y ago
brlewis 7y ago
> The issue is that as a society we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption. That sounds like the wrong basis for calling it extreme. It's not at all extreme to say that classifying an interview candidate based on correlated but non-causal characteristics is wrong, regardless of the statistical significance of those correlations.
- Enginerrrd 7y ago
  I just mean that I'd guess most stereotypes have a correlation nowhere near 0.50, but it'd be wrong to use them to screen candidates even if it was 0.50 or greater. I used the word "extreme" to convey the sense that such a scenario of stereotype accuracy is very unlikely.
  - bgilroy26 7y ago
    Most stereotypes are lagging indicators that do not tell us about the future, as the history of Fenty Beauty makes clear.
    - Noumenon72 7y ago
      I don't believe Fenty Beauty has been discussed on Hacker News before seeing as how it's a line of cosmetics, so maybe you could expand on what you know about its history that others may not have been following in as much detail?
      - bgilroy26 7y ago
        In 2017, Fenty Beauty found an underserved market (dark-skinned women and high-quality makeup) and made a killing selling them what they wanted. If you relied solely on history and stereotypes, you would believe nobody could make $500 million dollars that way. https://www.fool.com/investing/2019/05/15/rihannas-fenty-is-now-lvmhs-newest-growth-engine.aspx https://www.fool.com/investing/2019/05/15/rihannas-fenty-is-...
nostrademons 7y ago
See also Stucchio's impossibility theorem: it is impossible for a given algorithm to be procedurally fair, representationally fair, and utilitarian at the same time: Video: https://www.youtube.com/watch?v=Zn7oWIhFffs https://www.youtube.com/watch?v=Zn7oWIhFffs Slides: https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/slides.pdf https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl...
- AstralStorm 7y ago
  The general point is that you have to robustly compromise and satisfice all the goals. People tend to be rather good at it when taken as a group. (Any particular person may be bad at a given subset of all problems.) It is a kind of optimality condition on all three goals. The robustness additionally means that should conditions change, the algorithm usually will become better not worse and should a degradation still happen, it will be graceful and not catastrophical. It's a hard and open problem in ML and especially ANN, design of robust solutions in the space. Most have really bad problems with it even when debiased.
  - sgt101 7y ago
    Hello - do you have a good reference to this area? I bang on about similar ideas whenever allowed, but haven't found good support in the literature. Not that I mind that too much!
- missosoup 7y ago
  Good presentation
lone_haxx0r 7y ago
> we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption. Ironically, the only places where it's legally prohibited or frowned upon to use these heuristic techniques are situations that people have arbitrarily (heuristically or conveniently) decided. For example: It's "not fair" to hire someone because they're white (and consequently have a higher chance of being wealthy and hence a higher change of being educated.) But it's "fair" to choose a love partner based on their height, their waist-to-hip ratio, their weight (and hence having a higher chance of giving birth to healthy offspring, better physical protection, etc.). Maybe it's hypocritical, and I don't know if that's a good thing or not. Maybe being hypocritical helps us survive.
- mrkurt 7y ago
  That's not very arbitrary. If you're hiring someone, you're always in a position of power. If you're dating someone, there's no power differential (or if there is, that's a problem all by itself).
  - StavrosK 7y ago
    How are you "always in a position of power" when hiring someone? That's only true when there's more supply than demand, and it's the opposite in markets where the candidates get multiple high-quality offers to choose from.
    - untog 7y ago
      Because you are paying that person money and have the ability to fire them. In the US you're also probably providing their health care. I get what you're saying, but no one moves jobs every week. The sunk costs of switching employment are significant for the employee, less so for the company.
      - Double_a_92 7y ago
        They provide you valuable work in exchange for that... Internally you are probably imagining some big corp that can pick from 100s of replaceable workers.
bsanr2 7y ago
>The issue is that as a society we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption. This is likely due to an acknowledgment of the limits of human models to account for the full context surrounding correlated-but-non-causal classifications, such that conclusions drawn from them can have unforeseen or highly detrimental ramifications. Speaking to race in America specifically, the schema through which we judge people are highly susceptible to bias from the white supremacist bent of historical education and general discourse. This is how you end up with cycles like those within the justice system (pushed in part by sentencing software), wherein black defendents are assumed to have a higher likelihood of re-offending, therefore increasing the likelihood of any given black defendent not receiving bond or having a lengthy sentence if convicted. After all, blackness correlates with recidivism. Lying outside this correlative relationship is the likely causal relationships of longer stays in jail and lack of access to employment opportunities, which disproportionately affect black people, causing higher rates of recidivism, regardless of race.
- AstralStorm 7y ago
  You have other measures than prison. You can still have enhanced vigilance without enhanced annoyance and mistakes. There's often a superior choice lurking that nobody is thinking about, sometimes expensive, sometimes not, seemingly unrelated to such optimization. This is why ML is not intelligence, it cannot find new solutions you're not already looking for. The main problem of the judicial and police system is it tries to be procedurally fair and still fails at it anyway.
  - bsanr2 7y ago
    I'd counter that in many cases it doesn't even try to be fair. It privileges the ability to craft an argument over bare facts, which immediately privileges those who can afford professional representation. At the core of the fear of a surveillance state isn't simply the loss of privacy (which in and of itself could be worth the accuracy it would bring to judicial proceedings), but the fact that it would just bolster the ability of skilled narrative-builders to pull the most advantageous facts out of context and twist them to their whims.
foldingmoney 7y ago
>You often CAN tell information from people's outward appearance, albeit probabilistically. Therein lies the problem: you can very easily train an algorithm to be maximally right according to your cost function, but end up biased because the underlying distributions aren't the same between groups. In fact, an AI trained on aggregate data to probabilistically infer characteristics about individuals is _literally a stereotyping machine_. If people are upset that their stereotyping machine stereotypes people, they probably didn't fully understand what they were doing when they built it, because this is not a design flaw -- it's the design.
- AstralStorm 7y ago
  AI is very definitely not procedurally fair unless you can explain all behaviors. Which is why designers generally don't know what they built.
- learnstats2 7y ago
  Maybe people are upset that stereotyping machines are being given harder powers to make consequential decisions about individuals.
  - wolfgke 7y ago
    > Maybe people are upset that stereotyping machines are being given harder powers to make consequential decisions about individuals. Then these people should argue against AI instead of swinging the racism cudgel.
    - acollins1331 7y ago
      It could just be a path forward. In America a lot of the time we can't get any progress unless it's a racism issue. Things like gerrymandering or marijuana legalization get their first pushes because the most blatant group that suffers are POCs. The fact that it harms everyone is a little lost on most people, but in the end the racism cudgel can be effective for positive societal change. In this case, we can use it to get rid of automatically being identified for crimes or whatever by an AI.
sgt101 7y ago
I'd just flag that "EVEN in the extreme that you're right more often than you're wrong" is not a meaningful line (if someone wanted to apply it) because the fact is that it's not the number of wrong calls or right calls that matter here - it's which ones and when. For example the project under discussion classifies a photo of my mother as a young woman as a "flibbertigibbet", which is amusing, as a way to tease her now (she is a 70 year old ex prison and probation officer and needs no protection from teasing). However, had this been done to my daughter and the same result obtained (in fact the result was "blue" but there you go) by someone assessing her potential as a candidate for university - well there's damage.
thiccly 7y ago
The fact is that we live in a society
tmalsburg2 7y ago
> EVEN in the extreme that you're right more often than you're wrong if you make that assumption. And this is actually the rational thing to do. Reason: There are two potential errors involved here: labeling an innocent person a criminal (false positive) and labeling a criminal as innocent (false negative). The key is to realize that the cost of these two errors is not the same. For instance, treating an innocent person as a criminal could be much more expensive for the person and society than not detecting a criminal with a given classifier. For that very reason, we have the presumption of innocence as a principle in law. As a consequence, you don't want to select a classifier based on just the rate of errors overall, but you want to incorporate some kind of loss function that minimizes the cost for individuals and society. Under that loss function, the best classifier may actually be wrong more often than some other classifier.

um_ya 7y ago

In many cases, being logically correct is more important than being politically correct. For example: It may be politically correct to drive down a bad neighborhood in the middle of the night, but it isn't logically correct.

seph-reed 7y ago
True, but poorly stated. Which is to say, you aren't wrong, but you're missing the indications of nuance which are really, really important to the personal liberty of not being defined by superficial traits.
munificent 7y ago
It logically correct to do that when it's your neighborhood and you're driving home. But note the implicit bias in your own comment: you assume yourself and all the readers of the comment are not people who live in bad neighborhoods.
- serf 7y ago
  >But note the implicit bias in your own comment: you assume yourself and all the readers of the comment are not people who live in bad neighborhoods. that assumption results simply from the need for 'bad neighborhood' to be a negative scoring action. if you oversimplify it to get rid of 'implicit bias' (which I don't agree exists in the example), the results turn into near-meaningless babble. "For example: It may be politically correct to do something that ignores statistical dangers in favor of the promise of human goodness, which may result in the possibility for more personal endangerment than other choices, but it isn't logically sound to ignore such statistics for the hope of a less biased personal experience." The example requires the person driving to be detached from the bad neighborhood that they have a choice to drive through. How that isn't an obvious requirement for the example to have merit is beyond me.
- whenchamenia 7y ago
  While there may be some bias, it is as much yours. Even people who live in bad neighborhoods avoid driving through them when possible.
- sershe 7y ago
  That is not technically even true. If you live in a bad neighborhood it's still probabilistically better to drive home thru a good neighborhood than thru another bad neighborhood.
- wruza 7y ago
  Did you live in really bad neighborhood? I did. Out of two bus stops I always used one that required to cross two streets with no crosswalks. The other required to walk right through the middle of my lovely vicinity with 50% probability of giving something up to ‘charity’.
tsimionescu 7y ago
Well, it's especially in those situations that you need institutional controls to forbid the logically correct choice. After all, you don't really need a law telling companies they aren't allowed to hire infants as senior officers - it's already not in the company's interest to do so. However, when there is a logically correct but politically incorrect decision that the company could make, it is now that you need laws to prevent the company from taking that choice. Of course, as the weight of an institution's decisions goes down, so does the need to police its actions. In particular, it is rarely necessary to prevent an individual person from acting on their biases. Applying this to your example, if we had an AI that should suggest your best route home, and it avoided a short route through a bad neighborhood, that is likely ok. However, if a municipality used the same AI to decide where to prioritize changing street lights, that should be prohibited.

fao_ 7y ago

> The issue is that as a society we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption. Sorry, but it's not a decision. Science has found repeatedly that using outward characteristics does not work as a good classification measure. Society simply enforces not making bad judgements. See: Phrenology, Racism -- "appearance implies certain things about someone's character and intelligence" (https://en.wikipedia.org/wiki/Phrenology https://en.wikipedia.org/wiki/Phrenology) Sex -- "A person's gender presentation determines their gender or sex" (https://en.wikipedia.org/wiki/Transgender_history https://en.wikipedia.org/wiki/Transgender_history), "A person's genitalia determines their chromosones", "A person's chromosones determine their sex and their ability to give birth" (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2190741/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2190741/). Wealth -- "That person is wearing bad clothing/living an unassuming life therefore they must be poor" (https://www.npr.org/2018/12/29/680883772/social-worker-led-frugal-life-to-donate-nearly-11-million-to-childrens-charities https://www.npr.org/2018/12/29/680883772/social-worker-led-f...) Intelligence -- "This person scores badly on intelligence tests therefore they must not be worthwhile" (https://en.wikipedia.org/wiki/Richard_Feynman https://en.wikipedia.org/wiki/Richard_Feynman, https://www.psychologytoday.com/gb/blog/sudden-genius/201101/is-high-intelligence-necessary-be-genius https://www.psychologytoday.com/gb/blog/sudden-genius/201101...), "This person appears to be unskilled or incapable of many tasks therefore they must be totally unskilled" (http://en.wikipedia.org/wiki/Savant_syndrome http://en.wikipedia.org/wiki/Savant_syndrome)

deogeo 7y ago
> Science has found repeatedly that using outward characteristics does not work as a good classification measure. Science has found that, on average, it works [1]. Of course there will be many cases where it fails - that's how statistical inference works. Whether that makes it a 'good' measure, by whatever standard, is a different question. But there is no doubt information can be inferred from appearance. [1] http://www.spsp.org/news-center/blog/stereotype-accuracy-response http://www.spsp.org/news-center/blog/stereotype-accuracy-res...
Enginerrrd 7y ago

brlewis 7y ago

> The issue is that as a society we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption. That sounds like the wrong basis for calling it extreme. It's not at all extreme to say that classifying an interview candidate based on correlated but non-causal characteristics is wrong, regardless of the statistical significance of those correlations.

Enginerrrd 7y ago
I just mean that I'd guess most stereotypes have a correlation nowhere near 0.50, but it'd be wrong to use them to screen candidates even if it was 0.50 or greater. I used the word "extreme" to convey the sense that such a scenario of stereotype accuracy is very unlikely.
- bgilroy26 7y ago
  Most stereotypes are lagging indicators that do not tell us about the future, as the history of Fenty Beauty makes clear.
  - Noumenon72 7y ago
    I don't believe Fenty Beauty has been discussed on Hacker News before seeing as how it's a line of cosmetics, so maybe you could expand on what you know about its history that others may not have been following in as much detail?
    - bgilroy26 7y ago
      In 2017, Fenty Beauty found an underserved market (dark-skinned women and high-quality makeup) and made a killing selling them what they wanted. If you relied solely on history and stereotypes, you would believe nobody could make $500 million dollars that way. https://www.fool.com/investing/2019/05/15/rihannas-fenty-is-now-lvmhs-newest-growth-engine.aspx https://www.fool.com/investing/2019/05/15/rihannas-fenty-is-...

nostrademons 7y ago

See also Stucchio's impossibility theorem: it is impossible for a given algorithm to be procedurally fair, representationally fair, and utilitarian at the same time: Video: https://www.youtube.com/watch?v=Zn7oWIhFffs https://www.youtube.com/watch?v=Zn7oWIhFffs Slides: https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/slides.pdf https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl...

AstralStorm 7y ago
The general point is that you have to robustly compromise and satisfice all the goals. People tend to be rather good at it when taken as a group. (Any particular person may be bad at a given subset of all problems.) It is a kind of optimality condition on all three goals. The robustness additionally means that should conditions change, the algorithm usually will become better not worse and should a degradation still happen, it will be graceful and not catastrophical. It's a hard and open problem in ML and especially ANN, design of robust solutions in the space. Most have really bad problems with it even when debiased.
- sgt101 7y ago
  Hello - do you have a good reference to this area? I bang on about similar ideas whenever allowed, but haven't found good support in the literature. Not that I mind that too much!
missosoup 7y ago
Good presentation

lone_haxx0r 7y ago

> we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption. Ironically, the only places where it's legally prohibited or frowned upon to use these heuristic techniques are situations that people have arbitrarily (heuristically or conveniently) decided. For example: It's "not fair" to hire someone because they're white (and consequently have a higher chance of being wealthy and hence a higher change of being educated.) But it's "fair" to choose a love partner based on their height, their waist-to-hip ratio, their weight (and hence having a higher chance of giving birth to healthy offspring, better physical protection, etc.). Maybe it's hypocritical, and I don't know if that's a good thing or not. Maybe being hypocritical helps us survive.

mrkurt 7y ago
That's not very arbitrary. If you're hiring someone, you're always in a position of power. If you're dating someone, there's no power differential (or if there is, that's a problem all by itself).
- StavrosK 7y ago
  How are you "always in a position of power" when hiring someone? That's only true when there's more supply than demand, and it's the opposite in markets where the candidates get multiple high-quality offers to choose from.
  - untog 7y ago
    Because you are paying that person money and have the ability to fire them. In the US you're also probably providing their health care. I get what you're saying, but no one moves jobs every week. The sunk costs of switching employment are significant for the employee, less so for the company.
    - Double_a_92 7y ago
      They provide you valuable work in exchange for that... Internally you are probably imagining some big corp that can pick from 100s of replaceable workers.

bsanr2 7y ago

>The issue is that as a society we've (mostly) decided that unfairly classifying someone based on correlated but non-causal characteristics is wrong, EVEN in the extreme that you're right more often than you're wrong if you make that assumption. This is likely due to an acknowledgment of the limits of human models to account for the full context surrounding correlated-but-non-causal classifications, such that conclusions drawn from them can have unforeseen or highly detrimental ramifications. Speaking to race in America specifically, the schema through which we judge people are highly susceptible to bias from the white supremacist bent of historical education and general discourse. This is how you end up with cycles like those within the justice system (pushed in part by sentencing software), wherein black defendents are assumed to have a higher likelihood of re-offending, therefore increasing the likelihood of any given black defendent not receiving bond or having a lengthy sentence if convicted. After all, blackness correlates with recidivism. Lying outside this correlative relationship is the likely causal relationships of longer stays in jail and lack of access to employment opportunities, which disproportionately affect black people, causing higher rates of recidivism, regardless of race.

AstralStorm 7y ago
You have other measures than prison. You can still have enhanced vigilance without enhanced annoyance and mistakes. There's often a superior choice lurking that nobody is thinking about, sometimes expensive, sometimes not, seemingly unrelated to such optimization. This is why ML is not intelligence, it cannot find new solutions you're not already looking for. The main problem of the judicial and police system is it tries to be procedurally fair and still fails at it anyway.
- bsanr2 7y ago
  I'd counter that in many cases it doesn't even try to be fair. It privileges the ability to craft an argument over bare facts, which immediately privileges those who can afford professional representation. At the core of the fear of a surveillance state isn't simply the loss of privacy (which in and of itself could be worth the accuracy it would bring to judicial proceedings), but the fact that it would just bolster the ability of skilled narrative-builders to pull the most advantageous facts out of context and twist them to their whims.

foldingmoney 7y ago

>You often CAN tell information from people's outward appearance, albeit probabilistically. Therein lies the problem: you can very easily train an algorithm to be maximally right according to your cost function, but end up biased because the underlying distributions aren't the same between groups. In fact, an AI trained on aggregate data to probabilistically infer characteristics about individuals is _literally a stereotyping machine_. If people are upset that their stereotyping machine stereotypes people, they probably didn't fully understand what they were doing when they built it, because this is not a design flaw -- it's the design.

AstralStorm 7y ago
AI is very definitely not procedurally fair unless you can explain all behaviors. Which is why designers generally don't know what they built.
learnstats2 7y ago
Maybe people are upset that stereotyping machines are being given harder powers to make consequential decisions about individuals.
- wolfgke 7y ago
  > Maybe people are upset that stereotyping machines are being given harder powers to make consequential decisions about individuals. Then these people should argue against AI instead of swinging the racism cudgel.
  - acollins1331 7y ago
    It could just be a path forward. In America a lot of the time we can't get any progress unless it's a racism issue. Things like gerrymandering or marijuana legalization get their first pushes because the most blatant group that suffers are POCs. The fact that it harms everyone is a little lost on most people, but in the end the racism cudgel can be effective for positive societal change. In this case, we can use it to get rid of automatically being identified for crimes or whatever by an AI.

sgt101 7y ago

I'd just flag that "EVEN in the extreme that you're right more often than you're wrong" is not a meaningful line (if someone wanted to apply it) because the fact is that it's not the number of wrong calls or right calls that matter here - it's which ones and when. For example the project under discussion classifies a photo of my mother as a young woman as a "flibbertigibbet", which is amusing, as a way to tease her now (she is a 70 year old ex prison and probation officer and needs no protection from teasing). However, had this been done to my daughter and the same result obtained (in fact the result was "blue" but there you go) by someone assessing her potential as a candidate for university - well there's damage.

thiccly 7y ago

The fact is that we live in a society

tmalsburg2 7y ago

> EVEN in the extreme that you're right more often than you're wrong if you make that assumption. And this is actually the rational thing to do. Reason: There are two potential errors involved here: labeling an innocent person a criminal (false positive) and labeling a criminal as innocent (false negative). The key is to realize that the cost of these two errors is not the same. For instance, treating an innocent person as a criminal could be much more expensive for the person and society than not detecting a criminal with a given classifier. For that very reason, we have the presumption of innocence as a principle in law. As a consequence, you don't want to select a classifier based on just the rate of errors overall, but you want to incorporate some kind of loss function that minimizes the cost for individuals and society. Under that loss function, the best classifier may actually be wrong more often than some other classifier.