8 ms·

This is fundamentally a forest/trees type of situation. LeCun sees the issue with this one model, says, "If this had been trained on a different sample it would

by Kapura 6y ago

This is fundamentally a forest/trees type of situation. LeCun sees the issue with this one model, says, "If this had been trained on a different sample it wouldn't have this issue," and stops his train of reasoning there. The problem he is seeing is the mis-trained model, and nothing more. But the problem is larger than this single model, because this issue (or similar ones) are pervasive in the fields in which AI are being employed. If a neural net is helping a court hand down sentences, it is going to be trained on historical sentencing data, and will in turn reflect the biases present in that data. If you are still only seeing the one tree, you say "well we must correct for the historical bias," and absolve yourself of thinking of the larger problem. That forest problem is that we will always be feeding these algorithms biased inputs, unless we do the work to understand social biases and attempt to rectify them.

visarga 6y ago
The court example was good. Trying to make a list of possible domains where social bias is a factor: - court risk assessments - loan risk assessment - job application pre-screening - law enforcement face recognition - medical scan interpretation ... So, in essence: justice, banking, jobs, policing and health care. Anything else? Seems like social bias can only affect a small proportion of the ML application domains.
- trobertson 6y ago
  The classic (multi-decade problem repeated again, and again, and again...) is face detection. Face detection models continue to get trained by primarily white developers/researches who continue to use primarily white people as their training set. And then they're shocked when their model performs terribly for everyone who isn't of European decent. This case is so pervasive that it gets taught in basically every university that teaches ML - and yet those students go on to repeat the exact problem they were educated against.
  - visarga 6y ago
    > Face detection models continue to get trained by primarily white developers/researches who continue to use primarily white people as their training set. I guess you didn't watch Asian developments in face recognition closely in the last few years.
    - aparsons 6y ago
      Let’s pause all ML research until we have 50% black representation!
  - tarsinge 6y ago
    A lighthearted example: https://www.kotaku.co.uk/2017/01/12/how-we-accidentally-made-a-racist-videogame https://www.kotaku.co.uk/2017/01/12/how-we-accidentally-made...
  - spoopyskelly 6y ago
    Does the face recognition tech in China have big problems with european faces?
- Udik 6y ago
  In this case, ironically, it seems that an AI could incorrectly depixelate faces of suspects caught by surveillance cameras to white instead of black. Which would lead to a bias against white people.
  - visarga 6y ago
    Please take a look at the PULSE model card here: https://thegradient.pub/content/images/2020/06/image.png https://thegradient.pub/content/images/2020/06/image.png PULSE is published as an art project, not suitable for face recognition or upscaling. It can only generate 'imaginary faces'. They didn't even train the GAN they were using, it was borrowed from another paper. They probably used StyleGAN because it was a nice high quality generator and they invented a novel way to use GANs so they needed a toy model to showcase their algo.
    - Udik 6y ago
      Agreed. I was only pointing out that even bias is a matter of bias: if white faces had been depixelated to black faces instead of the other way around, the authors could have still been accused of a racist bias (because of the hypothetical scenario I've described above).
- mratsim 6y ago
  It's even worse than that. Assume you want to train an AI to recognize shops or buildings for example for a Google car. Well if you do it in the US with skyscraper, in Europe, in Africa, Middle-East or Asia, you will get completely different results and biases. Also, I don't see how anyone has the resources to compensate such a social bias, unless they plan to do a shooting trip in hard to access locations and try to justify such as "Seriously officer, the reason I take all those photos is to make sure my machine learning model is unbiaised so I don't classify shops in your country as shacks." And if we forget human activity, even looking at nature, the fauna and flora are different between areas their color, how sparse they are, etc. Last example, applying sparsity to human activity, what is actually a town in some country might be classified as a settlement for example due to bias.
Wolfenstein98k 6y ago
It's not the job of the AI researcher to solve "social biases" in every field, it's their job to build the AI. LeCun is right to focus on his task and correct it, not just start talking about structural issues to which he cannot have insight anyway. PS "Do the work" is a creepy phrase that is popping up everywhere in SocJus. I recommend describing the "work" that needs "doing" instead of just saying "the work" need be "done".
- staplers 6y ago
  This is like saying it's not a civil engineers job to solve the geography's faults. If you can't build a safe and balanced bridge, then don't. It has real world consequences that can drastically damage communities.
  - _Microft 6y ago
    A civil engineer should be able to do that (or decline to do it if they can not) and that's why it is their job and not the geologist's job to build the bridge. Research != engineering.
  - dzhiurgis 6y ago
    More like it's not concrete engineers job to find concrete mix that fits every possible usage case.
  - jollofricepeas 6y ago
    Another example... The current nightmare that is cyber security is caused by developers who do not understand that with great power comes great responsibility. The culture of software “engineering” and development is fundamentally broken and based entirely on “its not my problem if someone else gets hurt, I just build things.” There’s literally not a single other industry where this level of greed and willful neglect is acceptable. Of course, this would be reflected in the AI community as well.
    - Nasrudith 6y ago
      That shows a fundamental lack of understanding of the fields of software engineering, cyber security and the fields. Both terms are individually overloaded such that many issues aren't even within their domain let alone actually handled by developers! No matter how good the lock is it does no good when the key is left on /top of tbe doormat/ by the user. Blaming greed of engineers for security problems? Seriously?! That is like railing against EMTs as being responsible for the Coronavirus because they wanted to get rich without working hard. It conflates so many different areas and roles that it isn't even coherent logic and is nonsensical.
SI_Rob 6y ago
This issue reminds me of the IQ testing and measurement problem of regression bias, which likewise arises from statistical artifacts from decades past (when tests were administered to much more narrow and self-selecting groups of subjects than today). The means calculated using much less diverse series of historical cohorts are still being fed back in to present IQ calculations, which thus retain a persistent corresponding "echo" of the biases in favor of those early test results. The echo is gradually fading, but some say a clean reboot of the baselines is the right way to resolve historical sample biases which still skew IQ testing away from an accurate modeling of the diversity in present test populations.
- travisoneill1 6y ago
  Why does diversity matter here? They are just feeding the scores into the model, right?
joe_the_user 6y ago
The whole thing is an illustration of what a mess Twitter is. Sorting out what LeCun's overall message is seems next to impossible given the patchwork of ten line messages from who knows who. But I just wanted to say that in the instance of a neural net that grants bail or not, there's an issue beyond either biased data or the neural reproducing previously biased opinions. The modern notion of fairness implies that the individual be judged based on their personal merits rather than things roughly correlated with their surface characteristics. Being black is correlated with being poor and being poor is correlated with being a criminal and various other bad behaviors. But that doesn't mean it's fair to punish a given individual, who is only responsible for themselves, for such surface characteristic. Which is to say that if an AI crunches the numbers in a objective fashion with the aim to make decisions based on various correlations, that can fundamentally problematic regardless of the bias of the original data or people.
- thaumasiotes 6y ago
  > Being black is correlated with being poor and being poor is correlated with being a criminal and various other bad behaviors. Not a great example; the direct correlation between being black and being a criminal is much, much larger than the indirect correlation from being black to being a criminal through being poor. Which is to say, if blacks were criminal at the rate predicted by their poverty rate, they would be much, much less criminal than they actually are.
  - learnstats2 6y ago
    > Which is to say,... Since there is no point of comparison, this is not a valid statistical conclusion and is just a racist framing.
    - thaumasiotes 6y ago
      What do you mean? "If blacks were criminal at the rate predicted by their poverty rate, they would be much, much less criminal than they actually are" is a pure empirical observation. The logical point of comparison would be the nonblack population, but the conclusion is just as true for the entire population.
deleted 6y ago
[deleted]
randyrand 6y ago
Sure, but using the Duke paper example to illustrate that is clearly dumb. The toy AI network was designed that way. Yann was right to call it out.
nailer 6y ago
This is such a false binary. Of course we must correct for historical bias. Of course we must also proactively look for biases.
mnky9800n 6y ago
All input is biased. It is not our job as scientists to remove biases from a data set. It is our job to reduce bias and characterize bias, but it's impossible to have a data set that has zero bias. You cannot sample the whole universe.
scoutt 6y ago
> larger problem For a scientist doing a ML system to reconstruct pixelated faces, trained with white faces, why is he/she responsible for "insert larger problem" outside of her/his field? Do he/she has to also care about the brutal Tantalum Wars because there are some in the electronics they use? As far as I can see, ML is pretty new, and there is a lot of room for improvement. And I think people need to stop thinking the entire world is racist, or doesn't care or don't want to improve things. Changes take time, and won't happen today, or tomorrow or the next decade. There is a constant, furious need to point fingers, followed by fear of telling or not telling just the right words on the right order to the right audience. This is not how we are going to solve anything.
- foldr 6y ago
  You know I'm fine with that attitude if it's combined with actual humility. Often as not, however, these high profile guys are shooting off on Twitter about every issue under the sun. But when it comes to racism, suddenly they're just narrow technicians and it's nothing to do with them.
  - scoutt 6y ago
    I don't know if it is about racism. I think it's normal to become defensive when accused. If I say "your ML model is garbage" I am sure they will (maybe, if I am high-profile too) provide an explanation on why or why not.
    - foldr 6y ago
      This long twitter thread about racism isn't about racism?
      - scoutt 6y ago
        Sorry, I expressed it wrong. I wanted to say that when someone is accused of something, including of being a racist or something related to racism, it's normal to be defensive about it and take that posture. I don't think the researcher was being defensive just because it was about racism.

by Kapura 6y ago

visarga 6y ago

The court example was good. Trying to make a list of possible domains where social bias is a factor: - court risk assessments - loan risk assessment - job application pre-screening - law enforcement face recognition - medical scan interpretation ... So, in essence: justice, banking, jobs, policing and health care. Anything else? Seems like social bias can only affect a small proportion of the ML application domains.

trobertson 6y ago
The classic (multi-decade problem repeated again, and again, and again...) is face detection. Face detection models continue to get trained by primarily white developers/researches who continue to use primarily white people as their training set. And then they're shocked when their model performs terribly for everyone who isn't of European decent. This case is so pervasive that it gets taught in basically every university that teaches ML - and yet those students go on to repeat the exact problem they were educated against.
- visarga 6y ago
  > Face detection models continue to get trained by primarily white developers/researches who continue to use primarily white people as their training set. I guess you didn't watch Asian developments in face recognition closely in the last few years.
  - aparsons 6y ago
    Let’s pause all ML research until we have 50% black representation!
- tarsinge 6y ago
  A lighthearted example: https://www.kotaku.co.uk/2017/01/12/how-we-accidentally-made-a-racist-videogame https://www.kotaku.co.uk/2017/01/12/how-we-accidentally-made...
- spoopyskelly 6y ago
  Does the face recognition tech in China have big problems with european faces?
Udik 6y ago
In this case, ironically, it seems that an AI could incorrectly depixelate faces of suspects caught by surveillance cameras to white instead of black. Which would lead to a bias against white people.
- visarga 6y ago
  Please take a look at the PULSE model card here: https://thegradient.pub/content/images/2020/06/image.png https://thegradient.pub/content/images/2020/06/image.png PULSE is published as an art project, not suitable for face recognition or upscaling. It can only generate 'imaginary faces'. They didn't even train the GAN they were using, it was borrowed from another paper. They probably used StyleGAN because it was a nice high quality generator and they invented a novel way to use GANs so they needed a toy model to showcase their algo.
  - Udik 6y ago
    Agreed. I was only pointing out that even bias is a matter of bias: if white faces had been depixelated to black faces instead of the other way around, the authors could have still been accused of a racist bias (because of the hypothetical scenario I've described above).
mratsim 6y ago
It's even worse than that. Assume you want to train an AI to recognize shops or buildings for example for a Google car. Well if you do it in the US with skyscraper, in Europe, in Africa, Middle-East or Asia, you will get completely different results and biases. Also, I don't see how anyone has the resources to compensate such a social bias, unless they plan to do a shooting trip in hard to access locations and try to justify such as "Seriously officer, the reason I take all those photos is to make sure my machine learning model is unbiaised so I don't classify shops in your country as shacks." And if we forget human activity, even looking at nature, the fauna and flora are different between areas their color, how sparse they are, etc. Last example, applying sparsity to human activity, what is actually a town in some country might be classified as a settlement for example due to bias.

Wolfenstein98k 6y ago

It's not the job of the AI researcher to solve "social biases" in every field, it's their job to build the AI. LeCun is right to focus on his task and correct it, not just start talking about structural issues to which he cannot have insight anyway. PS "Do the work" is a creepy phrase that is popping up everywhere in SocJus. I recommend describing the "work" that needs "doing" instead of just saying "the work" need be "done".

staplers 6y ago
This is like saying it's not a civil engineers job to solve the geography's faults. If you can't build a safe and balanced bridge, then don't. It has real world consequences that can drastically damage communities.
- _Microft 6y ago
  A civil engineer should be able to do that (or decline to do it if they can not) and that's why it is their job and not the geologist's job to build the bridge. Research != engineering.
- dzhiurgis 6y ago
  More like it's not concrete engineers job to find concrete mix that fits every possible usage case.
- jollofricepeas 6y ago
  Another example... The current nightmare that is cyber security is caused by developers who do not understand that with great power comes great responsibility. The culture of software “engineering” and development is fundamentally broken and based entirely on “its not my problem if someone else gets hurt, I just build things.” There’s literally not a single other industry where this level of greed and willful neglect is acceptable. Of course, this would be reflected in the AI community as well.
  - Nasrudith 6y ago
    That shows a fundamental lack of understanding of the fields of software engineering, cyber security and the fields. Both terms are individually overloaded such that many issues aren't even within their domain let alone actually handled by developers! No matter how good the lock is it does no good when the key is left on /top of tbe doormat/ by the user. Blaming greed of engineers for security problems? Seriously?! That is like railing against EMTs as being responsible for the Coronavirus because they wanted to get rich without working hard. It conflates so many different areas and roles that it isn't even coherent logic and is nonsensical.

SI_Rob 6y ago

This issue reminds me of the IQ testing and measurement problem of regression bias, which likewise arises from statistical artifacts from decades past (when tests were administered to much more narrow and self-selecting groups of subjects than today). The means calculated using much less diverse series of historical cohorts are still being fed back in to present IQ calculations, which thus retain a persistent corresponding "echo" of the biases in favor of those early test results. The echo is gradually fading, but some say a clean reboot of the baselines is the right way to resolve historical sample biases which still skew IQ testing away from an accurate modeling of the diversity in present test populations.

travisoneill1 6y ago
Why does diversity matter here? They are just feeding the scores into the model, right?

joe_the_user 6y ago

The whole thing is an illustration of what a mess Twitter is. Sorting out what LeCun's overall message is seems next to impossible given the patchwork of ten line messages from who knows who. But I just wanted to say that in the instance of a neural net that grants bail or not, there's an issue beyond either biased data or the neural reproducing previously biased opinions. The modern notion of fairness implies that the individual be judged based on their personal merits rather than things roughly correlated with their surface characteristics. Being black is correlated with being poor and being poor is correlated with being a criminal and various other bad behaviors. But that doesn't mean it's fair to punish a given individual, who is only responsible for themselves, for such surface characteristic. Which is to say that if an AI crunches the numbers in a objective fashion with the aim to make decisions based on various correlations, that can fundamentally problematic regardless of the bias of the original data or people.

thaumasiotes 6y ago
> Being black is correlated with being poor and being poor is correlated with being a criminal and various other bad behaviors. Not a great example; the direct correlation between being black and being a criminal is much, much larger than the indirect correlation from being black to being a criminal through being poor. Which is to say, if blacks were criminal at the rate predicted by their poverty rate, they would be much, much less criminal than they actually are.
- learnstats2 6y ago
  > Which is to say,... Since there is no point of comparison, this is not a valid statistical conclusion and is just a racist framing.
  - thaumasiotes 6y ago
    What do you mean? "If blacks were criminal at the rate predicted by their poverty rate, they would be much, much less criminal than they actually are" is a pure empirical observation. The logical point of comparison would be the nonblack population, but the conclusion is just as true for the entire population.

deleted 6y ago

[deleted]

randyrand 6y ago

Sure, but using the Duke paper example to illustrate that is clearly dumb. The toy AI network was designed that way. Yann was right to call it out.

nailer 6y ago

This is such a false binary. Of course we must correct for historical bias. Of course we must also proactively look for biases.

mnky9800n 6y ago

All input is biased. It is not our job as scientists to remove biases from a data set. It is our job to reduce bias and characterize bias, but it's impossible to have a data set that has zero bias. You cannot sample the whole universe.

scoutt 6y ago

> larger problem For a scientist doing a ML system to reconstruct pixelated faces, trained with white faces, why is he/she responsible for "insert larger problem" outside of her/his field? Do he/she has to also care about the brutal Tantalum Wars because there are some in the electronics they use? As far as I can see, ML is pretty new, and there is a lot of room for improvement. And I think people need to stop thinking the entire world is racist, or doesn't care or don't want to improve things. Changes take time, and won't happen today, or tomorrow or the next decade. There is a constant, furious need to point fingers, followed by fear of telling or not telling just the right words on the right order to the right audience. This is not how we are going to solve anything.

foldr 6y ago
You know I'm fine with that attitude if it's combined with actual humility. Often as not, however, these high profile guys are shooting off on Twitter about every issue under the sun. But when it comes to racism, suddenly they're just narrow technicians and it's nothing to do with them.
- scoutt 6y ago
  I don't know if it is about racism. I think it's normal to become defensive when accused. If I say "your ML model is garbage" I am sure they will (maybe, if I am high-profile too) provide an explanation on why or why not.
  - foldr 6y ago
    This long twitter thread about racism isn't about racism?
    - scoutt 6y ago
      Sorry, I expressed it wrong. I wanted to say that when someone is accused of something, including of being a racist or something related to racism, it's normal to be defensive about it and take that posture. I don't think the researcher was being defensive just because it was about racism.