6 ms·

How would you describe what happened? LeCunn calmly and rationally explained that a particular model produced biased results, and that that this was due to the

How would you describe what happened? LeCunn calmly and rationally explained that a particular model produced biased results, and that that this was due to the bias in its training data. He was then excoriated as a racist by this woman and the Twitter mobs she incited, to the point where this Turing Award winner is no longer willing to engage with the public via that medium. LeCunn made repeated attempts to explain his position, only to be virtually shouted over about he wasn’t listening to people of color because he pointed out that a model trained on White people learned to produce images of White people, and that if it had been trained on Black people it would produce images of Black people. It was a research paper, not a production system, and Gebru and her acolytes were much more interested in scoring cheap points than in having a serious conversation about either ML fairness or the merits of this particular paper. Seriously, the primary sources can be read by anyone.

astrange 6y ago
> LeCunn calmly and rationally explained that a particular model produced biased results, and that that this was due to the bias in its training data. But this claim is certainly not necessarily correct, and he shouldn't have been so confident about it. Any part of a system can contribute to bias and that includes the model design, not just the data. If they want the system to work then they need to actually demonstrate it works, not just say it could with X change without testing that. Though I remember people arguing it didn't work properly at the same time they were saying it was evil (because it contributed to surveillance). Which is odd because if you don't want it to exist, you shouldn't want it to work either.
- fullstackPM 6y ago
  Wouldn't that go both ways though? Why was Gebru so confident that the bias is because of something else other than the training dataset? She immediately assumed that there are other items contributing to the bias and devolved the quality of the conversation as well by making those claims in a frustrated/exasperated tone. The most obvious and easiest way to clear this up would be for the original researches to train the model on a "Senegal" dataset as mentioned by LeCunn and see what the results are.
  - compycom 6y ago
    The whole point is that it never is trained on a "Senegal" dataset, is it? Why do we always see these errors where white-centric datasets produce white-centric results? While it would be a symmetric situation a vacuum, we do not live in a vacuum. And acknowledging dataset bias by itself doesn't address the bias meaningfully. In practice, we often treat the bias as an exogenous factor when it is not, moving it outside the scope of our responsibility. But it is very much the product of our work, a reflection of our choices, values, and beliefs about what to prioritize. We can't abdicate our responsibility for it; the stakes are too high.
    - 6gvONxR4sf7o 6y ago
      Does exogenous vs endogenous work in a vacuum either? It’s exogenous with respect to some things and endogenous with respect to others.
    - fullstackPM 6y ago
      Well that is the whole point. The entire controversy could have been cleared up if the original researchers or even Gebru would have trained the model on a "Senegal" dataset, or a dataset diverse enough to their own liking and then make the findings public. That, to me is research. Throwing a fit on twitter because people won't buy that the outcome of this model is racist is not research. I am not sure why the wording around this has to be so abstract.
    - visarga 6y ago
      This is a great goal but the time and place for it is not in a super-resolution paper. There are specific conferences, benchmarks and projects where it can be achieved with more efficiency.
    - PeterisP 6y ago
      The whole point is that working on more accurate methods for doing X is one direction of research and working on methods to correct for dataset bias in X is another direction of research. It's completely reasonable to work on either one of those separately, and it's unreasonable to demand (you may suggest, but not demand) that people working on the former direction fix the latter. Better algorithms and better datasets are both necessary but quite orthogonal. As you say, it's a reflection of our beliefs about what to prioritize. If you're interested in developing methods of image upscaling that generalize better and preserve facial properties that were underrepresented in the training data, that's an interesting area of research, and you're welcome to work on it, but you don't get to demand that people working on something else prioritize this instead, they get to choose their own priorities unless you're paying them for a particular direction of research (e.g. like Google should be able to direct Dr. Gebru while she was working for them). There's a responsibility to correct for bias when implementing models in production (e.g. if you'd be actually deploying it in Senegal, then there would be a responsibility to use a Senegal-appropriate dataset), there's a responsibility to acknowledge the bias of a particular algorithm if one exists - for example if it highly relies on contrast values which would be different for different types of faces, then that's relevant, because it's a statement about the generalization of the algorithm; but there's no proactive duty to "address the bias meaningfully", that's nice thing to do, but it's like charity - a voluntary choice to factilitate a social goal, but not a requirement or responsibility to do that. There's a responsibility to correct harms you caused, there is no responsibility to correct harms caused by others, that's a good thing to do, but not a moral duty. Someone who invests a lot into charity work or addressing bias is doing a good thing, but it doesn't mean that people who are doing other things are abdicating their responsibility - it's never was their responsibility in the first place to fix social issues in the wider society. You can't simply point at random people and declare that they're going to be responsible for something that they didn't personally cause and where they did nothing wrong, that accusatory behavior is unacceptable, so naturally there's a backlash to people who try to assert personal responsibility of others without a basis to do so.
- throwaway-kmc 6y ago
  I think it is reasonable for one of the founders and most highly respected members of the field to make confident claims about something that is as open and shut as this. The paper in question was training a network to upres faces. The researchers trained it on a dataset containing primarily White faces and released it on GitHub for reproducibility. Subsequent people used it on Black faces and found it make them white. This is not surprising. Black faces and White faces are not the same data manifold. This is like training a network to upsample oranges, then running it on an apple and being surprised when the result is an unusual color and texture for an apple. I’m not sure what else there is possibly wrong here. Is the width of their convolutions racist? Their choice to work on super resolution? The fact that they released their work for reproducibility?
  - astrange 6y ago
    > I’m not sure what else there is possibly wrong here. Is the width of their convolutions racist? Not that parameter specifically, but if you say the data is wrong then that means any other parameter might be wrong too. More data might mean the model is too small to fit it, or the hyperparameters might be wrong to train it, or you now have the wrong ratio of other phenotypes (let's say that instead of races…) in the training set and their results regress. Also, if you're adding people with darker skin, that of course means the pixel values are lower. That matters for image processing code, things like SAD thresholds or noise reduction will work differently. > Their choice to work on super resolution? Superresolution is only useful as a toy and they should have mentioned that when they put up a live colab, yes. They added a disclaimer later on and it was good - there was a big issue where people were convinced this was somehow a surveillance technology because they watched too many TV shows, even though it literally can't work that way!
    - visarga 6y ago
      Occam's razor makes us think in this case it's the data, the simplest explanation.
      - astrange 6y ago
- hintymad 6y ago
  LeCunn may not be correct, but attacking his character for be incorrect is even worse. Moralizing even scientific debate is the road to destruction.
veridies 6y ago
I'd never heard of this fight before, so I just looked it up. My interpretation is that he made that claim, Gebru responded by claiming that that was an overly reductionist view of things (given, presumably, the fact that biased assumptions could affect the coded assumptions as well as the training data), he maintained his opinion, and she felt frustrated that she didn't feel she was being listened to. I did not see her call him a racist, and I did not think either person was acting in bad faith; both seemed frustrated and unhappy that there was a disagreement. Am I missing anything?
- throwaway-kmc 6y ago
  Here’s a good summary: https://www.google.com/amp/s/syncedreview.com/2020/06/30/yann-lecun-quits-twitter-amid-acrimonious-exchanges-on-ai-bias/amp/ https://www.google.com/amp/s/syncedreview.com/2020/06/30/yan... I think it’s clear Gebru was acting in bad faith and LeCunn was baffled and trying to deescalate. Gebru responds to each attempt to deescalate by a reply designed to further rile up Twitter, without actually talking to any of the points LeCunn makes. Virtually all of her replies are some variant of “you are wrong, but I won’t say why” and “listen to me because I am Black”.
  - worker767424 6y ago
    I wouldn't be shocked if Google saw this drama, saw that she was offering to resign, and was like "that's a freebie."
    - thu2111 6y ago
      But that drama was some time ago. She should have been fired on the spot back then, but, at Google women are virtually unfireable, black women doubly so. Whatever she did to get canned like this must have been extreme. Either that or Google is learning how to hold these sorts of employees to the same standards as their less fashionable colleagues. Edit: incorrectly claimed the spat was before 2020, it wasn't. 2020 just feels like an incredibly long year.
      - 6y ago

astrange 6y ago

> LeCunn calmly and rationally explained that a particular model produced biased results, and that that this was due to the bias in its training data. But this claim is certainly not necessarily correct, and he shouldn't have been so confident about it. Any part of a system can contribute to bias and that includes the model design, not just the data. If they want the system to work then they need to actually demonstrate it works, not just say it could with X change without testing that. Though I remember people arguing it didn't work properly at the same time they were saying it was evil (because it contributed to surveillance). Which is odd because if you don't want it to exist, you shouldn't want it to work either.

fullstackPM 6y ago
Wouldn't that go both ways though? Why was Gebru so confident that the bias is because of something else other than the training dataset? She immediately assumed that there are other items contributing to the bias and devolved the quality of the conversation as well by making those claims in a frustrated/exasperated tone. The most obvious and easiest way to clear this up would be for the original researches to train the model on a "Senegal" dataset as mentioned by LeCunn and see what the results are.
- compycom 6y ago
  The whole point is that it never is trained on a "Senegal" dataset, is it? Why do we always see these errors where white-centric datasets produce white-centric results? While it would be a symmetric situation a vacuum, we do not live in a vacuum. And acknowledging dataset bias by itself doesn't address the bias meaningfully. In practice, we often treat the bias as an exogenous factor when it is not, moving it outside the scope of our responsibility. But it is very much the product of our work, a reflection of our choices, values, and beliefs about what to prioritize. We can't abdicate our responsibility for it; the stakes are too high.
  - 6gvONxR4sf7o 6y ago
    Does exogenous vs endogenous work in a vacuum either? It’s exogenous with respect to some things and endogenous with respect to others.
  - fullstackPM 6y ago
    Well that is the whole point. The entire controversy could have been cleared up if the original researchers or even Gebru would have trained the model on a "Senegal" dataset, or a dataset diverse enough to their own liking and then make the findings public. That, to me is research. Throwing a fit on twitter because people won't buy that the outcome of this model is racist is not research. I am not sure why the wording around this has to be so abstract.
  - visarga 6y ago
    This is a great goal but the time and place for it is not in a super-resolution paper. There are specific conferences, benchmarks and projects where it can be achieved with more efficiency.
  - PeterisP 6y ago
    The whole point is that working on more accurate methods for doing X is one direction of research and working on methods to correct for dataset bias in X is another direction of research. It's completely reasonable to work on either one of those separately, and it's unreasonable to demand (you may suggest, but not demand) that people working on the former direction fix the latter. Better algorithms and better datasets are both necessary but quite orthogonal. As you say, it's a reflection of our beliefs about what to prioritize. If you're interested in developing methods of image upscaling that generalize better and preserve facial properties that were underrepresented in the training data, that's an interesting area of research, and you're welcome to work on it, but you don't get to demand that people working on something else prioritize this instead, they get to choose their own priorities unless you're paying them for a particular direction of research (e.g. like Google should be able to direct Dr. Gebru while she was working for them). There's a responsibility to correct for bias when implementing models in production (e.g. if you'd be actually deploying it in Senegal, then there would be a responsibility to use a Senegal-appropriate dataset), there's a responsibility to acknowledge the bias of a particular algorithm if one exists - for example if it highly relies on contrast values which would be different for different types of faces, then that's relevant, because it's a statement about the generalization of the algorithm; but there's no proactive duty to "address the bias meaningfully", that's nice thing to do, but it's like charity - a voluntary choice to factilitate a social goal, but not a requirement or responsibility to do that. There's a responsibility to correct harms you caused, there is no responsibility to correct harms caused by others, that's a good thing to do, but not a moral duty. Someone who invests a lot into charity work or addressing bias is doing a good thing, but it doesn't mean that people who are doing other things are abdicating their responsibility - it's never was their responsibility in the first place to fix social issues in the wider society. You can't simply point at random people and declare that they're going to be responsible for something that they didn't personally cause and where they did nothing wrong, that accusatory behavior is unacceptable, so naturally there's a backlash to people who try to assert personal responsibility of others without a basis to do so.
throwaway-kmc 6y ago
I think it is reasonable for one of the founders and most highly respected members of the field to make confident claims about something that is as open and shut as this. The paper in question was training a network to upres faces. The researchers trained it on a dataset containing primarily White faces and released it on GitHub for reproducibility. Subsequent people used it on Black faces and found it make them white. This is not surprising. Black faces and White faces are not the same data manifold. This is like training a network to upsample oranges, then running it on an apple and being surprised when the result is an unusual color and texture for an apple. I’m not sure what else there is possibly wrong here. Is the width of their convolutions racist? Their choice to work on super resolution? The fact that they released their work for reproducibility?
- astrange 6y ago
  > I’m not sure what else there is possibly wrong here. Is the width of their convolutions racist? Not that parameter specifically, but if you say the data is wrong then that means any other parameter might be wrong too. More data might mean the model is too small to fit it, or the hyperparameters might be wrong to train it, or you now have the wrong ratio of other phenotypes (let's say that instead of races…) in the training set and their results regress. Also, if you're adding people with darker skin, that of course means the pixel values are lower. That matters for image processing code, things like SAD thresholds or noise reduction will work differently. > Their choice to work on super resolution? Superresolution is only useful as a toy and they should have mentioned that when they put up a live colab, yes. They added a disclaimer later on and it was good - there was a big issue where people were convinced this was somehow a surveillance technology because they watched too many TV shows, even though it literally can't work that way!
  - visarga 6y ago
    Occam's razor makes us think in this case it's the data, the simplest explanation.
    - astrange 6y ago
hintymad 6y ago
LeCunn may not be correct, but attacking his character for be incorrect is even worse. Moralizing even scientific debate is the road to destruction.

veridies 6y ago

I'd never heard of this fight before, so I just looked it up. My interpretation is that he made that claim, Gebru responded by claiming that that was an overly reductionist view of things (given, presumably, the fact that biased assumptions could affect the coded assumptions as well as the training data), he maintained his opinion, and she felt frustrated that she didn't feel she was being listened to. I did not see her call him a racist, and I did not think either person was acting in bad faith; both seemed frustrated and unhappy that there was a disagreement. Am I missing anything?

throwaway-kmc 6y ago
Here’s a good summary: https://www.google.com/amp/s/syncedreview.com/2020/06/30/yann-lecun-quits-twitter-amid-acrimonious-exchanges-on-ai-bias/amp/ https://www.google.com/amp/s/syncedreview.com/2020/06/30/yan... I think it’s clear Gebru was acting in bad faith and LeCunn was baffled and trying to deescalate. Gebru responds to each attempt to deescalate by a reply designed to further rile up Twitter, without actually talking to any of the points LeCunn makes. Virtually all of her replies are some variant of “you are wrong, but I won’t say why” and “listen to me because I am Black”.
- worker767424 6y ago
  I wouldn't be shocked if Google saw this drama, saw that she was offering to resign, and was like "that's a freebie."
  - thu2111 6y ago
    But that drama was some time ago. She should have been fired on the spot back then, but, at Google women are virtually unfireable, black women doubly so. Whatever she did to get canned like this must have been extreme. Either that or Google is learning how to hold these sorts of employees to the same standards as their less fashionable colleagues. Edit: incorrectly claimed the spat was before 2020, it wasn't. 2020 just feels like an incredibly long year.
    - 6y ago