9 ms·

The tone of her replies is regrettable: Yann: “ML systems are biased when data is biased" Timnit: “I’m sick of this framing. Tired of it. Many people have tri

The tone of her replies is regrettable: Yann: “ML systems are biased when data is biased" Timnit: “I’m sick of this framing. Tired of it. Many people have tried to explain, many scholars. Listen to us. You can’t just reduce harms caused by ML to dataset bias. Even amidst of world wide protests people don’t hear our voices and try to learn from us, they assume they’re experts in everything. Let us lead her and you follow. Just listen. And learn from scholars like @ruha9 [Ruha Benjamin, Associate Professor of African American Studies at Princeton University]" I don't like engaging this kind of discussion but have to call out the "I'm so tired" bit every time I see it.

tgv 6y ago
But isn't it commonly accepted that ML systems reflect the bias in the data, and that it's hard to predict and control the bias in advance?
- sudosysgen 6y ago
  Her point is more than that. I think that she was trying to say that ML systems have an inherent bias that is independent of their data. This is a consequence of the fact that it's not possible to do generalization, and thus learning, without bias to begin with. What an AI researcher does when deciding which architecture is essentially tuning the bias in the generalization system in order to produce better results, which is not in and of itself bad, and is indeed unavoidable. By bias, I mean really anything that means that the function g() is used over the function h(). In practice this can be anything from the seed for the random values, to the choice of activation function, to the architecture of the network, and so on. If the architectures are selected for their performance on dataset y, then it is possible that even when trained on dataset x, they retain some of the bias of dataset y, because their generalization bias, which is architectural, was tuned for dataset y. This is, admittedly, a theoretical result, and it's not clear that it applies in every case, but it is a solid point. For that reason the point that benchmarking against non-biased datasets is important was made, and thus that there is more to bias than the final dataset used for training during deployment. Therefore, it's not only a responsibility of the engineering community, but also of the scientific community. I've posted it somewhere here already, but here's a nice paper that explains the point that there is more to bias than simply the dataset : http://www.cs.cmu.edu/~tom/pubs/NeedForBias_1980.pdf http://www.cs.cmu.edu/~tom/pubs/NeedForBias_1980.pdf Interestingly enough, this is true of all learning systems, including humans, and is a proof of why it's impossible to be completely unbiased in really anything.
  - chokeartist 6y ago
    > Her point is more than that. I think that she was trying to say that ML systems have an inherent bias that is independent of their data. In general I take these types of tropes to be cloaked (politically correct) way of saying "fuck x group".
    - mlyle 6y ago
      Eh. I think there's some legitimate concern that AI isn't fair to marginalized groups for a variety of reasons. Dataset bias is only one way this occurs, though. Even if you include more colored faces, you still may be taking pictures with a camera that doesn't capture as much contrast. Or even if you retrain, you may have chosen model structure to optimize behavior in a biased dataset. Or... if you're considering the cost of misidentification by a facial recognition system used by, say, police, you need to be sure that you take a perspective that applies to society as a whole and not to your own interactions with police. It's perfectly reasonable to say "AI is worsening the experience of minorities in various ways" and to differ with responses that assume with slight additional care in curating datasets that the problem will be completely solved.
      - BlueTemplar 6y ago
        One of the huge questions is : who takes responsibility for the issues with the black box learning systems ? In France, since you have to be able to explain in plain words the algorithm that was used to help to make an administrative decision, it indirectly makes these black box learning systems illegal. (And yet they probably already have been used by the police?) Now, the naive response is that the engineer that implemented a system should take responsibility for any harm done by the system - and I guess that if these engineers could lose their license to use a computer and face jail time, they might also start to put pressure upstream on researchers to provide better systems ?
    - sudosysgen 6y ago
      I'm not sure I understood your comment correctly, so I'll apologize in advance if I misunderstood. That statement is contained in a paper I linked that dates from 1980. I'm pretty sure it's factually correct, because I'm trying to make nothing but a demonstration that an idée reçue, that ML systems are only based due to their data, is incorrect. There's really no cloaked message behind it, and I'm not sure why one would think there is (hence the disclaimer). I think it's a pretty uncharitable interpretation.
- deleted 6y ago
  [deleted]
- IfOnlyYouKnew 6y ago
  It's "commonly accepted" in that it is a well-known possibility and, therefore, something that practitioners should check for before publishing, using, or selling their models, yes. It's not "commonly accepted" in the sense of throwing one's hands up, saying "there's just no way we could find enough pictures of black people for our AI to no longer confuse them with Apes" and move on.
  - mav3rick 6y ago
    Sorry from her rebuttal it seemed like there were other ways than just more and better labelled data. Is this correct ?
webkike 6y ago
I don’t think that for a person who lives in a world where AI shops routinely sell software that discriminate against minorities, not just in ways that are explained by poorly thought out datasets, saying you’re “so tired” is at all wrong. This is the exact research that Timnit is an expert on, and these are systems that have real world consequences for minorities. It’s easy for people who are not minorities to say you should always a be polite but frankly I think she’s being extremely polite here given all of the circumstances.
- calf 6y ago
  I bet there is research showing how racism and discrimination literally tires out the people subjected to it. And now you have people saying they're not allowed to express that.
  - webkike 6y ago
    Indeed. reminds me of this speech by Jacob Blake’s sister: https://youtu.be/Ijp85on6UHY https://youtu.be/Ijp85on6UHY “I’m not sad. I’m not sorry. I’m angry. And I’m tired”
  - dash2 6y ago
    Nobody said it was disallowed. The grandparent said he called it out as a lazy rhetorical move. It's certainly true that oppression can be tiring (does that point need research?) It may equally be true that some people overclaim on the extent of their oppression, and the mental health harms they are suffering from it, and even make a profession of doing so.
  - 9HZZRfNlpR 6y ago
    People are just tired of assholes who only blame the world and never take responsibility.
- raxxorrax 6y ago
  How are these AI systems affecting people? Is there an example application?
IfOnlyYouKnew 6y ago
I find it strange to take issue with the "I'm so tired" line. It would seem to be entirely appropriate for circumstances where you see an argument that you have replied to many times but that completely ignores your reply. It also just doesn't strike me as particularly hostile or profane. Quitting twitter and making a show of it strikes me as a far more emotional reaction, but nobody here seems to take any issue with that? In this case, the issue is that "it's the dataset" is attacking a straw man: nobody is accusing ML practitioners to be explicit racists actively conspiring for their models to be racially biased. Everyone understands that it is (often) a result of the underlying dataset. But, the argument goes, understanding the genesis of such bias isn't enough to excuse it. At least not when those models are put to actual use in, say, law-enforcement contexts. Or credit agencies, or hiring decisions, or really anything. If your dog happened to bite any red-haired kid it came across, it doesn't matter that you think red-haired kids are just as good as others, or that the dog was once mistreated by a red-haired kid. You're not going to allow the dog to get near any red-haired kids. And if you do, you'll face charges of at least negligence when some red-haired kid gets mauled. To tie this back to AI, it would mean that as long as your models produce racially biased results, they simply aren't ready for deployment or publication. Go find better data until your work is no longer liable to inflict harm on anybody. That's the standard for all other industries: if your car is shown to reliably kill pedestrians below a certain height in crash tests, you go back to the drawing board. Adding a sticker "don't operate car in the vicinity of children" isn't enough. You can disagree with that reasoning (although you'd be wrong). What you can't do, as a leader in AI, is to pretend to have never heard of it, or that it is so obviously wrong that it warrants no reply whatsoever. > have to call out the "I'm so tired" bit every time I see it. so... I guess you're tired of it?
- uoaei 6y ago
  > so... I guess you're tired of it? Yeah GP is modeling exactly the behavior they're railing against. Claiming that something is wrong without any practical insights for how that thing might be better.
  - visarga 6y ago
    It's not exactly the same behavior - it's along the lines of "can't tolerate intolerance" (meta).
colinmhayes 6y ago
Tbh Lecunn kind of deserves it. He has a long history of twitter ass-ery.
antiquark 6y ago
> "I'm so tired" bit Yeah, it sort of implies that, yes, you're angry about something, but actually have no constructive advice or solutions, other than to accuse someone of being a bad person or even a racist. The "I'm so tired" bit is similar to the "I'm not here to teach you" bit. Maybe someone can't teach you because they actually have nothing to teach.
- IfOnlyYouKnew 6y ago
  No, "I'm so tired" means that it's an argument they have replied to before. She has written dozens of research papers, some of which probably contained arguments other than repetition of "you're a racist and a bad person". You of course know about this, or would now about this considering she is called an "AI researcher at Google" in the title of this tread. So the only lazy and un-constructive criticism is to assume she has not made any arguments beyond your one-line caricature of what you believe her to be.
  - antiquark 6y ago
    Then what is Gebru's solution to the FlickFaceHQ database having too many pictures of white people?
    - IfOnlyYouKnew 6y ago
      Expanding the dataset and/or using a subset of it?
      - antiquark 6y ago
        Gebru would disagree: > “I think that now a lot of people have understood that we need to have more diverse datasets, but unfortunately I felt like that’s kind of where the understanding has stopped. It’s like ‘let’s diversify our datasets. And that’s kind of ethics and fairness, right?’ But you can’t ignore social and structural problems.“ This seems to be saying that Yann LeCun should actually be addressing social and structural problems, rather that just applying neural networks to data sets. https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter-amid-acrimonious-exchanges-on-ai-bias/ https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
malcolmgreaves 6y ago
Women and people of color are often silenced using the tactic you just employed -- ignoring the substance of a message and dismissing all concerns because the "tone" doesn't seem "appropriate" to you.
jumby 6y ago
Honestly, I am not even sure _how_ they would fix the system(s) they complain most about? Assume Yann is wrong and Timnit is right, what is the proposal for fixing the bias that somehow is inherent in the system? She says 'let us lead and you follow' -- sure...where? What are we concretely doing about the "in-built bias" problem?
spoonjim 6y ago
The quality of the "argument" here is F--. Basically, it's, "nothing you say matters, only what we say matters." Phrases like "just listen" are essentially, "you will respect my authority!"

tgv 6y ago

But isn't it commonly accepted that ML systems reflect the bias in the data, and that it's hard to predict and control the bias in advance?

sudosysgen 6y ago
Her point is more than that. I think that she was trying to say that ML systems have an inherent bias that is independent of their data. This is a consequence of the fact that it's not possible to do generalization, and thus learning, without bias to begin with. What an AI researcher does when deciding which architecture is essentially tuning the bias in the generalization system in order to produce better results, which is not in and of itself bad, and is indeed unavoidable. By bias, I mean really anything that means that the function g() is used over the function h(). In practice this can be anything from the seed for the random values, to the choice of activation function, to the architecture of the network, and so on. If the architectures are selected for their performance on dataset y, then it is possible that even when trained on dataset x, they retain some of the bias of dataset y, because their generalization bias, which is architectural, was tuned for dataset y. This is, admittedly, a theoretical result, and it's not clear that it applies in every case, but it is a solid point. For that reason the point that benchmarking against non-biased datasets is important was made, and thus that there is more to bias than the final dataset used for training during deployment. Therefore, it's not only a responsibility of the engineering community, but also of the scientific community. I've posted it somewhere here already, but here's a nice paper that explains the point that there is more to bias than simply the dataset : http://www.cs.cmu.edu/~tom/pubs/NeedForBias_1980.pdf http://www.cs.cmu.edu/~tom/pubs/NeedForBias_1980.pdf Interestingly enough, this is true of all learning systems, including humans, and is a proof of why it's impossible to be completely unbiased in really anything.
- chokeartist 6y ago
  > Her point is more than that. I think that she was trying to say that ML systems have an inherent bias that is independent of their data. In general I take these types of tropes to be cloaked (politically correct) way of saying "fuck x group".
  - mlyle 6y ago
    Eh. I think there's some legitimate concern that AI isn't fair to marginalized groups for a variety of reasons. Dataset bias is only one way this occurs, though. Even if you include more colored faces, you still may be taking pictures with a camera that doesn't capture as much contrast. Or even if you retrain, you may have chosen model structure to optimize behavior in a biased dataset. Or... if you're considering the cost of misidentification by a facial recognition system used by, say, police, you need to be sure that you take a perspective that applies to society as a whole and not to your own interactions with police. It's perfectly reasonable to say "AI is worsening the experience of minorities in various ways" and to differ with responses that assume with slight additional care in curating datasets that the problem will be completely solved.
    - BlueTemplar 6y ago
      One of the huge questions is : who takes responsibility for the issues with the black box learning systems ? In France, since you have to be able to explain in plain words the algorithm that was used to help to make an administrative decision, it indirectly makes these black box learning systems illegal. (And yet they probably already have been used by the police?) Now, the naive response is that the engineer that implemented a system should take responsibility for any harm done by the system - and I guess that if these engineers could lose their license to use a computer and face jail time, they might also start to put pressure upstream on researchers to provide better systems ?
  - sudosysgen 6y ago
    I'm not sure I understood your comment correctly, so I'll apologize in advance if I misunderstood. That statement is contained in a paper I linked that dates from 1980. I'm pretty sure it's factually correct, because I'm trying to make nothing but a demonstration that an idée reçue, that ML systems are only based due to their data, is incorrect. There's really no cloaked message behind it, and I'm not sure why one would think there is (hence the disclaimer). I think it's a pretty uncharitable interpretation.
deleted 6y ago
[deleted]
IfOnlyYouKnew 6y ago
It's "commonly accepted" in that it is a well-known possibility and, therefore, something that practitioners should check for before publishing, using, or selling their models, yes. It's not "commonly accepted" in the sense of throwing one's hands up, saying "there's just no way we could find enough pictures of black people for our AI to no longer confuse them with Apes" and move on.
- mav3rick 6y ago
  Sorry from her rebuttal it seemed like there were other ways than just more and better labelled data. Is this correct ?

webkike 6y ago

I don’t think that for a person who lives in a world where AI shops routinely sell software that discriminate against minorities, not just in ways that are explained by poorly thought out datasets, saying you’re “so tired” is at all wrong. This is the exact research that Timnit is an expert on, and these are systems that have real world consequences for minorities. It’s easy for people who are not minorities to say you should always a be polite but frankly I think she’s being extremely polite here given all of the circumstances.

calf 6y ago
I bet there is research showing how racism and discrimination literally tires out the people subjected to it. And now you have people saying they're not allowed to express that.
- webkike 6y ago
  Indeed. reminds me of this speech by Jacob Blake’s sister: https://youtu.be/Ijp85on6UHY https://youtu.be/Ijp85on6UHY “I’m not sad. I’m not sorry. I’m angry. And I’m tired”
- dash2 6y ago
  Nobody said it was disallowed. The grandparent said he called it out as a lazy rhetorical move. It's certainly true that oppression can be tiring (does that point need research?) It may equally be true that some people overclaim on the extent of their oppression, and the mental health harms they are suffering from it, and even make a profession of doing so.
- 9HZZRfNlpR 6y ago
  People are just tired of assholes who only blame the world and never take responsibility.
raxxorrax 6y ago
How are these AI systems affecting people? Is there an example application?

IfOnlyYouKnew 6y ago

I find it strange to take issue with the "I'm so tired" line. It would seem to be entirely appropriate for circumstances where you see an argument that you have replied to many times but that completely ignores your reply. It also just doesn't strike me as particularly hostile or profane. Quitting twitter and making a show of it strikes me as a far more emotional reaction, but nobody here seems to take any issue with that? In this case, the issue is that "it's the dataset" is attacking a straw man: nobody is accusing ML practitioners to be explicit racists actively conspiring for their models to be racially biased. Everyone understands that it is (often) a result of the underlying dataset. But, the argument goes, understanding the genesis of such bias isn't enough to excuse it. At least not when those models are put to actual use in, say, law-enforcement contexts. Or credit agencies, or hiring decisions, or really anything. If your dog happened to bite any red-haired kid it came across, it doesn't matter that you think red-haired kids are just as good as others, or that the dog was once mistreated by a red-haired kid. You're not going to allow the dog to get near any red-haired kids. And if you do, you'll face charges of at least negligence when some red-haired kid gets mauled. To tie this back to AI, it would mean that as long as your models produce racially biased results, they simply aren't ready for deployment or publication. Go find better data until your work is no longer liable to inflict harm on anybody. That's the standard for all other industries: if your car is shown to reliably kill pedestrians below a certain height in crash tests, you go back to the drawing board. Adding a sticker "don't operate car in the vicinity of children" isn't enough. You can disagree with that reasoning (although you'd be wrong). What you can't do, as a leader in AI, is to pretend to have never heard of it, or that it is so obviously wrong that it warrants no reply whatsoever. > have to call out the "I'm so tired" bit every time I see it. so... I guess you're tired of it?

uoaei 6y ago
> so... I guess you're tired of it? Yeah GP is modeling exactly the behavior they're railing against. Claiming that something is wrong without any practical insights for how that thing might be better.
- visarga 6y ago
  It's not exactly the same behavior - it's along the lines of "can't tolerate intolerance" (meta).

colinmhayes 6y ago

Tbh Lecunn kind of deserves it. He has a long history of twitter ass-ery.

antiquark 6y ago

> "I'm so tired" bit Yeah, it sort of implies that, yes, you're angry about something, but actually have no constructive advice or solutions, other than to accuse someone of being a bad person or even a racist. The "I'm so tired" bit is similar to the "I'm not here to teach you" bit. Maybe someone can't teach you because they actually have nothing to teach.

IfOnlyYouKnew 6y ago
No, "I'm so tired" means that it's an argument they have replied to before. She has written dozens of research papers, some of which probably contained arguments other than repetition of "you're a racist and a bad person". You of course know about this, or would now about this considering she is called an "AI researcher at Google" in the title of this tread. So the only lazy and un-constructive criticism is to assume she has not made any arguments beyond your one-line caricature of what you believe her to be.
- antiquark 6y ago
  Then what is Gebru's solution to the FlickFaceHQ database having too many pictures of white people?
  - IfOnlyYouKnew 6y ago
    Expanding the dataset and/or using a subset of it?
    - antiquark 6y ago
      Gebru would disagree: > “I think that now a lot of people have understood that we need to have more diverse datasets, but unfortunately I felt like that’s kind of where the understanding has stopped. It’s like ‘let’s diversify our datasets. And that’s kind of ethics and fairness, right?’ But you can’t ignore social and structural problems.“ This seems to be saying that Yann LeCun should actually be addressing social and structural problems, rather that just applying neural networks to data sets. https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter-amid-acrimonious-exchanges-on-ai-bias/ https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...

malcolmgreaves 6y ago

Women and people of color are often silenced using the tactic you just employed -- ignoring the substance of a message and dismissing all concerns because the "tone" doesn't seem "appropriate" to you.

jumby 6y ago

Honestly, I am not even sure _how_ they would fix the system(s) they complain most about? Assume Yann is wrong and Timnit is right, what is the proposal for fixing the bias that somehow is inherent in the system? She says 'let us lead and you follow' -- sure...where? What are we concretely doing about the "in-built bias" problem?

spoonjim 6y ago

The quality of the "argument" here is F--. Basically, it's, "nothing you say matters, only what we say matters." Phrases like "just listen" are essentially, "you will respect my authority!"