6 ms·

I'm in the field - though not as prominent as Yann (who has been very nice and helpful in my few interactions with him) - and your interpretation is off. People

I'm in the field - though not as prominent as Yann (who has been very nice and helpful in my few interactions with him) - and your interpretation is off. People are disagreeing with his stance that researchers should not bother exploring bias implications of their research. (He says this is because bias is a problem of data - and therefore we should focus on building cool models and let production engineers worry about training production models on unbiased data.) People are disagreeing not because of political correctness, but because this is a fundamental mischaracterization of how research works and how it gets transferred to "real world" applications. (1) Data fuels modern machine learning. It shapes research directions in a really fundamental way. People decide what to work on based on what huge amounts of data they can get their hands on. Saying "engineers should be the ones to worry about bias because it's a data problem" is like saying "I'm a physicist, here's a cool model, I'll let the engineers worry about whether it works on any known particle in any known world." (2) Most machine learning research is empirical (though not all). It's very rare to see a paper (if not impossible nowadays, since large deep neural networks are so massive and opaque) that works purely off math without showing that its conclusions improve some task on some dataset. No one is doing research without data, and saying "my method is good because it works on this data" means you are making choices and statements about what it means to "work" - which, as we've seen, involves quite a lot of bias. (3) Almost all prominent ML researchers work for massively rich corporations. He and his colleagues don't work in ivory towers where they develop pure algorithms which are then released over the ivy walls into the wild, to be contaminated by filthy reality. He works for Facebook. He's paid with Facebook money. So why draw this imaginary line between research and production? He is paid to do research that will go into production. So his statement is so wildly disconnected from research reality that it seems like it was not made in good faith - or at least without much thought - which is what people are responding to. Also, language tip - a "woman researcher" is a "researcher".

deleted 6y ago
[deleted]
tinyhouse 6y ago
Don't have a lot of time to respond now, but will try to do it later. Just a quick note. I agree his comment about engineers need to worry more about bias than researchers is strange. But in my opinion it wasn't the focus of what he was tying to say. I used "woman researcher" since it was important for the context as people accused him of mansplaining.
dcolkitt 6y ago
> He works for Facebook. He's paid with Facebook money. So why draw this imaginary line between research and production? He is paid to do research that will go into production. This is a silly standard to uphold. The sizable bulk of American academic researchers are at least partially funded by grants made from the US federal budget. If you were to enforce your standards consistently, then all of those researchers would be held responsible for any eventual usage of their research by the US federal government. I really doubt you apply the same standard. So, the criticism mostly seems to be an isolated demand for rigor. You're holding Facebook Research to a different standard than the average university researcher funded by a federal grant.
- nlpprof 6y ago
  Did you read what I wrote? I don't think his argument is true. (That is, I do think researchers should keep bias in mind when developing machine learning projects.) (Regardless of their funding sources.) Because of his employment, this argument is a particularly silly one for him to make.
- ncallaway 6y ago
  This seems almost purposefully disingenuous to me. Yann LeCun isn't receiving a partial research grant from Facebook. He's literally an employee of Facebook. His job title is "VP & Chief AI Scientist" (at least according to LinkedIn). There's an obvious and clear distinction between an employee and a research grant, and this feels like it's almost wilfully obtuse.
vlthr 6y ago
I agree with all of your points about the diffusion of responsibility that is common in ML, though I think you may not be sensitive enough to the harmful framing being created by the "anti-bias" side. The original locus of the debate was how the recent face-depixelation paper turned out to depixelate pictures of black faces into ones with white features. That discovery is an interesting and useful showcase for talking about how ML can demonstrate unexpected racial bias, and it should be talked about. As often happens, the nuances of what exactly this discovery means and what we can learn from it quickly got simplified away. Just hours later, the paper was being showcased as a prime example of unethical and racist research. When LeCun originally commented on this, I took his point to be pretty simple: that for an algorithm trained to depixelate faces, it's no surprise that it fills in the blank with white features because that's just what the FlickFaceHQ dataset looks like. If you had trained it on a majority-black dataset, we would expect the inverse. That in no way dismisses all of the real concerns people have (and should have!) about bias in ML. But many critics of this paper seem far too willing to catastrophize about how irresponsible and unethical this paper is. LeCun's original point was (as I understand it) that this criticism goes overboard given that the training dataset is an obvious culprit for the observed behavior. Following his original comment, he has been met with some extremely uncharitable responses. The most circulated example is this tweet (https://twitter.com/timnitGebru/status/1274809417653866496?s=20 https://twitter.com/timnitGebru/status/1274809417653866496?s...) where a bias-in-ml researcher calls him out without as much as a mention of why he is wrong, or even what he is wrong about. LeCun responds with a 17-tweet thread clarifying his stance, and her response is to claim that educating him is not worth her time (https://twitter.com/timnitGebru/status/1275191341455048704?s=20 https://twitter.com/timnitGebru/status/1275191341455048704?s...). The overwhelming attitude there and elsewhere is in support of the attacker. Not of the attacker's arguments - they were never presented - but of the symbolic identity she takes on as the anti-racist fighting the racist old elite. I apologize if my frustration with their behavior shines through, but it really pains me to see this identity-driven mob mentality take hold in our community. Fixing problems requires talking about them and understanding them, and this really isn't it.
- sinity 6y ago
  I think this is relevant: https://twitter.com/AnimaAnandkumar/status/1271137176529416193 https://twitter.com/AnimaAnandkumar/status/12711371765294161... Nvidia AI researcher calling out OpenAI's GPT-2 over how GPT-2 is horrible because it's trained on Reddit (except it includes contents of submissions, and I'm not sure if there's no data except Reddit) Reddit is supposedly not a good source of data to train NLP models because it's... racist? sexist? Like it's even rightist in general... Anyway; the table looks horrific - why would they include these results? Oh, turns out paper was on bias: https://arxiv.org/pdf/1909.01326.pdf https://arxiv.org/pdf/1909.01326.pdf Anyway; one can toy with GPT-2 large (paper is on medium, so it might be different) at talktotransformer.com "The woman worked as a ": 2x receptionist, teacher's aide, waitress. Man: waiter, fitness instructor, spot worker, (construction?) engineer. Black man: farm hand, carpenter, carpet installer(?), technician. White man: assistant architect, [carpenter but became a shoemaker], general in the army, blacksmith. I didn't read the paper, I admit, maybe I'm missing something here. But these tweets look like... person responsible should be fired.
smsm42 6y ago
So, your argument is that you disagree with data being the root of the problem by arguing that data "shapes research directions in a really fundamental way", research is "empirical" (i.e. based on data) and his research can't be isolated from data it'd be used on in production? Looks to me that you're argumentatively agreeing with Yann.
- joshuamorton 6y ago
  Not really, Yann's original claim (which he sort of kind of partially walked back) was that data is the only source of bias [0][1]. He walked that back somewhat to claim that he was being very particular in this case[2], which is perhaps true, but still harmful. The right thing to do when you make a mistake is apologize. Not double down and badly re-explain what other experts have been telling you back at them. So then Yann notes that generic models don't have bias[3]. This is, probably, true. I'd be surprised if on the whole, "CNNs" encoded racial bias. But the specific networks we use, say ResNet, which are optimized to perform well on biased datasets, may themselves encode bias in the model architecture[4]. That is, the models that perform best on a biased dataset may themselves be architecturally biased. In fact, we'd sort of expect it. And that all ignores one of the major issues which Yann entirely skips, but which Timnit covers in some of her work: training on data, even "representative data" encodes the biases that are present in the world today. You see this come up often with questions about tools like "crime predictors based on faces". In that context it's blatantly obvious that no, what the model learns will not be how criminal someone is, but how they are treated by the justice system today. Those two things might be somewhat correlated, but they're not causally related, and so trying to predict one from the other is a fool's errand and a dangerous fool's errand since the model will serve to encode existing biases behind a facade of legitimacy. Yann doesn't ever respond to that criticism, seemingly because he hasn't taken the time to actually look at the research in this area. So insofar as data is the root of the problem, yes. Insofar as the solution is to just use more representative data in the same systems, no. That doesn't fix things. You have to go further and use different systems or even ask different questions (or rule out certain questions as too fraught with problems to be able to ask). [0]: https://twitter.com/ylecun/status/1203211859366576128 https://twitter.com/ylecun/status/1203211859366576128 [1]: https://twitter.com/ylecun/status/1274782757907030016 https://twitter.com/ylecun/status/1274782757907030016 [2]: https://twitter.com/ylecun/status/1275162732166361088 https://twitter.com/ylecun/status/1275162732166361088 [3]: https://twitter.com/ylecun/status/1275167319157870592 https://twitter.com/ylecun/status/1275167319157870592 [4]: https://twitter.com/hardmaru/status/1275214381509300224 https://twitter.com/hardmaru/status/1275214381509300224. This actually goes a bit further, suggesting that as a leader in the field one has a responsibility to encourage ethics as part of the decision making process in how/what we research, but let's leave that aside.

deleted 6y ago

[deleted]

tinyhouse 6y ago

Don't have a lot of time to respond now, but will try to do it later. Just a quick note. I agree his comment about engineers need to worry more about bias than researchers is strange. But in my opinion it wasn't the focus of what he was tying to say. I used "woman researcher" since it was important for the context as people accused him of mansplaining.

dcolkitt 6y ago

> He works for Facebook. He's paid with Facebook money. So why draw this imaginary line between research and production? He is paid to do research that will go into production. This is a silly standard to uphold. The sizable bulk of American academic researchers are at least partially funded by grants made from the US federal budget. If you were to enforce your standards consistently, then all of those researchers would be held responsible for any eventual usage of their research by the US federal government. I really doubt you apply the same standard. So, the criticism mostly seems to be an isolated demand for rigor. You're holding Facebook Research to a different standard than the average university researcher funded by a federal grant.

nlpprof 6y ago
Did you read what I wrote? I don't think his argument is true. (That is, I do think researchers should keep bias in mind when developing machine learning projects.) (Regardless of their funding sources.) Because of his employment, this argument is a particularly silly one for him to make.
ncallaway 6y ago
This seems almost purposefully disingenuous to me. Yann LeCun isn't receiving a partial research grant from Facebook. He's literally an employee of Facebook. His job title is "VP & Chief AI Scientist" (at least according to LinkedIn). There's an obvious and clear distinction between an employee and a research grant, and this feels like it's almost wilfully obtuse.

vlthr 6y ago

I agree with all of your points about the diffusion of responsibility that is common in ML, though I think you may not be sensitive enough to the harmful framing being created by the "anti-bias" side. The original locus of the debate was how the recent face-depixelation paper turned out to depixelate pictures of black faces into ones with white features. That discovery is an interesting and useful showcase for talking about how ML can demonstrate unexpected racial bias, and it should be talked about. As often happens, the nuances of what exactly this discovery means and what we can learn from it quickly got simplified away. Just hours later, the paper was being showcased as a prime example of unethical and racist research. When LeCun originally commented on this, I took his point to be pretty simple: that for an algorithm trained to depixelate faces, it's no surprise that it fills in the blank with white features because that's just what the FlickFaceHQ dataset looks like. If you had trained it on a majority-black dataset, we would expect the inverse. That in no way dismisses all of the real concerns people have (and should have!) about bias in ML. But many critics of this paper seem far too willing to catastrophize about how irresponsible and unethical this paper is. LeCun's original point was (as I understand it) that this criticism goes overboard given that the training dataset is an obvious culprit for the observed behavior. Following his original comment, he has been met with some extremely uncharitable responses. The most circulated example is this tweet (https://twitter.com/timnitGebru/status/1274809417653866496?s=20 https://twitter.com/timnitGebru/status/1274809417653866496?s...) where a bias-in-ml researcher calls him out without as much as a mention of why he is wrong, or even what he is wrong about. LeCun responds with a 17-tweet thread clarifying his stance, and her response is to claim that educating him is not worth her time (https://twitter.com/timnitGebru/status/1275191341455048704?s=20 https://twitter.com/timnitGebru/status/1275191341455048704?s...). The overwhelming attitude there and elsewhere is in support of the attacker. Not of the attacker's arguments - they were never presented - but of the symbolic identity she takes on as the anti-racist fighting the racist old elite. I apologize if my frustration with their behavior shines through, but it really pains me to see this identity-driven mob mentality take hold in our community. Fixing problems requires talking about them and understanding them, and this really isn't it.

sinity 6y ago
I think this is relevant: https://twitter.com/AnimaAnandkumar/status/1271137176529416193 https://twitter.com/AnimaAnandkumar/status/12711371765294161... Nvidia AI researcher calling out OpenAI's GPT-2 over how GPT-2 is horrible because it's trained on Reddit (except it includes contents of submissions, and I'm not sure if there's no data except Reddit) Reddit is supposedly not a good source of data to train NLP models because it's... racist? sexist? Like it's even rightist in general... Anyway; the table looks horrific - why would they include these results? Oh, turns out paper was on bias: https://arxiv.org/pdf/1909.01326.pdf https://arxiv.org/pdf/1909.01326.pdf Anyway; one can toy with GPT-2 large (paper is on medium, so it might be different) at talktotransformer.com "The woman worked as a ": 2x receptionist, teacher's aide, waitress. Man: waiter, fitness instructor, spot worker, (construction?) engineer. Black man: farm hand, carpenter, carpet installer(?), technician. White man: assistant architect, [carpenter but became a shoemaker], general in the army, blacksmith. I didn't read the paper, I admit, maybe I'm missing something here. But these tweets look like... person responsible should be fired.

smsm42 6y ago

So, your argument is that you disagree with data being the root of the problem by arguing that data "shapes research directions in a really fundamental way", research is "empirical" (i.e. based on data) and his research can't be isolated from data it'd be used on in production? Looks to me that you're argumentatively agreeing with Yann.

joshuamorton 6y ago
Not really, Yann's original claim (which he sort of kind of partially walked back) was that data is the only source of bias [0][1]. He walked that back somewhat to claim that he was being very particular in this case[2], which is perhaps true, but still harmful. The right thing to do when you make a mistake is apologize. Not double down and badly re-explain what other experts have been telling you back at them. So then Yann notes that generic models don't have bias[3]. This is, probably, true. I'd be surprised if on the whole, "CNNs" encoded racial bias. But the specific networks we use, say ResNet, which are optimized to perform well on biased datasets, may themselves encode bias in the model architecture[4]. That is, the models that perform best on a biased dataset may themselves be architecturally biased. In fact, we'd sort of expect it. And that all ignores one of the major issues which Yann entirely skips, but which Timnit covers in some of her work: training on data, even "representative data" encodes the biases that are present in the world today. You see this come up often with questions about tools like "crime predictors based on faces". In that context it's blatantly obvious that no, what the model learns will not be how criminal someone is, but how they are treated by the justice system today. Those two things might be somewhat correlated, but they're not causally related, and so trying to predict one from the other is a fool's errand and a dangerous fool's errand since the model will serve to encode existing biases behind a facade of legitimacy. Yann doesn't ever respond to that criticism, seemingly because he hasn't taken the time to actually look at the research in this area. So insofar as data is the root of the problem, yes. Insofar as the solution is to just use more representative data in the same systems, no. That doesn't fix things. You have to go further and use different systems or even ask different questions (or rule out certain questions as too fraught with problems to be able to ask). [0]: https://twitter.com/ylecun/status/1203211859366576128 https://twitter.com/ylecun/status/1203211859366576128 [1]: https://twitter.com/ylecun/status/1274782757907030016 https://twitter.com/ylecun/status/1274782757907030016 [2]: https://twitter.com/ylecun/status/1275162732166361088 https://twitter.com/ylecun/status/1275162732166361088 [3]: https://twitter.com/ylecun/status/1275167319157870592 https://twitter.com/ylecun/status/1275167319157870592 [4]: https://twitter.com/hardmaru/status/1275214381509300224 https://twitter.com/hardmaru/status/1275214381509300224. This actually goes a bit further, suggesting that as a leader in the field one has a responsibility to encourage ethics as part of the decision making process in how/what we research, but let's leave that aside.