6 ms·
> LeCunn calmly and rationally explained that a particular model produced biased results, and that that this was due to the bias in its training data. But thi
by astrange 6y ago
> LeCunn calmly and rationally explained that a particular model produced biased results, and that that this was due to the bias in its training data.
But this claim is certainly not necessarily correct, and he shouldn't have been so confident about it. Any part of a system can contribute to bias and that includes the model design, not just the data. If they want the system to work then they need to actually demonstrate it works, not just say it could with X change without testing that.
Though I remember people arguing it didn't work properly at the same time they were saying it was evil (because it contributed to surveillance). Which is odd because if you don't want it to exist, you shouldn't want it to work either.
- fullstackPM 6y agoWouldn't that go both ways though? Why was Gebru so confident that the bias is because of something else other than the training dataset? She immediately assumed that there are other items contributing to the bias and devolved the quality of the conversation as well by making those claims in a frustrated/exasperated tone. The most obvious and easiest way to clear this up would be for the original researches to train the model on a "Senegal" dataset as mentioned by LeCunn and see what the results are.
- compycom 6y agoThe whole point is that it never is trained on a "Senegal" dataset, is it? Why do we always see these errors where white-centric datasets produce white-centric results? While it would be a symmetric situation a vacuum, we do not live in a vacuum. And acknowledging dataset bias by itself doesn't address the bias meaningfully. In practice, we often treat the bias as an exogenous factor when it is not, moving it outside the scope of our responsibility. But it is very much the product of our work, a reflection of our choices, values, and beliefs about what to prioritize. We can't abdicate our responsibility for it; the stakes are too high.
- 6gvONxR4sf7o 6y agoDoes exogenous vs endogenous work in a vacuum either? It’s exogenous with respect to some things and endogenous with respect to others.
- fullstackPM 6y agoWell that is the whole point. The entire controversy could have been cleared up if the original researchers or even Gebru would have trained the model on a "Senegal" dataset, or a dataset diverse enough to their own liking and then make the findings public. That, to me is research. Throwing a fit on twitter because people won't buy that the outcome of this model is racist is not research. I am not sure why the wording around this has to be so abstract.
- visarga 6y agoThis is a great goal but the time and place for it is not in a super-resolution paper. There are specific conferences, benchmarks and projects where it can be achieved with more efficiency.
- PeterisP 6y agoThe whole point is that working on more accurate methods for doing X is one direction of research and working on methods to correct for dataset bias in X is another direction of research. It's completely reasonable to work on either one of those separately, and it's unreasonable to demand (you may suggest, but not demand) that people working on the former direction fix the latter. Better algorithms and better datasets are both necessary but quite orthogonal. As you say, it's a reflection of our beliefs about what to prioritize. If you're interested in developing methods of image upscaling that generalize better and preserve facial properties that were underrepresented in the training data, that's an interesting area of research, and you're welcome to work on it, but you don't get to demand that people working on something else prioritize this instead, they get to choose their own priorities unless you're paying them for a particular direction of research (e.g. like Google should be able to direct Dr. Gebru while she was working for them). There's a responsibility to correct for bias when implementing models in production (e.g. if you'd be actually deploying it in Senegal, then there would be a responsibility to use a Senegal-appropriate dataset), there's a responsibility to acknowledge the bias of a particular algorithm if one exists - for example if it highly relies on contrast values which would be different for different types of faces, then that's relevant, because it's a statement about the generalization of the algorithm; but there's no proactive duty to "address the bias meaningfully", that's nice thing to do, but it's like charity - a voluntary choice to factilitate a social goal, but not a requirement or responsibility to do that. There's a responsibility to correct harms you caused, there is no responsibility to correct harms caused by others, that's a good thing to do, but not a moral duty. Someone who invests a lot into charity work or addressing bias is doing a good thing, but it doesn't mean that people who are doing other things are abdicating their responsibility - it's never was their responsibility in the first place to fix social issues in the wider society. You can't simply point at random people and declare that they're going to be responsible for something that they didn't personally cause and where they did nothing wrong, that accusatory behavior is unacceptable, so naturally there's a backlash to people who try to assert personal responsibility of others without a basis to do so.
- jl2718 6y agoNo. The easiest way to clear this up would be for Gebru to make a specific claim that was actually related to the model and include a pull request of her own modifications that she had tested to deliver superior results. This would be called “research”. That is what she was hired to do at Google. For everybody of every race ethnicity orientation able status and background that would be honored to give their best effort at the opportunity to do research at google, acceptance of resignation was the right thing.
- fullstackPM 6y agoOf course. Or at the very least, she herself could have trained the model on a "Senegal" dataset and make the findings public. My only point around her feud with LeCunn is that she threw a fit because people including LeCunn pointed out that it is not necessarily a race thing, and she wanted to give it a race spin without confirming that it was actually the case. And I am not at all commenting on all this drama around her resignation/being fired without getting a look at the contents of the two emails: the one she sent to Brain Women and the one with her demands. But I know this: if I were to issue an ultimatum to my employer, I am treading dangerous waters and should be able to digest the outcome of that, including being fired.
- throwaway-kmc 6y agoI think it is reasonable for one of the founders and most highly respected members of the field to make confident claims about something that is as open and shut as this. The paper in question was training a network to upres faces. The researchers trained it on a dataset containing primarily White faces and released it on GitHub for reproducibility. Subsequent people used it on Black faces and found it make them white. This is not surprising. Black faces and White faces are not the same data manifold. This is like training a network to upsample oranges, then running it on an apple and being surprised when the result is an unusual color and texture for an apple. I’m not sure what else there is possibly wrong here. Is the width of their convolutions racist? Their choice to work on super resolution? The fact that they released their work for reproducibility?
- astrange 6y ago> I’m not sure what else there is possibly wrong here. Is the width of their convolutions racist? Not that parameter specifically, but if you say the data is wrong then that means any other parameter might be wrong too. More data might mean the model is too small to fit it, or the hyperparameters might be wrong to train it, or you now have the wrong ratio of other phenotypes (let's say that instead of races…) in the training set and their results regress. Also, if you're adding people with darker skin, that of course means the pixel values are lower. That matters for image processing code, things like SAD thresholds or noise reduction will work differently. > Their choice to work on super resolution? Superresolution is only useful as a toy and they should have mentioned that when they put up a live colab, yes. They added a disclaimer later on and it was good - there was a big issue where people were convinced this was somehow a surveillance technology because they watched too many TV shows, even though it literally can't work that way!
- hintymad 6y agoLeCunn may not be correct, but attacking his character for be incorrect is even worse. Moralizing even scientific debate is the road to destruction.