5 ms·
But isn't it commonly accepted that ML systems reflect the bias in the data, and that it's hard to predict and control the bias in advance?
by tgv 6y ago
But isn't it commonly accepted that ML systems reflect the bias in the data, and that it's hard to predict and control the bias in advance?
- sudosysgen 6y agoHer point is more than that. I think that she was trying to say that ML systems have an inherent bias that is independent of their data. This is a consequence of the fact that it's not possible to do generalization, and thus learning, without bias to begin with. What an AI researcher does when deciding which architecture is essentially tuning the bias in the generalization system in order to produce better results, which is not in and of itself bad, and is indeed unavoidable. By bias, I mean really anything that means that the function g() is used over the function h(). In practice this can be anything from the seed for the random values, to the choice of activation function, to the architecture of the network, and so on. If the architectures are selected for their performance on dataset y, then it is possible that even when trained on dataset x, they retain some of the bias of dataset y, because their generalization bias, which is architectural, was tuned for dataset y. This is, admittedly, a theoretical result, and it's not clear that it applies in every case, but it is a solid point. For that reason the point that benchmarking against non-biased datasets is important was made, and thus that there is more to bias than the final dataset used for training during deployment. Therefore, it's not only a responsibility of the engineering community, but also of the scientific community. I've posted it somewhere here already, but here's a nice paper that explains the point that there is more to bias than simply the dataset : http://www.cs.cmu.edu/~tom/pubs/NeedForBias_1980.pdf http://www.cs.cmu.edu/~tom/pubs/NeedForBias_1980.pdf Interestingly enough, this is true of all learning systems, including humans, and is a proof of why it's impossible to be completely unbiased in really anything.
- chokeartist 6y ago> Her point is more than that. I think that she was trying to say that ML systems have an inherent bias that is independent of their data. In general I take these types of tropes to be cloaked (politically correct) way of saying "fuck x group".
- mlyle 6y agoEh. I think there's some legitimate concern that AI isn't fair to marginalized groups for a variety of reasons. Dataset bias is only one way this occurs, though. Even if you include more colored faces, you still may be taking pictures with a camera that doesn't capture as much contrast. Or even if you retrain, you may have chosen model structure to optimize behavior in a biased dataset. Or... if you're considering the cost of misidentification by a facial recognition system used by, say, police, you need to be sure that you take a perspective that applies to society as a whole and not to your own interactions with police. It's perfectly reasonable to say "AI is worsening the experience of minorities in various ways" and to differ with responses that assume with slight additional care in curating datasets that the problem will be completely solved.
- BlueTemplar 6y agoOne of the huge questions is : who takes responsibility for the issues with the black box learning systems ? In France, since you have to be able to explain in plain words the algorithm that was used to help to make an administrative decision, it indirectly makes these black box learning systems illegal. (And yet they probably already have been used by the police?) Now, the naive response is that the engineer that implemented a system should take responsibility for any harm done by the system - and I guess that if these engineers could lose their license to use a computer and face jail time, they might also start to put pressure upstream on researchers to provide better systems ?
- sudosysgen 6y agoI'm not sure I understood your comment correctly, so I'll apologize in advance if I misunderstood. That statement is contained in a paper I linked that dates from 1980. I'm pretty sure it's factually correct, because I'm trying to make nothing but a demonstration that an idée reçue, that ML systems are only based due to their data, is incorrect. There's really no cloaked message behind it, and I'm not sure why one would think there is (hence the disclaimer). I think it's a pretty uncharitable interpretation.
- 6y ago
- deleted 6y ago[deleted]
- IfOnlyYouKnew 6y agoIt's "commonly accepted" in that it is a well-known possibility and, therefore, something that practitioners should check for before publishing, using, or selling their models, yes. It's not "commonly accepted" in the sense of throwing one's hands up, saying "there's just no way we could find enough pictures of black people for our AI to no longer confuse them with Apes" and move on.
- mav3rick 6y agoSorry from her rebuttal it seemed like there were other ways than just more and better labelled data. Is this correct ?