4 ms·
Except only including facts can still reinforce unfair bias. For example, it's true that there are more men than women in software engineering. Whether someone
by tylerhou 7y ago
Except only including facts can still reinforce unfair bias. For example, it's true that there are more men than women in software engineering. Whether someone is a software engineer or not is a fact. If you have a "representative" dataset with only facts, then it's possible that an AI would have a higher chance as labeling men as software engineers than women, simply because it begins to associate masculine facial features with software engineering.
In my eyes, this result would reinforce unfair bias, and a thus well-designed AI should avoid this (i.e. with all else equal, a well-designed AI should suggest the label "software engineer" at the same rate for both men and women).
- chii 7y agoIf it's true that there are more male software engineers, then why is it wrong for the AI to "learn" that? If the AI did start classifying masculine features biased towards software engineers, then the AI has learnt the above fact, and thus can be used to make predictions. The moral standpoint that there shouldn't be more male software engineers than female engineers is a personal and subjective ideal, and if you lament bias, then why isn't this kind of bias given the same treatment?
- abathur 7y agoThe moral standpoint isn't that there shouldn't be more (or less) male software engineers. The moral standpoint is that there shouldn't be an AICandidateFilter|HumanPrejudicialInterviewer that only coincidentally appears to beat a coin-flip because it has learned non-causal correlations which it uses to dust out qualified stereotype-defying human candidates because they don't look stereotypical enough on the axes that the dataset--which almost inevitably has a status-quo bias--suggests are relevant.
- esyir 7y agoSo, it depends on what you want to do here. If the task is just "predict if the person is a software engineer". I'd say go ahead, bias it away. Here, anything that boosts accuracy is game to me. But if the task is say the pre-screening side. This becomes a more ethically/morally tricky question. If and only if that sex is not a predictive factor for engineer quality, you would then expect to see similar classifier performance for male / female samples. Given that assumption, significant (hah) divergence from equal performance would be something to correct. Of course there are other issues to handle, such as the unbalanced state of the dataset and so on.
- jacquesm 7y agoIt is wrong because there is no causal relationship between the two so none can be inferred.
- lmm 7y agoCitation needed. A human's physical sex correlates with differences in almost every measurable characteristic. For some we know about the causal mechanism, for others the details remain an open research question. But it would be crazy to assume that just because we don't know what causes a correlation, that causal link must not exist. And it would be even more crazy to assume that software engineering is somehow magically unrelated to every measurable characteristic of people, or that the characteristics we haven't been able to measure are somehow radically different from the ones we have been able to.
- jacquesm 7y agoYou are making a logic error. When there is no causal connection between two items it is very well possible that there is a connection that allows you to say something about populations. But you will never be able to say something about an individual. And that is where all these arguments flounder, we put population information in, in order to engineer features that then allow us to make decisions about individuals. For those cases where feature engineering can dig up causal connections this works wonders, for those cases where it does not or gives you apparent connections that are not really there you end up with problems.