3 ms·

I imagine that the model would also score "he was murdered" higher than "she was murdered". Models reflect their inputs, and it happens that yes, murder victim

I imagine that the model would also score "he was murdered" higher than "she was murdered". Models reflect their inputs, and it happens that yes, murder victims are disproportionately likely to be male and nurses are disproportionately likely to be female. Is there a problem we should address here? Absolutely -- but the problem is that men keep on getting murdered, not that the model recognizes truths with which we are uncomfortable.

gowld 9y ago
No one is saying the model is wrong. As the article states: > As with Tia, Tamera has several choices she can make. She could simply accept these biases as is and do nothing, though at least now she won't be caught off-guard if users complain. > She could make changes in the user interface, for example by having it present two gendered responses instead of just one, though she might not want to do that if the input message has a gendered pronoun (e.g., "Will she be there today?"). > She could try retraining the embedding model using a bias mitigation technique (e.g., as in Bolukbasi et al.) and examining how this affects downstream performance, or she might mitigate bias in the classifier directly when training her classifier (e.g., as in Dixon et al. [1], Beutel et al. [10], or Zhang et al. [11]). > No matter what she decides to do, it's important that Tamera has done this type of analysis so that she's aware of what her product does and can make informed decisions.
deleted 9y ago
[deleted]
allenz 9y ago
Biased models can cause ethical and legal problems. While your specific example is not a huge deal, the article gives the example of making hiring decisions in part based on sentiment analysis of candidates' text reviews. In this context, an engineer has responsibility to ensure that the model has no gender, race, or age bias towards candidates' names. For a real life example, in 2017 Google was more likely to filter the comment "I am a woman" than "I am a man": https://www.engadget.com/2017/09/01/google-perspective-comment-ranking-system/ https://www.engadget.com/2017/09/01/google-perspective-comme... Or consider the impact of any bias in AI for criminal sentencing recommendations: https://www.wired.com/2017/04/courts-using-ai-sentence-criminals-must-stop-now/ https://www.wired.com/2017/04/courts-using-ai-sentence-crimi...
- home_boi 9y ago
  If the biased models replace human decision making, then it just has to be shown that the models are less biased than humans which may not be that high of a bar to pass.
  - allenz 9y ago
    US law prohibits businesses from discrimination. If you're sued, you can't argue that you only discriminated the average amount.

gowld 9y ago

No one is saying the model is wrong. As the article states: > As with Tia, Tamera has several choices she can make. She could simply accept these biases as is and do nothing, though at least now she won't be caught off-guard if users complain. > She could make changes in the user interface, for example by having it present two gendered responses instead of just one, though she might not want to do that if the input message has a gendered pronoun (e.g., "Will she be there today?"). > She could try retraining the embedding model using a bias mitigation technique (e.g., as in Bolukbasi et al.) and examining how this affects downstream performance, or she might mitigate bias in the classifier directly when training her classifier (e.g., as in Dixon et al. [1], Beutel et al. [10], or Zhang et al. [11]). > No matter what she decides to do, it's important that Tamera has done this type of analysis so that she's aware of what her product does and can make informed decisions.

deleted 9y ago

[deleted]

allenz 9y ago

Biased models can cause ethical and legal problems. While your specific example is not a huge deal, the article gives the example of making hiring decisions in part based on sentiment analysis of candidates' text reviews. In this context, an engineer has responsibility to ensure that the model has no gender, race, or age bias towards candidates' names. For a real life example, in 2017 Google was more likely to filter the comment "I am a woman" than "I am a man": https://www.engadget.com/2017/09/01/google-perspective-comment-ranking-system/ https://www.engadget.com/2017/09/01/google-perspective-comme... Or consider the impact of any bias in AI for criminal sentencing recommendations: https://www.wired.com/2017/04/courts-using-ai-sentence-criminals-must-stop-now/ https://www.wired.com/2017/04/courts-using-ai-sentence-crimi...

home_boi 9y ago
If the biased models replace human decision making, then it just has to be shown that the models are less biased than humans which may not be that high of a bar to pass.
- allenz 9y ago
  US law prohibits businesses from discrimination. If you're sued, you can't argue that you only discriminated the average amount.