5 ms·

From the article: When a scientist talked about meeting an investor in January, for example, Gmail offered the follow-up "do you want to meet him" -- not consi

From the article: When a scientist talked about meeting an investor in January, for example, Gmail offered the follow-up "do you want to meet him" -- not considering the possibility that the investor could be a woman. The problem is a typical one with natural language generation systems like Google's: it's based on huge volumes of historical data. As certain fields tend to be dominated by people from one gender, the AI can sometimes assume that a person belongs to that gender.

funkythings 8y ago
The AI just generalizes to match the best possible output. all this bias reducing stuff in ML will backfire
- crooked-v 8y ago
  Machine learning matches the input and the training it's put through. This is not even close to the same thing as "best possible output".
- hnbroseph 8y ago
  what do you mean by "best"?
vorticalbox 8y ago
Then isn't that a fair assumption for an AI or any one to make? If you're offended by how a computer looked at statistics and came to a likely conclusion, it's probally time take a step back and have a good think about why you're even offended.
- hnbroseph 8y ago
  > fair assumption does this have any relevance? a variety of things could be a "fair assumption", but it's not like you must make some assumptions. > you're offended this seems reductive if not an outright strawman. in many ways, language is power, and being a little conscientious isn't going to end the world.
- jakelazaroff 8y ago
  Maybe computers that interact with humans should sometimes err on the side of those humans' feelings rather than cold statistics? If I were addressing someone whose gender I didn't know, I would just omit a gendered term rather than assume their gender. There's no reason a computer shouldn't do the same.
- jkaplowitz 8y ago
  If the AI is telling you, "based on the statistics in the field, the odds of this person being a woman, a man, or a non-binary gender are X%, Y%, and Z% respectively," I agree there's no reason for offense. But if the AI has no information about the email recipient's individual gender, it's much safer for the AI to use the default pronoun (singular they) that English already has where we don't want to assume the gender of a particular person, or else for it to avoid the use of a pronoun. If a person who is not a man works in a male-dominated field and gets mislabeled as a man, it just makes them feel out of place and unwelcome. Doubly so if the AI-generated words are attributed to an individual colleague who knows their actual gender but doesn't fully edit the suggestion before sending.
  - vorticalbox 8y ago
    Except this isn't a person walking into a place and being mislabeled by another person. it's a computer that has zero concept of gender. I don't think that being mistake for a different sex is even cause to be offended. That person doesn't know and can only go only by what they have seen before.
    - EthanHeilman 8y ago
      Not sure why you are making this about "being offended" or not. In some case a program makes mistakes --> a simple solution is implemented.
    - jkaplowitz 8y ago
      The mistake is being attributed to a human email sender as perceived by the email recipient, not to an AI, since it's an email suggestion. Many such email senders will know the recipient's gender better than the AI, and the recipient will know that. As for your broader doubt that misgendering leads to offense, that's a longer conversation than I can shoehorn into this thread today, but the Geek Feminism wiki is one of several good places to which you can turn to start learning about that topic if you want to. I'm a cisgender man myself, but I have heard enough personal stories from friends who aren't to know that misgendering can and often does offend.
- jmilloy 8y ago
  > Then isn't that a fair assumption for an AI or any one to make? Sure. They are just choosing not to make that fair assumption even though they could make a correct guess more often than not. > If you're offended I don't think it's about being offended. It's about reducing bias.
  - opwieurposiu 8y ago
    In the statistical sense, assuming equal gender ratios in contexts that do not actually have equal gender ratios is an INCREASE in bias.
    - jeromegv 8y ago
      It doesn’t assume equal gender ratio. It will use a neutral pronom that will ensure it gets the response 100% right. Solution will be right 100% of the time, wouldn’t you prefer than may be being right at 70%?
- crooked-v 8y ago
  Why do you assume that correcting bias must come from some position of personal offense?
crooked-v 8y ago
In other words, machine learning is very good at giving you output like the input, but that doesn't mean the input is good.

funkythings 8y ago

The AI just generalizes to match the best possible output. all this bias reducing stuff in ML will backfire

crooked-v 8y ago
Machine learning matches the input and the training it's put through. This is not even close to the same thing as "best possible output".
hnbroseph 8y ago
what do you mean by "best"?

vorticalbox 8y ago

Then isn't that a fair assumption for an AI or any one to make? If you're offended by how a computer looked at statistics and came to a likely conclusion, it's probally time take a step back and have a good think about why you're even offended.

hnbroseph 8y ago
> fair assumption does this have any relevance? a variety of things could be a "fair assumption", but it's not like you must make some assumptions. > you're offended this seems reductive if not an outright strawman. in many ways, language is power, and being a little conscientious isn't going to end the world.
jakelazaroff 8y ago
Maybe computers that interact with humans should sometimes err on the side of those humans' feelings rather than cold statistics? If I were addressing someone whose gender I didn't know, I would just omit a gendered term rather than assume their gender. There's no reason a computer shouldn't do the same.
jkaplowitz 8y ago
If the AI is telling you, "based on the statistics in the field, the odds of this person being a woman, a man, or a non-binary gender are X%, Y%, and Z% respectively," I agree there's no reason for offense. But if the AI has no information about the email recipient's individual gender, it's much safer for the AI to use the default pronoun (singular they) that English already has where we don't want to assume the gender of a particular person, or else for it to avoid the use of a pronoun. If a person who is not a man works in a male-dominated field and gets mislabeled as a man, it just makes them feel out of place and unwelcome. Doubly so if the AI-generated words are attributed to an individual colleague who knows their actual gender but doesn't fully edit the suggestion before sending.
- vorticalbox 8y ago
  Except this isn't a person walking into a place and being mislabeled by another person. it's a computer that has zero concept of gender. I don't think that being mistake for a different sex is even cause to be offended. That person doesn't know and can only go only by what they have seen before.
  - EthanHeilman 8y ago
    Not sure why you are making this about "being offended" or not. In some case a program makes mistakes --> a simple solution is implemented.
  - jkaplowitz 8y ago
    The mistake is being attributed to a human email sender as perceived by the email recipient, not to an AI, since it's an email suggestion. Many such email senders will know the recipient's gender better than the AI, and the recipient will know that. As for your broader doubt that misgendering leads to offense, that's a longer conversation than I can shoehorn into this thread today, but the Geek Feminism wiki is one of several good places to which you can turn to start learning about that topic if you want to. I'm a cisgender man myself, but I have heard enough personal stories from friends who aren't to know that misgendering can and often does offend.
jmilloy 8y ago
> Then isn't that a fair assumption for an AI or any one to make? Sure. They are just choosing not to make that fair assumption even though they could make a correct guess more often than not. > If you're offended I don't think it's about being offended. It's about reducing bias.
- opwieurposiu 8y ago
  In the statistical sense, assuming equal gender ratios in contexts that do not actually have equal gender ratios is an INCREASE in bias.
  - jeromegv 8y ago
    It doesn’t assume equal gender ratio. It will use a neutral pronom that will ensure it gets the response 100% right. Solution will be right 100% of the time, wouldn’t you prefer than may be being right at 70%?
crooked-v 8y ago
Why do you assume that correcting bias must come from some position of personal offense?

crooked-v 8y ago

In other words, machine learning is very good at giving you output like the input, but that doesn't mean the input is good.