7 ms·

Cloud Vision API will not return gendered labels such as 'man' and 'woman'

47 points by jawngee 7y ago

I just received an email from Google Cloud Platform: Hello Google Cloud Vision API customer, We are writing to let you know that starting February 19, 2020, the Cloud Vision API will no longer return gendered labels such as 'man' and 'woman' that describe persons in an image when using the ‘LABEL_DETECTION’ feature. What do I need to know? As you know, the Cloud Vision API can perform feature detection on a local image file for the purpose of identifying persons by sending the contents of the image file through ‘LABEL_DETECTION’. Currently, when you request the API to annotate an image with labels, if you use this feature on images with people, it may return labels describing them in an image with gendered terms, like ‘man’ or 'woman’. Given that a person’s gender cannot be inferred by appearance, we have decided to remove these labels in order to align with the Artificial Intelligence Principles at Google, specifically Principle #2: Avoid creating or reinforcing unfair bias. https://ai.google/principles/

MidnightRaver 7y ago
I wonder if this is actually a marketing gimmick to invoke outrage and therefore clicks? The change certainly reduces the usefulness of the API by a fraction of a fraction of a percentage.
- rl3 7y ago
  No, it's legitimate. Google is the last company that needs marketing gimmicks, let alone ones based on outrage.
comsolo 7y ago
Another instance of SJW types causing problems in the coding world (although the vast majority of actual coders aren't on the SJW train). The fact is gender can be determined by appearance in 99.5% of circumstances or more, which is probably higher than the overall accuracy of their ML labelling system. The idea of "avoiding unfair bias" is highly vague and problematic since, theoretically nearly any statement or view beyond "particle X is at Y" type statements has a somewhat "unfair" bias reflecting the lens of the human experience, which has developed over millions of years of biology and thousands of years of culture. Finally, seeing as the fact is that 99% of people would not be negatively affected by this feature remaining whatsoever, and the very slight negative impact on the other (intersex type) 1% or so is minimal at best, while the total impact of removing the feature is mildly annoying for many users, this seems a lot like irrational, hollow, virtue signalling.
- foldr 7y ago
  >the other (intersex type) The biggest group likely to be impacted is trans people, not intersex people.
  - comsolo 7y ago
    Still less than 1% of the population, and that number could dwindle if and when treatment sorts out the underlying problems with the mental wiring causing the gender dysmorphia, and the current trend dies out.
    - lordCarbonFiber 7y ago
      Red heads are less than 2% of the population and yet if you built a classification system that failed to identify them as people that wouldn't be seen as acceptable either. Trans people have been around forever, and aren't going away just to make image classification less complex. It's not a "trend", treatment options just happen to be more available and the internet makes communities more visible now.
      - ralfd 7y ago
        > Red heads are less than 2% of the population and yet if you built a classification system that failed to identify them as people that wouldn't be seen as acceptable either. So the solution would be to not identify _anyone_ as people? As a ginger I doubt that.
        foldr 7y ago
        The solution would be to accept that the classification algorithm was too error probe to be usable for non-toy applications.
        ralfd 7y ago
        We are going in circles. An error rate of 2% (heck! 20%!) would not make a classification unusable, that was what one of the grandfather posts was arguing. https://adssettings.google.com/u/0/authenticated https://adssettings.google.com/u/0/authenticated Here Google can infer my gender and my age and personal interests just from my search history. I am sure this is not perfect either, it is still immensely useful for advertising.
codesternews 7y ago
Are we becoming more stupid?
0x1221 7y ago
> Given that a person’s gender cannot be inferred by appearance, we have decided to remove these labels in order to align with the Artificial Intelligence Principles at Google, specifically Principle #2: Avoid creating or reinforcing unfair bias. Isn't this just outright wrong? A person's gender can be inferred by appearance, and with extremely high accuracy at that.
- rl3 7y ago
  While I tend to agree with you from a technical standpoint, I can't help but think Google is actually doing the morally correct thing here. If organizations are creating systems using that API, most will just roll with man/woman labeling. If those systems are important, a lot of trans and non-binary people will end up fucked over. At the end of the day, if the deprecated functionality is still needed, just set a threshold on masculine vs feminine. It's a far more elegant approach, even if more work.
  - johnlorentzson 7y ago
    Automatic gender detection should certainly not be used for data entry where the data is of any real consequence (even if we ignored trans/non-binary people), simply due to the fact that it can't be proven to be perfect.
  - muzani 7y ago
    I think yeah, it's easier to use masculine/feminine. Something like long hair, soft skin, pink cheeks, more curves is feminine. Someone with more squares/edges (jaw, cheeks, shoulders) is more masculine. Humans have trouble with this too, e.g. many crushed on Taylor Hanson.
  - brodouevencode 7y ago
    I always get it confused: is it sex or gender that people can assign to themselves?
    - foldr 7y ago
      Are you really confused about this, or just rhetorically confused? You can answer the question by googling "sex vs gender".
growlist 7y ago
In a way I find it offensive that this service effectively denies my biological masculinity, which I consider to be an essential and distinguishing part of my identity. However, I'll not bother making a fuss because I realise there are far more important things in life: this being the difference between me and the virtue-signalling intellectual and moral pygmies in charge of ridiculous decisions like this.
- foldr 7y ago
  >In a way I find it offensive that this service effectively denies my biological masculinity All it's saying is that it won't make gender classifications based on perceived masculinity/femininity. Just because you can automatically classify certain features to some degree of accuracy doesn't mean you should. A company like Google also has an ethical obligation to think about how the technology will be used.
- s3krit 7y ago
  >effectively denies my biological masculinity How does it do that? All they are doing is no longer using perceived masculinity as a metric for determining whether you might be male or female. >I'll not bother making a fuss And yet here we are
  - GreenJelloShot 7y ago
    > And yet here we are A two sentence post on HN is "making a fuss"???
  - growlist 7y ago
    A post on HN is a bit different to the mind-melting full-spectrum 'there's no such thing as men and women guyz!' etc. subversion efforts afoot.
fyp 7y ago
My google ads profile correctly identifies me as male: https://adssettings.google.com/ https://adssettings.google.com/ I imagine the high level business use cases for these machine learning APIs are similar. For example, analyze shopping mall camera footage to figure out demographics shopping there and coordinate the (socially accepted and segregated) men/women department stores accordingly. This is such an profitable use case I cynically can't see why they would cripple their API like this, regardless of their ethical stance.
mateo1 7y ago
That's just hilarious. We live in an era where the statistical differences between the appearance of men and women are better understood than ever before, thanks to artificial neural networks, and we can in fact pinpoint the exact combinations of low-level features that make a face look like a man or woman. And then you have people in such a deep denial of reality they pull off this type of joke. Sure, you cannot identify sex with absolute certainty only with a photo, but ignoring fringe cases you can guess with very high certainty.
emilfihlman 7y ago
It's funny how ads can be targeted at men or women, but everything else is just "nooooo, we can't do that!". I wish the adbuyers would flex that money muscle a bit and put google in line again.
he11ow 7y ago
Wow, the responses to this thread are such an excellent marker for skin colour. I don't know the Google stats, but as far as labeling black women as male, Amazon and IBM get it wrong 30% of the time. Microsoft errs in 21% of test cases. As for Google's rates, not all that long ago they were labeling black people as gorillas so... [0] Put it another way, how impressed would your boss be if you were to say "Yo, I've nailed this. I've got a kick ass algorithm, check this out. Oh, yeah, it fucks up 30% of the time." Bias in AI is bad, but what makes it much much worse is arrogant attitudes claiming it doesn't exist. And you know what, that speaks to the idiocy around AI more than anything. All these debates on AGI are moot, because people are already so keen to believe they got it all figured out it doesn't even matter if the actual tech is any good at all. [0] https://www.wired.com/story/photo-algorithms-id-white-men-fineblack-women-not-so-much/ https://www.wired.com/story/photo-algorithms-id-white-men-fi...
- raxxorrax 7y ago
  > Wow, the responses to this thread are such an excellent marker for skin colour. As is pronouncing the effects of bias and prejudice. You can almost guarantee the person is white without using any AI. I was or still am pretty critical of AI since the quantification of every property of people will probably not be very enjoyable for anyone. What changed my mind a bit is that there are people wanting to employ it to improve reality. That is far worse and I hadn't had that on my radar before.

47 points by jawngee 7y ago

MidnightRaver 7y ago

I wonder if this is actually a marketing gimmick to invoke outrage and therefore clicks? The change certainly reduces the usefulness of the API by a fraction of a fraction of a percentage.

rl3 7y ago
No, it's legitimate. Google is the last company that needs marketing gimmicks, let alone ones based on outrage.

comsolo 7y ago

Another instance of SJW types causing problems in the coding world (although the vast majority of actual coders aren't on the SJW train). The fact is gender can be determined by appearance in 99.5% of circumstances or more, which is probably higher than the overall accuracy of their ML labelling system. The idea of "avoiding unfair bias" is highly vague and problematic since, theoretically nearly any statement or view beyond "particle X is at Y" type statements has a somewhat "unfair" bias reflecting the lens of the human experience, which has developed over millions of years of biology and thousands of years of culture. Finally, seeing as the fact is that 99% of people would not be negatively affected by this feature remaining whatsoever, and the very slight negative impact on the other (intersex type) 1% or so is minimal at best, while the total impact of removing the feature is mildly annoying for many users, this seems a lot like irrational, hollow, virtue signalling.

foldr 7y ago
>the other (intersex type) The biggest group likely to be impacted is trans people, not intersex people.
- comsolo 7y ago
  Still less than 1% of the population, and that number could dwindle if and when treatment sorts out the underlying problems with the mental wiring causing the gender dysmorphia, and the current trend dies out.
  - lordCarbonFiber 7y ago
    Red heads are less than 2% of the population and yet if you built a classification system that failed to identify them as people that wouldn't be seen as acceptable either. Trans people have been around forever, and aren't going away just to make image classification less complex. It's not a "trend", treatment options just happen to be more available and the internet makes communities more visible now.
    - ralfd 7y ago
      > Red heads are less than 2% of the population and yet if you built a classification system that failed to identify them as people that wouldn't be seen as acceptable either. So the solution would be to not identify _anyone_ as people? As a ginger I doubt that.
      - foldr 7y ago
        The solution would be to accept that the classification algorithm was too error probe to be usable for non-toy applications.
        ralfd 7y ago
        We are going in circles. An error rate of 2% (heck! 20%!) would not make a classification unusable, that was what one of the grandfather posts was arguing. https://adssettings.google.com/u/0/authenticated https://adssettings.google.com/u/0/authenticated Here Google can infer my gender and my age and personal interests just from my search history. I am sure this is not perfect either, it is still immensely useful for advertising.

codesternews 7y ago

Are we becoming more stupid?

0x1221 7y ago

> Given that a person’s gender cannot be inferred by appearance, we have decided to remove these labels in order to align with the Artificial Intelligence Principles at Google, specifically Principle #2: Avoid creating or reinforcing unfair bias. Isn't this just outright wrong? A person's gender can be inferred by appearance, and with extremely high accuracy at that.

rl3 7y ago
While I tend to agree with you from a technical standpoint, I can't help but think Google is actually doing the morally correct thing here. If organizations are creating systems using that API, most will just roll with man/woman labeling. If those systems are important, a lot of trans and non-binary people will end up fucked over. At the end of the day, if the deprecated functionality is still needed, just set a threshold on masculine vs feminine. It's a far more elegant approach, even if more work.
- johnlorentzson 7y ago
  Automatic gender detection should certainly not be used for data entry where the data is of any real consequence (even if we ignored trans/non-binary people), simply due to the fact that it can't be proven to be perfect.
- muzani 7y ago
  I think yeah, it's easier to use masculine/feminine. Something like long hair, soft skin, pink cheeks, more curves is feminine. Someone with more squares/edges (jaw, cheeks, shoulders) is more masculine. Humans have trouble with this too, e.g. many crushed on Taylor Hanson.
- brodouevencode 7y ago
  I always get it confused: is it sex or gender that people can assign to themselves?
  - foldr 7y ago
    Are you really confused about this, or just rhetorically confused? You can answer the question by googling "sex vs gender".

growlist 7y ago

In a way I find it offensive that this service effectively denies my biological masculinity, which I consider to be an essential and distinguishing part of my identity. However, I'll not bother making a fuss because I realise there are far more important things in life: this being the difference between me and the virtue-signalling intellectual and moral pygmies in charge of ridiculous decisions like this.

foldr 7y ago
>In a way I find it offensive that this service effectively denies my biological masculinity All it's saying is that it won't make gender classifications based on perceived masculinity/femininity. Just because you can automatically classify certain features to some degree of accuracy doesn't mean you should. A company like Google also has an ethical obligation to think about how the technology will be used.
s3krit 7y ago
>effectively denies my biological masculinity How does it do that? All they are doing is no longer using perceived masculinity as a metric for determining whether you might be male or female. >I'll not bother making a fuss And yet here we are
- GreenJelloShot 7y ago
  > And yet here we are A two sentence post on HN is "making a fuss"???
- growlist 7y ago
  A post on HN is a bit different to the mind-melting full-spectrum 'there's no such thing as men and women guyz!' etc. subversion efforts afoot.

fyp 7y ago

My google ads profile correctly identifies me as male: https://adssettings.google.com/ https://adssettings.google.com/ I imagine the high level business use cases for these machine learning APIs are similar. For example, analyze shopping mall camera footage to figure out demographics shopping there and coordinate the (socially accepted and segregated) men/women department stores accordingly. This is such an profitable use case I cynically can't see why they would cripple their API like this, regardless of their ethical stance.

mateo1 7y ago

That's just hilarious. We live in an era where the statistical differences between the appearance of men and women are better understood than ever before, thanks to artificial neural networks, and we can in fact pinpoint the exact combinations of low-level features that make a face look like a man or woman. And then you have people in such a deep denial of reality they pull off this type of joke. Sure, you cannot identify sex with absolute certainty only with a photo, but ignoring fringe cases you can guess with very high certainty.

emilfihlman 7y ago

It's funny how ads can be targeted at men or women, but everything else is just "nooooo, we can't do that!". I wish the adbuyers would flex that money muscle a bit and put google in line again.

he11ow 7y ago

Wow, the responses to this thread are such an excellent marker for skin colour. I don't know the Google stats, but as far as labeling black women as male, Amazon and IBM get it wrong 30% of the time. Microsoft errs in 21% of test cases. As for Google's rates, not all that long ago they were labeling black people as gorillas so... [0] Put it another way, how impressed would your boss be if you were to say "Yo, I've nailed this. I've got a kick ass algorithm, check this out. Oh, yeah, it fucks up 30% of the time." Bias in AI is bad, but what makes it much much worse is arrogant attitudes claiming it doesn't exist. And you know what, that speaks to the idiocy around AI more than anything. All these debates on AGI are moot, because people are already so keen to believe they got it all figured out it doesn't even matter if the actual tech is any good at all. [0] https://www.wired.com/story/photo-algorithms-id-white-men-fineblack-women-not-so-much/ https://www.wired.com/story/photo-algorithms-id-white-men-fi...

raxxorrax 7y ago
> Wow, the responses to this thread are such an excellent marker for skin colour. As is pronouncing the effects of bias and prejudice. You can almost guarantee the person is white without using any AI. I was or still am pretty critical of AI since the quantification of every property of people will probably not be very enjoyable for anyone. What changed my mind a bit is that there are people wanting to employ it to improve reality. That is far worse and I hadn't had that on my radar before.