9 ms·

"Although we find no evidence the saliency model explicitly encodes male gaze, in cases where the model crops out a woman’s head in favor of their body due to a

"Although we find no evidence the saliency model explicitly encodes male gaze, in cases where the model crops out a woman’s head in favor of their body due to a jersey or lettering, the chosen crop still runs the risk of representational harm for women. In these instances, it is important to remember that users are unaware of the details of the saliency model and the cropping algorithm. Regardless of the underlying mechanism, when an image cropped to a woman’s body area is viewed on social media, due to the historical hyper-sexualization and objectification of women’s bodies, the image runs this risk of being instilled with the notion of male gaze." "Point multipliers are applied for those harms that particularly affect marginalized communities since Twitter’s goal is to responsibly and equitably serve the public conversation." I'm sure Twitter is just responding to the controversy from Fall of 2020 and doing due diligence to address the problem. However, how do you award a bounty for addressing "risk of representational harm" due to historical biases not inherent in the model? Genuine question here and one I'm always curious about. Seems difficult if not impossible.

cryptoz 5y ago
The quote you cited says "no evidence the saliency model explicitly encodes male gaze" emphasis mine. Your re-phrasing however changes the meaning in your question, "historical biases not inherent in the model". Explicitly encodes != inherent in. The difference between the two might answer your question. You can solve for something that is inherent in a system (ie, this), but was not explicitly encoded. They are trying to say it isn't their fault specifically, they find no evidence this was done on purpose or in a targeted way, but that it happens as an accident / lack of testing on the current model.
- stolenmerch 5y ago
  They are very clear about 'intentional' vs. 'unintentional' harm in their judging metric[1], so I get that I think. However, there is no possible way a saliency model could crop an image without someone reading bias into it. E.g. only cropping a jersey, only cropping a face, not cropping at all, etc. can all be signs of historical bias. At some point it just becomes a rorschach test for the reviewer. Not saying discrimination isn't happening or can't happen, just saying this very tenuous and weak connection to "historical bias" interpretation lacks a certain rigor in a bug bounty program or model optimization. [1] https://hackerone.com/twitter-algorithmic-bias https://hackerone.com/twitter-algorithmic-bias
basseq 5y ago
For anyone else wondering about "the controversy from Fall 2020": https://www.theguardian.com/technology/2020/sep/21/twitter-apologises-for-racist-image-cropping-algorithm https://www.theguardian.com/technology/2020/sep/21/twitter-a...
- deleted 5y ago
  [deleted]
bastawhiz 5y ago
> However, how do you award a bounty for addressing "risk of representational harm" due to historical biases not inherent in the model? I think it's pretty straightforward. Most sensible, considerate humans would avoid cropping an image of a woman to her boobs simply because it's insensitive to do so. Just because the machine is trained to highlight text or other visual features doesn't preclude it from ALSO understanding human concepts which are difficult to express to the computer in a straightforward way. There are plenty of ways the model can be improved (e.g., not preferring white faces over Black faces), and they're certainly difficult. If they weren't difficult, Twitter would have simply fixed the problem and there wouldn't be a competition. Arguably, though, if the job can reasonably accomplished manually by a human then a computer should be able to do a similarly effective job. Figuring out how is why it's a challenge. And if we can't figure out how, that's another interesting point in the ongoing conversation about ML.
- stolenmerch 5y ago
  Training a ML model to understand how "most sensible, considerate humans" would act is anything but straightforward. I'm not even sure most people would even consider cropping an athlete at the jersey number regardless of race or gender - it just doesn't make sense yet the machine seemed to do it at the same rate for male vs. female. Retrofitting discrimination onto this result only once you learn the label of the data isn't particularly useful. We want to know how to make good non-harmful predictions in the future.
  - bastawhiz 5y ago
    > Training a ML model to understand how "most sensible, considerate humans" would act is anything but straightforward. This is exactly why it's a challenge! The point is that the goal should be to do it the way a human would find satisfactory and not the way that's easy. Even if the machine wasn't trained to be biased, the machine should still produce results which people see as good. We didn't invent machines to do jobs just so people could say "but that's a bad result" and reply with "yes but the machine wouldn't know that". We should strive for better.
- naasking 5y ago
hncurious 5y ago
These puritan activists larping as tech savvy victorian moralists should be ignored. While the “male gaze” jibe sounds enlightened and à la mode, it implicitly denies that women (especially the lesbian ones) have a gaze of their own, or asserts that it has been colonised by the omnipotent male gaze. Don't take my word for it, here's noted feminist and professor Camille Paglia and Christina Hoff Sommers on the absurdity of the male gaze theory: https://www.youtube.com/watch?v=a3pjGd1v5xY https://www.youtube.com/watch?v=a3pjGd1v5xY
- jauer 5y ago
  Yet somehow lesbians aren't the ones eyefucking my breasts when I'm at the grocery store.
  - djoldman 5y ago
    I think this is an interesting comment and I immediately thought to ask: What percentage of the people offending you by looking at you like that have you confirmed are male and what percentage have you confirmed are not gay? I'm asking without snark, I'm genuinely curious if you've asked people.
    - jauer 5y ago
      Didn't say I was offended, but let me elaborate on that part, because I think some guys get confused on what's offensive. Look, I like guys. I like it when guys appreciate my appearance. They have to look at me to do so. I'm not even offended when they mutter an "mmm hmm" or whatever. The offensive part comes when someone, no matter gender or sexuality, express their appreciation (or attempt to demonstrate power over me) in an manner that hijacks my attention or disrupts my activities. They do this by: 1. stopping at the end of an aisle, looking you up and down, and staring at $BODY_PART for several seconds. 2. Initiating small-talk from a non-interpersonal distance and being persistent about it if you don't respond (there's a difference in expression and tone vs. being neighborly or whatever). There's a difference between "hi!" or "excuse me" as you move past and "dang girl, how you doing?" 3. Making comments and then following you around (sometimes continuing to comment unless you stop and say _something_ to sooth their ego). re: How many gay: I have yet to have a gay guy hit on me, or check me out. Some do judge my outfits. I have a reply elsewhere on the difference between normal perception, judging clothing, and "male gaze" (which TBH should be called "aggressive sexualized gaze" for precision and to avoid discussions like this, even though the shorter form is precise enough. re: How many male: I dunno, the masc-presenting trans people I know are too self-conscious about appearing creepy to engage in these behaviors. So are most of the techies I run into :) The straight trans-masc guys I know are also tend to fit in the "def dudes that appreciate but aren't creepy about it" category.
IfOnlyYouKnew 5y ago
I believe the principle is fairly easy to understand: imagine you're missing two fingers. Because kids are cruel, your middle school experience involved lots of shaming and name-calling, and that really got to you at the time. Everything is fine now, because nobody actually cares. But then along comes this algorithm and, for some reason, it crops your profile picture to be just that hand of yours. Of course one or two "friends" from middle school are following you on twitter, and start reminiscing about the times they called you three-finger-bob. Is it conceivable/realistic/justified that this would, at least for a while, hurt? Meanwhile, for someone who didn't go through that experience, the algorithm doing the very same thing is just... a curious bug? As to how they are going to score this: it cannot be scored only quantitively, as they specifically say in the description. They are going to read your hypothesis–something like the above, but maybe a little more serious–and score it, probably along a a few categories and by a few different people etc.
- stolenmerch 5y ago
  A better comparison would be if there was some salient feature surrounding my hands such as me holding an object. The object would be the equivalent to the jersey number. It's clearly not cropping on the thing that makes me feel bad but it did anyway. To avoid this, it has to be trained to take into account all possible human reactions. Basically, it has to know that these pixel values activate some social controversy and it's best to avoid. Not saying this isn't possible, but it seems absolutely fraught with peril and potentially more harmful than the original saliency, especially for social constructs with no clear consensus.
  - IfOnlyYouKnew 5y ago
    Yes, I didn't mean to suggest any specific reason for the algorithm's output, because it doesn't matter. This issue isn't about anyone's "guilt", least of all the algorithm's. It's about harm. Harm is to be avoided, even if it is the result of a completely benign process with no ill intentions. And you aren't going to be able to explain away some decision that causes harm by explaining the algorithm. Even knowing everything about the algorithm, I would prefer it doesn't focus on whatever my weak spot is. And even knowing how GPT-3 works, I still tend to anthromorphize it. To some approximation, exactly nobody in the real world is going to give the company the benefit of the doubt and study its algorithm, nor should they be expected to. It's like that escaped lion that keeps eating the schoolchildren: it's what lions do, an expression of its basic lioness! It's not evil, guilty, or even culpable: it just can't help itself, but to help itself to an early dinner. And yet, we are going to shoot the lion. Not as punishment, but as a simple, mechanistic, prevention of harm to people, who, in our system of values, rank higher. Algorithms rank far below lions. An algorithm that causes harm, no matter how or why, goes the way of php (taken out back and shot, unless it's run by Facebook). And anything that happens is considered to be caused by the algorithm, even if some humans happen to provoke it by somehow being different than other humans, or our expectation of rationality. Because we cannot change humans, and because nobody should be expected to change to accommodate technology, especially if they were never asked if they want that technology.
nitwit005 5y ago
Fixing the AI to avoid such issues is indeed likely impossible, but adding a feature to correct the AI's decision would probably take a week or two of engineering effort.

cryptoz 5y ago

The quote you cited says "no evidence the saliency model explicitly encodes male gaze" emphasis mine. Your re-phrasing however changes the meaning in your question, "historical biases not inherent in the model". Explicitly encodes != inherent in. The difference between the two might answer your question. You can solve for something that is inherent in a system (ie, this), but was not explicitly encoded. They are trying to say it isn't their fault specifically, they find no evidence this was done on purpose or in a targeted way, but that it happens as an accident / lack of testing on the current model.

stolenmerch 5y ago
They are very clear about 'intentional' vs. 'unintentional' harm in their judging metric[1], so I get that I think. However, there is no possible way a saliency model could crop an image without someone reading bias into it. E.g. only cropping a jersey, only cropping a face, not cropping at all, etc. can all be signs of historical bias. At some point it just becomes a rorschach test for the reviewer. Not saying discrimination isn't happening or can't happen, just saying this very tenuous and weak connection to "historical bias" interpretation lacks a certain rigor in a bug bounty program or model optimization. [1] https://hackerone.com/twitter-algorithmic-bias https://hackerone.com/twitter-algorithmic-bias

basseq 5y ago

For anyone else wondering about "the controversy from Fall 2020": https://www.theguardian.com/technology/2020/sep/21/twitter-apologises-for-racist-image-cropping-algorithm https://www.theguardian.com/technology/2020/sep/21/twitter-a...

deleted 5y ago
[deleted]

bastawhiz 5y ago

> However, how do you award a bounty for addressing "risk of representational harm" due to historical biases not inherent in the model? I think it's pretty straightforward. Most sensible, considerate humans would avoid cropping an image of a woman to her boobs simply because it's insensitive to do so. Just because the machine is trained to highlight text or other visual features doesn't preclude it from ALSO understanding human concepts which are difficult to express to the computer in a straightforward way. There are plenty of ways the model can be improved (e.g., not preferring white faces over Black faces), and they're certainly difficult. If they weren't difficult, Twitter would have simply fixed the problem and there wouldn't be a competition. Arguably, though, if the job can reasonably accomplished manually by a human then a computer should be able to do a similarly effective job. Figuring out how is why it's a challenge. And if we can't figure out how, that's another interesting point in the ongoing conversation about ML.

stolenmerch 5y ago
Training a ML model to understand how "most sensible, considerate humans" would act is anything but straightforward. I'm not even sure most people would even consider cropping an athlete at the jersey number regardless of race or gender - it just doesn't make sense yet the machine seemed to do it at the same rate for male vs. female. Retrofitting discrimination onto this result only once you learn the label of the data isn't particularly useful. We want to know how to make good non-harmful predictions in the future.
- bastawhiz 5y ago
  > Training a ML model to understand how "most sensible, considerate humans" would act is anything but straightforward. This is exactly why it's a challenge! The point is that the goal should be to do it the way a human would find satisfactory and not the way that's easy. Even if the machine wasn't trained to be biased, the machine should still produce results which people see as good. We didn't invent machines to do jobs just so people could say "but that's a bad result" and reply with "yes but the machine wouldn't know that". We should strive for better.
naasking 5y ago

hncurious 5y ago

These puritan activists larping as tech savvy victorian moralists should be ignored. While the “male gaze” jibe sounds enlightened and à la mode, it implicitly denies that women (especially the lesbian ones) have a gaze of their own, or asserts that it has been colonised by the omnipotent male gaze. Don't take my word for it, here's noted feminist and professor Camille Paglia and Christina Hoff Sommers on the absurdity of the male gaze theory: https://www.youtube.com/watch?v=a3pjGd1v5xY https://www.youtube.com/watch?v=a3pjGd1v5xY

jauer 5y ago
Yet somehow lesbians aren't the ones eyefucking my breasts when I'm at the grocery store.
- djoldman 5y ago
  I think this is an interesting comment and I immediately thought to ask: What percentage of the people offending you by looking at you like that have you confirmed are male and what percentage have you confirmed are not gay? I'm asking without snark, I'm genuinely curious if you've asked people.
  - jauer 5y ago
    Didn't say I was offended, but let me elaborate on that part, because I think some guys get confused on what's offensive. Look, I like guys. I like it when guys appreciate my appearance. They have to look at me to do so. I'm not even offended when they mutter an "mmm hmm" or whatever. The offensive part comes when someone, no matter gender or sexuality, express their appreciation (or attempt to demonstrate power over me) in an manner that hijacks my attention or disrupts my activities. They do this by: 1. stopping at the end of an aisle, looking you up and down, and staring at $BODY_PART for several seconds. 2. Initiating small-talk from a non-interpersonal distance and being persistent about it if you don't respond (there's a difference in expression and tone vs. being neighborly or whatever). There's a difference between "hi!" or "excuse me" as you move past and "dang girl, how you doing?" 3. Making comments and then following you around (sometimes continuing to comment unless you stop and say _something_ to sooth their ego). re: How many gay: I have yet to have a gay guy hit on me, or check me out. Some do judge my outfits. I have a reply elsewhere on the difference between normal perception, judging clothing, and "male gaze" (which TBH should be called "aggressive sexualized gaze" for precision and to avoid discussions like this, even though the shorter form is precise enough. re: How many male: I dunno, the masc-presenting trans people I know are too self-conscious about appearing creepy to engage in these behaviors. So are most of the techies I run into :) The straight trans-masc guys I know are also tend to fit in the "def dudes that appreciate but aren't creepy about it" category.

IfOnlyYouKnew 5y ago

I believe the principle is fairly easy to understand: imagine you're missing two fingers. Because kids are cruel, your middle school experience involved lots of shaming and name-calling, and that really got to you at the time. Everything is fine now, because nobody actually cares. But then along comes this algorithm and, for some reason, it crops your profile picture to be just that hand of yours. Of course one or two "friends" from middle school are following you on twitter, and start reminiscing about the times they called you three-finger-bob. Is it conceivable/realistic/justified that this would, at least for a while, hurt? Meanwhile, for someone who didn't go through that experience, the algorithm doing the very same thing is just... a curious bug? As to how they are going to score this: it cannot be scored only quantitively, as they specifically say in the description. They are going to read your hypothesis–something like the above, but maybe a little more serious–and score it, probably along a a few categories and by a few different people etc.

stolenmerch 5y ago
A better comparison would be if there was some salient feature surrounding my hands such as me holding an object. The object would be the equivalent to the jersey number. It's clearly not cropping on the thing that makes me feel bad but it did anyway. To avoid this, it has to be trained to take into account all possible human reactions. Basically, it has to know that these pixel values activate some social controversy and it's best to avoid. Not saying this isn't possible, but it seems absolutely fraught with peril and potentially more harmful than the original saliency, especially for social constructs with no clear consensus.
- IfOnlyYouKnew 5y ago
  Yes, I didn't mean to suggest any specific reason for the algorithm's output, because it doesn't matter. This issue isn't about anyone's "guilt", least of all the algorithm's. It's about harm. Harm is to be avoided, even if it is the result of a completely benign process with no ill intentions. And you aren't going to be able to explain away some decision that causes harm by explaining the algorithm. Even knowing everything about the algorithm, I would prefer it doesn't focus on whatever my weak spot is. And even knowing how GPT-3 works, I still tend to anthromorphize it. To some approximation, exactly nobody in the real world is going to give the company the benefit of the doubt and study its algorithm, nor should they be expected to. It's like that escaped lion that keeps eating the schoolchildren: it's what lions do, an expression of its basic lioness! It's not evil, guilty, or even culpable: it just can't help itself, but to help itself to an early dinner. And yet, we are going to shoot the lion. Not as punishment, but as a simple, mechanistic, prevention of harm to people, who, in our system of values, rank higher. Algorithms rank far below lions. An algorithm that causes harm, no matter how or why, goes the way of php (taken out back and shot, unless it's run by Facebook). And anything that happens is considered to be caused by the algorithm, even if some humans happen to provoke it by somehow being different than other humans, or our expectation of rationality. Because we cannot change humans, and because nobody should be expected to change to accommodate technology, especially if they were never asked if they want that technology.

nitwit005 5y ago

Fixing the AI to avoid such issues is indeed likely impossible, but adding a feature to correct the AI's decision would probably take a week or two of engineering effort.