3 ms·

> However, how do you award a bounty for addressing "risk of representational harm" due to historical biases not inherent in the model? I think it's pretty str

> However, how do you award a bounty for addressing "risk of representational harm" due to historical biases not inherent in the model? I think it's pretty straightforward. Most sensible, considerate humans would avoid cropping an image of a woman to her boobs simply because it's insensitive to do so. Just because the machine is trained to highlight text or other visual features doesn't preclude it from ALSO understanding human concepts which are difficult to express to the computer in a straightforward way. There are plenty of ways the model can be improved (e.g., not preferring white faces over Black faces), and they're certainly difficult. If they weren't difficult, Twitter would have simply fixed the problem and there wouldn't be a competition. Arguably, though, if the job can reasonably accomplished manually by a human then a computer should be able to do a similarly effective job. Figuring out how is why it's a challenge. And if we can't figure out how, that's another interesting point in the ongoing conversation about ML.

stolenmerch 5y ago
Training a ML model to understand how "most sensible, considerate humans" would act is anything but straightforward. I'm not even sure most people would even consider cropping an athlete at the jersey number regardless of race or gender - it just doesn't make sense yet the machine seemed to do it at the same rate for male vs. female. Retrofitting discrimination onto this result only once you learn the label of the data isn't particularly useful. We want to know how to make good non-harmful predictions in the future.
- bastawhiz 5y ago
  > Training a ML model to understand how "most sensible, considerate humans" would act is anything but straightforward. This is exactly why it's a challenge! The point is that the goal should be to do it the way a human would find satisfactory and not the way that's easy. Even if the machine wasn't trained to be biased, the machine should still produce results which people see as good. We didn't invent machines to do jobs just so people could say "but that's a bad result" and reply with "yes but the machine wouldn't know that". We should strive for better.
naasking 5y ago
> Just because the machine is trained to highlight text or other visual features doesn't preclude it from ALSO understanding human concepts which are difficult to express to the computer in a straightforward way. I honestly don't know how you reached this conclusion. You're skirting the line of contradicting yourself. > Arguably, though, if the job can reasonably accomplished manually by a human then a computer should be able to do a similarly effective job. I also don't see how this could possibly be true. Certainly the sphere of tasks that computers competently handle compared to humans is growing, but it's nowhere near "any job a human can reasonably do".
- bastawhiz 5y ago
  > I also don't see how this could possibly be true. Certainly the sphere of tasks that computers competently handle compared to humans is growing, but it's nowhere near "any job a human can reasonably do". If we admit it's a job that a computer cannot reasonably do, then why do we have a computer doing it in the first place? We shouldn't accept the status quo with "well it's okay-enough" if it has limitations that are significant (and frequent!) enough to cause a large controversy. In fact, Twitter's response was _to remove the algorithm from production_. The whole point of this challenge is to find out if there's a way to automate this task well. It doesn't have to be perfect, it has to be good enough that the times where it's wrong are not blatant and reproducible, like when this was initially made apparent: https://www.theguardian.com/technology/2020/sep/21/twitter-apologises-for-racist-image-cropping-algorithm https://www.theguardian.com/technology/2020/sep/21/twitter-a...
  - naasking 5y ago
    > If we admit it's a job that a computer cannot reasonably do, then why do we have a computer doing it in the first place? That's not what you said. You said if it's a job that can be reasonably done by a human, then a computer should be similarly effective. That's clearly false. > if it has limitations that are significant (and frequent!) enough to cause a large controversy. Controversies are not necessarily a sign that a problem is significant. Consider all the "war on Christmas" nonsense.
- 5y ago
acituan 5y ago
> Most sensible, considerate humans would avoid cropping an image of a woman to her boobs simply because it's insensitive to do so The manners of sexualization of the human form, even nudity, is not a human universal, not even a western universal, eg Nordics and Germany. Even though through omission, this move is still overemphasizing the sexuality of breasts, which is basically pushing American cultural sensitivities upon the world.
- bastawhiz 5y ago
  You're very much missing the point. The machine obviously isn't intentionally sexualizing anyone, but it's producing a bad result, and not only is it bad, it can be perceived as sexualization (regardless of whether there's bias or not). The machine lacks understanding, producing a bad result, and the bad result is Extra Bad for some people. Let's say I started a service and designed a machine to produce nice textile patterns for my customers based on their perceived preferences. If the machine started producing some ugly textiles with patterns that could be perceived as swastikas, the answer is not to say "well there are many cultures where the swastika is not a harmful symbol and we never trained the machine on nazi data". The answer is to look at why the machine went in that direction in the first place and change it to not make ugly patterns, and maybe teach it "there are some people who don't like swastikas, maybe avoid making those". It's a machine built to serve humans, and if it's not serving the humans in way that the humans say is good, it should be changed. There's no business loss to having a no-swastika policy, just as there's no business loss that says "don't zoom in on boobs for photos where the boobs aren't the point of the photo". This problem has _nothing_ to do with sensitivities, it's about teaching the machine to crop images in an intelligent way. Even if you weren't offended by the result of a machine cropping an image in a sexualized way, most folks would agree that cropping the image to the text on a jersey is not the right output of that model. Being offensive to women with American sensibilities (a huge portion of Twitter's users, I might add[0]) is a side effect of the machine doing a crappy job in the first place. [0] https://www.statista.com/statistics/242606/number-of-active-twitter-users-in-selected-countries/ https://www.statista.com/statistics/242606/number-of-active-...
  - acituan 5y ago

stolenmerch 5y ago

Training a ML model to understand how "most sensible, considerate humans" would act is anything but straightforward. I'm not even sure most people would even consider cropping an athlete at the jersey number regardless of race or gender - it just doesn't make sense yet the machine seemed to do it at the same rate for male vs. female. Retrofitting discrimination onto this result only once you learn the label of the data isn't particularly useful. We want to know how to make good non-harmful predictions in the future.

bastawhiz 5y ago
> Training a ML model to understand how "most sensible, considerate humans" would act is anything but straightforward. This is exactly why it's a challenge! The point is that the goal should be to do it the way a human would find satisfactory and not the way that's easy. Even if the machine wasn't trained to be biased, the machine should still produce results which people see as good. We didn't invent machines to do jobs just so people could say "but that's a bad result" and reply with "yes but the machine wouldn't know that". We should strive for better.

naasking 5y ago

> Just because the machine is trained to highlight text or other visual features doesn't preclude it from ALSO understanding human concepts which are difficult to express to the computer in a straightforward way. I honestly don't know how you reached this conclusion. You're skirting the line of contradicting yourself. > Arguably, though, if the job can reasonably accomplished manually by a human then a computer should be able to do a similarly effective job. I also don't see how this could possibly be true. Certainly the sphere of tasks that computers competently handle compared to humans is growing, but it's nowhere near "any job a human can reasonably do".

bastawhiz 5y ago
> I also don't see how this could possibly be true. Certainly the sphere of tasks that computers competently handle compared to humans is growing, but it's nowhere near "any job a human can reasonably do". If we admit it's a job that a computer cannot reasonably do, then why do we have a computer doing it in the first place? We shouldn't accept the status quo with "well it's okay-enough" if it has limitations that are significant (and frequent!) enough to cause a large controversy. In fact, Twitter's response was _to remove the algorithm from production_. The whole point of this challenge is to find out if there's a way to automate this task well. It doesn't have to be perfect, it has to be good enough that the times where it's wrong are not blatant and reproducible, like when this was initially made apparent: https://www.theguardian.com/technology/2020/sep/21/twitter-apologises-for-racist-image-cropping-algorithm https://www.theguardian.com/technology/2020/sep/21/twitter-a...
- naasking 5y ago
  > If we admit it's a job that a computer cannot reasonably do, then why do we have a computer doing it in the first place? That's not what you said. You said if it's a job that can be reasonably done by a human, then a computer should be similarly effective. That's clearly false. > if it has limitations that are significant (and frequent!) enough to cause a large controversy. Controversies are not necessarily a sign that a problem is significant. Consider all the "war on Christmas" nonsense.
5y ago

acituan 5y ago

> Most sensible, considerate humans would avoid cropping an image of a woman to her boobs simply because it's insensitive to do so The manners of sexualization of the human form, even nudity, is not a human universal, not even a western universal, eg Nordics and Germany. Even though through omission, this move is still overemphasizing the sexuality of breasts, which is basically pushing American cultural sensitivities upon the world.

bastawhiz 5y ago
You're very much missing the point. The machine obviously isn't intentionally sexualizing anyone, but it's producing a bad result, and not only is it bad, it can be perceived as sexualization (regardless of whether there's bias or not). The machine lacks understanding, producing a bad result, and the bad result is Extra Bad for some people. Let's say I started a service and designed a machine to produce nice textile patterns for my customers based on their perceived preferences. If the machine started producing some ugly textiles with patterns that could be perceived as swastikas, the answer is not to say "well there are many cultures where the swastika is not a harmful symbol and we never trained the machine on nazi data". The answer is to look at why the machine went in that direction in the first place and change it to not make ugly patterns, and maybe teach it "there are some people who don't like swastikas, maybe avoid making those". It's a machine built to serve humans, and if it's not serving the humans in way that the humans say is good, it should be changed. There's no business loss to having a no-swastika policy, just as there's no business loss that says "don't zoom in on boobs for photos where the boobs aren't the point of the photo". This problem has _nothing_ to do with sensitivities, it's about teaching the machine to crop images in an intelligent way. Even if you weren't offended by the result of a machine cropping an image in a sexualized way, most folks would agree that cropping the image to the text on a jersey is not the right output of that model. Being offensive to women with American sensibilities (a huge portion of Twitter's users, I might add[0]) is a side effect of the machine doing a crappy job in the first place. [0] https://www.statista.com/statistics/242606/number-of-active-twitter-users-in-selected-countries/ https://www.statista.com/statistics/242606/number-of-active-...
- acituan 5y ago