8 ms·

Black person here. So, the lack of outrage here (I mean in hacker news) is unsurprising, and would of course be greater if there were more black people "in tec

by jrm4 5y ago

Black person here. So, the lack of outrage here (I mean in hacker news) is unsurprising, and would of course be greater if there were more black people "in tech" and by extension here. And even as I write this, I'm not trying to judge -- just more open the door to understanding, like, y'all understand this fact, right? I find myself searching for metaphorical equivalents. I'm thinking something like "what if it labeled Jewish people as rats" or something similar. Even if that doesn't work -- I suppose the broader point I'd try to make here is: A lot of y'all (and me, often) assume the following very wrong idea, subconsciously -- "Because I'm working with a computer I can assume that what we are doing is infused with objectivity, like math" Important to remember nooooooooope. This is dead wrong. We're ALWAYS bringing our subconscious whatevers to this and there is very little, if ANYTHING at all, that can be correctly considered "coldly and obviously correct and object like math is." (perhaps even what we imagine math to be)

hodgesrm 5y ago
Glad you posted. A lot of AI code simply cannot be trusted. In particularly, it's pretty obvious that the teams using it either don't do a lot of QA or don't understand how to test their systems in the first place. I find it mind-boggling that this technology is so widely used with such limited understanding. The interesting question is how the hell do we fix this?
- jpambrun 5y ago
  One example doesn't support your claim that QA was not sufficient. You will always get misclassifications, most won't be offensive to anyone. This is not a medical device or a self driving car, this is Facebook trying to squeeze more engagement..
  - hodgesrm 5y ago
    This type of misclassification is offensive and for good reason. Similar errors have happened regularly for a long time and in many different systems. [1] The comments about enlarging training sets illustrate the blinders that many technical people have about this technology. There does not seem to be a notion of independent quality assurance to detect what should by now be obvious problems. [1] https://www.bostonmagazine.com/news/2018/02/23/artificial-intelligence-race-dark-skin-bias/ https://www.bostonmagazine.com/news/2018/02/23/artificial-in... Edit: oops, left out source
  - jrm4 5y ago
    I have never seen a more sadly oblivious statement than this: the suggestion that Facebook squeezing engagement is significantly less dangerous than malfunctioning medical devices or self driving cars. Facebook is more dangerous; precisely because the harm in getting it wrong is so much easier to hide.
whoaisme 5y ago
Your comparison of this error to Jews being labelled rats is obviously a comparison made in bad faith. Also I think it's funny that you assume that if there were more black people in the industry there would be more outrage. It seems regardless of what happens, black people want more outrage even when outrage is not justified. Even math is racist and not objective. Jesus.
chuckee 5y ago
> So, the lack of outrage here (I mean in hacker news) is unsurprising Facebook deployed a flawed AI system, and when the flaw was pointed out to them, they disabled it, and apologized. What else would you have them do, that makes this so outrageous? Donate $10 million to racial justice groups [1]? Impose diversity quotas on its law firms [2]? Commit to spend $1 billion with diverse suppliers in 2021, and $100 million specifically with Black-owned businesses [3]? [1] https://www.nbcnews.com/business/consumer/want-know-where-all-those-corporate-donations-blm-are-going-n1225371 https://www.nbcnews.com/business/consumer/want-know-where-al... [2] https://www.nytimes.com/2017/04/02/business/dealbook/facebook-pushes-outside-law-firms-to-become-more-diverse.html https://www.nytimes.com/2017/04/02/business/dealbook/faceboo... [3] https://diversity.fb.com/read-report/ https://diversity.fb.com/read-report/
ratsthrowaway 5y ago
>"what if it labeled Jewish people as rats" Shit man, if only.
suifbwish 5y ago
I also happen to be black but that actually has very little bearing on what I have to say and am merely mentioning it to prevent anyone from suggesting that my opinion on the matter is related to my skin color. The problem with this is no one actually trained the AI to have a racially bias but rather did not train it on recognizing the difference. Why? Probably because no one knew the computer was going to identify this resemblance and immediately label it as such (I have heard babies learning to talk and name shapes/colors do the exact same thing with no outside influence) Computer vision operates on matrices of color coordinates that share resemblances with other similar matrices and attempts to match patterns between multiple sets to establish resemblance. It is not the fault of the computer or the programmer that there are coincidental resemblances that exceed it’s learned threshold where it decides that the two sets represents objects of the same type. I personally am ok with admitting that my skin color visually is more similar to images of certain non human primates and can understand why this problem is not a programmer bias but actually a complicated computer vision problem as it’s easy to spot the resemblance but often difficult to train a neural net on why the images resemble eachother but are not related.
- jrm4 5y ago
  That first sentence, man. Like, I feel bad second guessing people, in some ways it's not my right -- but it so feels like the kind of "respectability" talk I used to say out of what I thought was "the truth," but now realize it was a sort of habitual way to defuse the room. So I will state clearly -- yes, I am aware my opinions are absolutely related to my skin color and that's precisely why you all need to pay special attention to them. Not because of some sort of silly inherent genetic thing, but because I have experiences that many of you don't.
  - jrm4 5y ago
    Said it before, saying it again. I merely said "pay special attention," not "always 100% agree." You've got two black folks here with differing opinions ostensibly ready to discuss. I have no problem with being disagreed with, but I'm still good suggesting that mere downvoting isn't helping.
  - 5y ago
skinkestek 5y ago
White person here: At this point, if a web scale service like this do this it should be grounds for significant sanctions: increasing fines (think GDPR scale) and closer scrutiny (think mandatory inspections and quality controls every month, paid for by the company) should do I think. But, while I'm really annoyed by this and support you I'm also sure there will be enough outrage against it in time from.
- pc86 5y ago
  Fined for what, exactly? They deployed a flawed AI model. The flaw was pointed out, and they removed the model and apologized. I'm genuinely curious what else should have happened.
  - skinkestek 5y ago
    What should have happened is they did basic testing up front? And since they fixed it right away small fines only, like nuisance small. And small follow up.
    - pc86 5y ago
      You didn't answer the question. Fine for what? Are you suggesting broken AI models should incur criminal charges? What exactly are they being fined for?
      - skinkestek 5y ago
        I'm suggesting that when companies that print money have so broken AI models that the confuse dark skinned humans for animals we should incentivize them to fix it. If this ends up giving entry level jobs to people who have a hard time entering tech so they can help testing this, I can live with that too. But frankly, I don't see your point: is it OK to call dark skinned people monkeys if the system who does it is automated && the company who run it is willing to apologize and stop it after getting caught? PS: this is a weird situation for me, I'm a normal white bloke, not woke or anything, I just try to push companies to show basic decency.
Redoubts 5y ago
Now imagine the care taken when applying this technology to dating sites, and other social services; for clustering or recommendation engines, or prediction services. This whole field is a joke, that refuses any introspection.
ant6n 5y ago
I find a lot of the discussion here baffling and tone deaf. There are a lot of comments saying "but humans are primates". It's a dismissal of systemic racism issues coming from a privileged position that's akin to responding the black lives matter movement with "all lives matter". Some of the commenters may exactly think that the algorithm is objective, forgetting that it's built by ppl, for ppl, tested by ppl. If the algorithm kept labelling bearded white dudes as vaginas, it wouldn't be pushed to production, because it would found as a bug and dealt with somehow. But labelling people of color as monkeys is apparently something they're unaware of, which doesn't necessarily point to deliberate racism, but a systemic racism issue. In a lot of discussion here, everyone, on all sides of the argument, is getting voted down. Perhaps this shows this community is not ready for these types of discussions.
- jrm4 5y ago
  Sadly but unsurprisingly, I'm seeing it mostly isn't. But we soldier on.
shadowgovt 5y ago
All of this is correct. Neural nets, in particular, aren't objective... They're bias-learning engines. Their whole raison d'etre is to take (in this case) a bunch of images and go "Okay... Humans believe these random patterns of pixels have semantically different meanings. I'm going to try to explore a space of what those differences could be until they tell me I got it right." If there are things we as humans care about classifying differently that we fail to tell the computer, it won't know to separate them. The fact we keep making this mistake is a real problem. And it's a human problem, not an algorithm problem.
tm11zz 5y ago
Introducing your "metaphorical equivalent" whataboutism doesn't add anything to the conversation except illuminate your own biases.

by jrm4 5y ago

hodgesrm 5y ago

Glad you posted. A lot of AI code simply cannot be trusted. In particularly, it's pretty obvious that the teams using it either don't do a lot of QA or don't understand how to test their systems in the first place. I find it mind-boggling that this technology is so widely used with such limited understanding. The interesting question is how the hell do we fix this?

jpambrun 5y ago
One example doesn't support your claim that QA was not sufficient. You will always get misclassifications, most won't be offensive to anyone. This is not a medical device or a self driving car, this is Facebook trying to squeeze more engagement..
- hodgesrm 5y ago
  This type of misclassification is offensive and for good reason. Similar errors have happened regularly for a long time and in many different systems. [1] The comments about enlarging training sets illustrate the blinders that many technical people have about this technology. There does not seem to be a notion of independent quality assurance to detect what should by now be obvious problems. [1] https://www.bostonmagazine.com/news/2018/02/23/artificial-intelligence-race-dark-skin-bias/ https://www.bostonmagazine.com/news/2018/02/23/artificial-in... Edit: oops, left out source
- jrm4 5y ago
  I have never seen a more sadly oblivious statement than this: the suggestion that Facebook squeezing engagement is significantly less dangerous than malfunctioning medical devices or self driving cars. Facebook is more dangerous; precisely because the harm in getting it wrong is so much easier to hide.

whoaisme 5y ago

Your comparison of this error to Jews being labelled rats is obviously a comparison made in bad faith. Also I think it's funny that you assume that if there were more black people in the industry there would be more outrage. It seems regardless of what happens, black people want more outrage even when outrage is not justified. Even math is racist and not objective. Jesus.

chuckee 5y ago

> So, the lack of outrage here (I mean in hacker news) is unsurprising Facebook deployed a flawed AI system, and when the flaw was pointed out to them, they disabled it, and apologized. What else would you have them do, that makes this so outrageous? Donate $10 million to racial justice groups [1]? Impose diversity quotas on its law firms [2]? Commit to spend $1 billion with diverse suppliers in 2021, and $100 million specifically with Black-owned businesses [3]? [1] https://www.nbcnews.com/business/consumer/want-know-where-all-those-corporate-donations-blm-are-going-n1225371 https://www.nbcnews.com/business/consumer/want-know-where-al... [2] https://www.nytimes.com/2017/04/02/business/dealbook/facebook-pushes-outside-law-firms-to-become-more-diverse.html https://www.nytimes.com/2017/04/02/business/dealbook/faceboo... [3] https://diversity.fb.com/read-report/ https://diversity.fb.com/read-report/

ratsthrowaway 5y ago

>"what if it labeled Jewish people as rats" Shit man, if only.

suifbwish 5y ago

I also happen to be black but that actually has very little bearing on what I have to say and am merely mentioning it to prevent anyone from suggesting that my opinion on the matter is related to my skin color. The problem with this is no one actually trained the AI to have a racially bias but rather did not train it on recognizing the difference. Why? Probably because no one knew the computer was going to identify this resemblance and immediately label it as such (I have heard babies learning to talk and name shapes/colors do the exact same thing with no outside influence) Computer vision operates on matrices of color coordinates that share resemblances with other similar matrices and attempts to match patterns between multiple sets to establish resemblance. It is not the fault of the computer or the programmer that there are coincidental resemblances that exceed it’s learned threshold where it decides that the two sets represents objects of the same type. I personally am ok with admitting that my skin color visually is more similar to images of certain non human primates and can understand why this problem is not a programmer bias but actually a complicated computer vision problem as it’s easy to spot the resemblance but often difficult to train a neural net on why the images resemble eachother but are not related.

jrm4 5y ago
That first sentence, man. Like, I feel bad second guessing people, in some ways it's not my right -- but it so feels like the kind of "respectability" talk I used to say out of what I thought was "the truth," but now realize it was a sort of habitual way to defuse the room. So I will state clearly -- yes, I am aware my opinions are absolutely related to my skin color and that's precisely why you all need to pay special attention to them. Not because of some sort of silly inherent genetic thing, but because I have experiences that many of you don't.
- jrm4 5y ago
  Said it before, saying it again. I merely said "pay special attention," not "always 100% agree." You've got two black folks here with differing opinions ostensibly ready to discuss. I have no problem with being disagreed with, but I'm still good suggesting that mere downvoting isn't helping.
- 5y ago

skinkestek 5y ago

White person here: At this point, if a web scale service like this do this it should be grounds for significant sanctions: increasing fines (think GDPR scale) and closer scrutiny (think mandatory inspections and quality controls every month, paid for by the company) should do I think. But, while I'm really annoyed by this and support you I'm also sure there will be enough outrage against it in time from.

pc86 5y ago
Fined for what, exactly? They deployed a flawed AI model. The flaw was pointed out, and they removed the model and apologized. I'm genuinely curious what else should have happened.
- skinkestek 5y ago
  What should have happened is they did basic testing up front? And since they fixed it right away small fines only, like nuisance small. And small follow up.
  - pc86 5y ago
    You didn't answer the question. Fine for what? Are you suggesting broken AI models should incur criminal charges? What exactly are they being fined for?
    - skinkestek 5y ago
      I'm suggesting that when companies that print money have so broken AI models that the confuse dark skinned humans for animals we should incentivize them to fix it. If this ends up giving entry level jobs to people who have a hard time entering tech so they can help testing this, I can live with that too. But frankly, I don't see your point: is it OK to call dark skinned people monkeys if the system who does it is automated && the company who run it is willing to apologize and stop it after getting caught? PS: this is a weird situation for me, I'm a normal white bloke, not woke or anything, I just try to push companies to show basic decency.

Redoubts 5y ago

Now imagine the care taken when applying this technology to dating sites, and other social services; for clustering or recommendation engines, or prediction services. This whole field is a joke, that refuses any introspection.

ant6n 5y ago

I find a lot of the discussion here baffling and tone deaf. There are a lot of comments saying "but humans are primates". It's a dismissal of systemic racism issues coming from a privileged position that's akin to responding the black lives matter movement with "all lives matter". Some of the commenters may exactly think that the algorithm is objective, forgetting that it's built by ppl, for ppl, tested by ppl. If the algorithm kept labelling bearded white dudes as vaginas, it wouldn't be pushed to production, because it would found as a bug and dealt with somehow. But labelling people of color as monkeys is apparently something they're unaware of, which doesn't necessarily point to deliberate racism, but a systemic racism issue. In a lot of discussion here, everyone, on all sides of the argument, is getting voted down. Perhaps this shows this community is not ready for these types of discussions.

jrm4 5y ago
Sadly but unsurprisingly, I'm seeing it mostly isn't. But we soldier on.

shadowgovt 5y ago

All of this is correct. Neural nets, in particular, aren't objective... They're bias-learning engines. Their whole raison d'etre is to take (in this case) a bunch of images and go "Okay... Humans believe these random patterns of pixels have semantically different meanings. I'm going to try to explore a space of what those differences could be until they tell me I got it right." If there are things we as humans care about classifying differently that we fail to tell the computer, it won't know to separate them. The fact we keep making this mistake is a real problem. And it's a human problem, not an algorithm problem.

tm11zz 5y ago

Introducing your "metaphorical equivalent" whataboutism doesn't add anything to the conversation except illuminate your own biases.