4 ms·

I can't help but shake my head at this whole story. Someone (who by the way describes themselves more as an artist than anything else in public bios) builds a

by creato 6y ago

I can't help but shake my head at this whole story. Someone (who by the way describes themselves more as an artist than anything else in public bios) builds a ML "bullshit artist" (literally) that hallucinates portraits from ridiculously undersampled data. Someone else runs a low resolution image of Obama through a hallucinating computer program, and gets a hallucination. The story should end here at "these hallucinated portraits have basically nothing to do with the input data, they're meaningless". I would bet that input images that are this low resolution are difficult or impossible to determine the race of, no matter how diverse the training data for this model could have been. Note that the skin color of the low resolution Obama and the random "white" guy it generated are basically the same!? I doubt this pipeline will ever be capable of not confusing the race of its input and output. What is on display here is the uselessness of this ML pipeline, not its racist nature. I have seen probably a dozen of examples of this ML pipeline failing hilariously in ways that have nothing to do with race. Yet this failure is suddenly fascinating to the entire tech journalism space as an example of ML algorithms being racist?! And somehow, despite the absurdity of this situation, even ML "luminaries" get drawn into a destructive whirlpool of social justice debate about a hallucinating computer program that doesn't even work sometimes? What the fuck are we all doing here?

karpierz 6y ago
If you look at this single instance, independent of the world around it, it looks ridiculous. You need to look at it in the context of a long[1] list[2] of works[3] detailing this exact problem, and the frustration that an leading expert in the field still doesn't take it seriously, despite the real-world consequences of that attitude. 1) https://www.pnas.org/content/117/14/7684 https://www.pnas.org/content/117/14/7684 2) http://proceedings.mlr.press/v81/buolamwini18a.html?mod=article_inline http://proceedings.mlr.press/v81/buolamwini18a.html?mod=arti... 3) https://www.liebertpub.com/doi/full/10.1089/big.2016.0047 https://www.liebertpub.com/doi/full/10.1089/big.2016.0047
- free_rms 6y ago
  Why not pick all those other hills to die on, then? If you want to talk about statistical methods being a horrible idea for policing, I'm with you. Probably most machine learning experts would be, too, just like most software engineers don't trust electronic voting. Instead, people chose "dataset bias affects machine learning models", the most basic Machine Learning 101 fact ever, to try and debate against. It's like the woke people seek out the weakest possible ground because it generates in-group opposition that they can fight with and out-woke. Stronger ground could have national consensus, way beyond bay area consensus.
  - joshuamorton 6y ago
    > Instead, people chose "dataset bias affects machine learning models", the most basic Machine Learning 101 fact ever, to try and debate against. No one disagrees with that statement. The point that the ethicists are making is that that isn't the whole story, and that focusing only on the lowest hanging fruit (dataset bias) is ultimately harmful since it shifts focus away from the other, deeper problems.
  - cxw 6y ago
    Actually, that's Gebru's point. Gebru was pointing out frustration that LeCun was focusing on this narrow question of statistical bias, rather than the broader social issues you're getting at.
    - saym 6y ago
      Your comment, among the sea of others, got through and made some sense to me. My follow-up questions are: Does that negate the truthfulness of LeCun's response? What would Gebru rather he do or say? I feel like I can't tell what the desired form of action is from the article, and I'm unsure what good it does for removing bias from ML to attack prominent figures (on twitter let alone anywhere public).
- AndrewKemendo 6y ago
  The problem is, the "Lions of AI" do take them seriously - they just aren't beating their drums about it and making it the top thing they shout about. All of the same concerns are discussed in their works about representational data and biased algorithms etc...
- deleted 6y ago
  [deleted]
- fny 6y ago
  I'm not clear about how Yann does not take it seriously? He even admitted respect for Gebru's work. He's just making a technical claim that this is due to data bias. Whether that's true, who knows, but I really don't see how he is being callous in all of this. I'm not sure I understand what the issue is at all.
  - zucker42 6y ago
    To agree with your point, LeCun did many things which indicate he takes this seriously: 1. Responded with a detailed summary of his views. 2. Said he was open to changing his mind in the face of convincing arguments; indicated a desire to learn. 3. Asked people to stop attacking his opponent, in response to unfair attacks on her. 4. Only a week ago, sent multiple tweets out about the problems with a paper which demonstrated racial bias. https://medium.com/@CoalitionForCriticalTechnology/abolish-the-techtoprisonpipeline-9b5b14366b16 https://medium.com/@CoalitionForCriticalTechnology/abolish-t... There's something in between not taking someone seriously and agreeing with their argument.
CoolGuySteve 6y ago
"Garbage In, Garbage Out" apparently dates back to 1957. I remember the expression from the very first computer book I read in the 80s. The idea that this concept, one almost as old as computers and practically a theorem of computation, would somehow be racist or trying to paper over racism is ridiculous.
- kennywinker 6y ago
  GIGO, which in this context would translate to something like "using a bad dataset means there is a problem with the data, not that the tool is racist". I'm not an expert, but I think the argument here is it's far far too easy to have a "garbage" dataset with these tools. I.e. one that only functions correctly for some racial groups (like this one does). That's only racist if you then try to use it in a situation where it comes in contact with multiple racial groups - like, virtually anywhere in the real world.
- ashtonkem 6y ago
  Narrowly scoped to GIGO, the concept can be traced more generally back to Charles Babbage in the 1800s. > On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. I couldn’t tell you when the discussion about bias and unreliability in ML training sets started, not my field of expertise.
KerrickStaley 6y ago
I think you're right that this algorithm doesn't do a good job of reconstructing high res faces from pixelated ones, simply because it can't—there simply isn't enough information in the input, and there is a huge set of real people (from many races) that could map to a given pixelated image. But I do think that when someone who is white (or another race) says "here's a cool thing, check it out" and someone who is black (or another race, different from the first) tries it and realizes it doesn't work for them because of their race, there is a bit of inadvertent racism there. There is an inadvertent exclusion of that second person. I think this happens a lot and in some cases is unavoidable (e.g. language is intertwined with race, and there is a vast amount of information—literature, news, art—that will be inaccessible to you depending on which language(s) you speak), but it is a problem, and it deserves fixing when possible. The implications in this case are currently small—it's a demo. But there is a risk that these biases could make it into real systems and have real impact on people's lives, and to combat that we need more awareness of this effect in the ML community and more research into possible solutions.

by creato 6y ago

karpierz 6y ago

If you look at this single instance, independent of the world around it, it looks ridiculous. You need to look at it in the context of a long[1] list[2] of works[3] detailing this exact problem, and the frustration that an leading expert in the field still doesn't take it seriously, despite the real-world consequences of that attitude. 1) https://www.pnas.org/content/117/14/7684 https://www.pnas.org/content/117/14/7684 2) http://proceedings.mlr.press/v81/buolamwini18a.html?mod=article_inline http://proceedings.mlr.press/v81/buolamwini18a.html?mod=arti... 3) https://www.liebertpub.com/doi/full/10.1089/big.2016.0047 https://www.liebertpub.com/doi/full/10.1089/big.2016.0047

free_rms 6y ago
Why not pick all those other hills to die on, then? If you want to talk about statistical methods being a horrible idea for policing, I'm with you. Probably most machine learning experts would be, too, just like most software engineers don't trust electronic voting. Instead, people chose "dataset bias affects machine learning models", the most basic Machine Learning 101 fact ever, to try and debate against. It's like the woke people seek out the weakest possible ground because it generates in-group opposition that they can fight with and out-woke. Stronger ground could have national consensus, way beyond bay area consensus.
- joshuamorton 6y ago
  > Instead, people chose "dataset bias affects machine learning models", the most basic Machine Learning 101 fact ever, to try and debate against. No one disagrees with that statement. The point that the ethicists are making is that that isn't the whole story, and that focusing only on the lowest hanging fruit (dataset bias) is ultimately harmful since it shifts focus away from the other, deeper problems.
- cxw 6y ago
  Actually, that's Gebru's point. Gebru was pointing out frustration that LeCun was focusing on this narrow question of statistical bias, rather than the broader social issues you're getting at.
  - saym 6y ago
    Your comment, among the sea of others, got through and made some sense to me. My follow-up questions are: Does that negate the truthfulness of LeCun's response? What would Gebru rather he do or say? I feel like I can't tell what the desired form of action is from the article, and I'm unsure what good it does for removing bias from ML to attack prominent figures (on twitter let alone anywhere public).
AndrewKemendo 6y ago
The problem is, the "Lions of AI" do take them seriously - they just aren't beating their drums about it and making it the top thing they shout about. All of the same concerns are discussed in their works about representational data and biased algorithms etc...
deleted 6y ago
[deleted]
fny 6y ago
I'm not clear about how Yann does not take it seriously? He even admitted respect for Gebru's work. He's just making a technical claim that this is due to data bias. Whether that's true, who knows, but I really don't see how he is being callous in all of this. I'm not sure I understand what the issue is at all.
- zucker42 6y ago
  To agree with your point, LeCun did many things which indicate he takes this seriously: 1. Responded with a detailed summary of his views. 2. Said he was open to changing his mind in the face of convincing arguments; indicated a desire to learn. 3. Asked people to stop attacking his opponent, in response to unfair attacks on her. 4. Only a week ago, sent multiple tweets out about the problems with a paper which demonstrated racial bias. https://medium.com/@CoalitionForCriticalTechnology/abolish-the-techtoprisonpipeline-9b5b14366b16 https://medium.com/@CoalitionForCriticalTechnology/abolish-t... There's something in between not taking someone seriously and agreeing with their argument.

CoolGuySteve 6y ago

"Garbage In, Garbage Out" apparently dates back to 1957. I remember the expression from the very first computer book I read in the 80s. The idea that this concept, one almost as old as computers and practically a theorem of computation, would somehow be racist or trying to paper over racism is ridiculous.

kennywinker 6y ago
GIGO, which in this context would translate to something like "using a bad dataset means there is a problem with the data, not that the tool is racist". I'm not an expert, but I think the argument here is it's far far too easy to have a "garbage" dataset with these tools. I.e. one that only functions correctly for some racial groups (like this one does). That's only racist if you then try to use it in a situation where it comes in contact with multiple racial groups - like, virtually anywhere in the real world.
ashtonkem 6y ago
Narrowly scoped to GIGO, the concept can be traced more generally back to Charles Babbage in the 1800s. > On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. I couldn’t tell you when the discussion about bias and unreliability in ML training sets started, not my field of expertise.

KerrickStaley 6y ago

I think you're right that this algorithm doesn't do a good job of reconstructing high res faces from pixelated ones, simply because it can't—there simply isn't enough information in the input, and there is a huge set of real people (from many races) that could map to a given pixelated image. But I do think that when someone who is white (or another race) says "here's a cool thing, check it out" and someone who is black (or another race, different from the first) tries it and realizes it doesn't work for them because of their race, there is a bit of inadvertent racism there. There is an inadvertent exclusion of that second person. I think this happens a lot and in some cases is unavoidable (e.g. language is intertwined with race, and there is a vast amount of information—literature, news, art—that will be inaccessible to you depending on which language(s) you speak), but it is a problem, and it deserves fixing when possible. The implications in this case are currently small—it's a demo. But there is a risk that these biases could make it into real systems and have real impact on people's lives, and to combat that we need more awareness of this effect in the ML community and more research into possible solutions.