7 ms·

When Data Science Destabilizes Democracy and Facilitates Genocide

32 points by thuuuomas 9y ago

nl 9y ago
Oh please. This article is completely correct of course. But there is zero hope that it'll be taken seriously at all. Just the other day, here on HN someone seriously proposed the idea that hate speech doesn't lead to violence against minorities[1]. I pointed at the Rwanda genocide, and got voted down (a lot), because the OP claimed that 10% of deaths didn't prove my point. Then I pointed out that was at least 60,000 deaths, and that was downvoted too. Enough people clearly want to ignore things which disagree with their world view that there is zero chance that this destabilising behaviour will stop. https://news.ycombinator.com/item?id=15936785 https://news.ycombinator.com/item?id=15936785
- jlg23 9y ago
  You told people to look up a topic but did not provide any link as a recommended starting point nor any justification why Rwanda's history is relevant in the context of hate speech; now you just refer to yourself.
  - nl 9y ago
    I have zero sympathy for someone who claims to want studies, is referenced an incident, looks it up, sees the reference and claims it says the opposite of what it does. Googling 'Rwanda hatespeech' isn't hard.
  - anigbrowl 9y ago
    You shouldn't need to provide links for every topic of general knowledge, like the factual existence of wars that have taken place within living memory or are part of an educational curriculum. Wikipedia is an adequate place to begin researching a topic like that. If you can't see why Rwanda's example is relevant in this context then I urge you become more informed about the topic.
    - jlg23 9y ago
      I am actually very familiar with the topic and I am thus aware that one can find many, seemingly contradictory, sources. But thanks anyway. I was simply justifying my downvote.
- dang 9y ago
  When you post like this, what people hear is the story of how right you were on the internet, how wrong others were, and how they cravenly downvoted you. This only leads to more downvotes, and rightly so, since it's off topic and also a kind of pose. This doesn't mean, of course, that you're wrong about genocide, ethics, Rwanda, etc. It means that if you want to post about these things you're responsible for separating out the dross that people will otherwise react to. The contrast between the magnitude of those topics and the pettiness of I-was-right-on-the-internet only calls attention to the latter.
  - nl 9y ago
    I hesitated writing this because of the push back I knew I'd get. But I think YCombinator needs to take this problem seriously. The Sam Altmann piece made it completely clear he has no idea how some of the ideas he thinks are bad are promoted and grow on HN. The fact we can't talk about how ridiculous downvoted on that post are means it just keeps happening. I think there is complete blindness about how completely outrageous that is, and how it reinforces the negative stereotypes of silicon valley. I probably could argue the point better, but honestly I don't think arguing about it is the point. Really, all the moderation team here could read this article.
    - dang 9y ago
      I agree that this and many other things are more important than most topics on HN. Does that mean we should moderate HN with that in mind? Tempting as that sounds, I think it would do more harm than good. We need to remember what HN is and what it isn't. HN's mandate is the gratification of intellectual curiosity. Intellectual curiosity is playful, wants to learn, wants variety. It is more fond of trinkets than of causes. Its relationship to More Important Things is itself curious: it inevitably touches on larger topics, yet if we were to swap priorities and put More Important Things first and curiosity second, I think it would kill the site. Certainly it would turn it into a fundamentally different thing, and you can't do that to something alive without killing it. So the question is not "are there more important things in the world", it's "does a site dedicated to intellectual curiosity have a right to exist, even though there are more important things in the world". I think it does, because there is room for many kinds of website. This is just the kind of website that HN is. If you accept that, you need to understand that such a site, simply by being interesting, comes under constant pressure to be something else. From a moderation viewpoint it's our job to protect it for what it is and resist the pressure to turn it into something else, even when we know the something elses are more important. Often we personally agree with the very things we moderate. That sounds weird if not impossible, but sheer quantity has a way of changing a person and you get pretty good at it after a while. When you say "some of the ideas he thinks are bad are promoted and grow on HN" I don't know what ideas you're talking about. But if you mean ideological ideas, yes, they certainly crop up here, but I don't believe they're growing here. I think they're floating around everywhere. They also damage intellectual curiosity and so there's an easy argument that they don't belong on HN. Convincing everyone else of that of course is not so easy.
gt_ 9y ago
Most it this made sense to me, but the section on ‘bias’ seemed to leave the ground a little more than I think is healthy. The problem is correctly identified. The solutions offered are OK, but the examples and concluded causes appear unreasoned. Probably the clearest example is in the section on stereotypes. Again, the problems are solidly identified but the understandings of their causes are lacking. The implication of surprise that an algorithm with real-world data would associate being a doctor with being a male and being a nurse with being a female is highly suspicious. There’s plenty of reason to assume a social bias feedback loop as discussed in the beginning of the article, but the argument that this is a result of bias rsearch is unsound. This correlation is consistent in the statistical data, in all societies and cultures on Earth. Again, there’s plenty of reason to argue the results correlate with societal gender bias, but not researcher gender bias. Big difference. The race/skin color correlations argues in the article do have sound arguments for researcher bias.
- nl 9y ago
  The whole point of this article is that good data science involve detecting and avoiding these biased feedback loops, because software can amplify them. It literally says: “It’s not that data can be biased. Data is biased.” Know how your data was generated and what biases it may contain. We are encoding and even amplifying societal biases in the algorithms we create.
  - gt_ 9y ago
    ??? I just read the article... not sure what you're point is. My point is that the author "encoded and even amplified" bias that the data did not have. Most of the article concerns being aware of inherent bias in data. My point concerns bias that the author has ascribed to the data.
    - nl 9y ago
      Gender bias in Word2Vec data is a well known problem. The article provides references. It's an unsupervised algorithm, and the Google pretrained vectors are trained from news coverage, so not really a researcher issue (insofar as they select an unusual datasets or something). Edit: to clarify, your claim is that Word2Vec data isn't biased even though there is a link right there showing how it is? Why do you think that? If you use that data in a system then you reinforce that bias.

32 points by thuuuomas 9y ago

nl 9y ago

Oh please. This article is completely correct of course. But there is zero hope that it'll be taken seriously at all. Just the other day, here on HN someone seriously proposed the idea that hate speech doesn't lead to violence against minorities[1]. I pointed at the Rwanda genocide, and got voted down (a lot), because the OP claimed that 10% of deaths didn't prove my point. Then I pointed out that was at least 60,000 deaths, and that was downvoted too. Enough people clearly want to ignore things which disagree with their world view that there is zero chance that this destabilising behaviour will stop. https://news.ycombinator.com/item?id=15936785 https://news.ycombinator.com/item?id=15936785

jlg23 9y ago
You told people to look up a topic but did not provide any link as a recommended starting point nor any justification why Rwanda's history is relevant in the context of hate speech; now you just refer to yourself.
- nl 9y ago
  I have zero sympathy for someone who claims to want studies, is referenced an incident, looks it up, sees the reference and claims it says the opposite of what it does. Googling 'Rwanda hatespeech' isn't hard.
- anigbrowl 9y ago
  You shouldn't need to provide links for every topic of general knowledge, like the factual existence of wars that have taken place within living memory or are part of an educational curriculum. Wikipedia is an adequate place to begin researching a topic like that. If you can't see why Rwanda's example is relevant in this context then I urge you become more informed about the topic.
  - jlg23 9y ago
    I am actually very familiar with the topic and I am thus aware that one can find many, seemingly contradictory, sources. But thanks anyway. I was simply justifying my downvote.
dang 9y ago
When you post like this, what people hear is the story of how right you were on the internet, how wrong others were, and how they cravenly downvoted you. This only leads to more downvotes, and rightly so, since it's off topic and also a kind of pose. This doesn't mean, of course, that you're wrong about genocide, ethics, Rwanda, etc. It means that if you want to post about these things you're responsible for separating out the dross that people will otherwise react to. The contrast between the magnitude of those topics and the pettiness of I-was-right-on-the-internet only calls attention to the latter.
- nl 9y ago
  I hesitated writing this because of the push back I knew I'd get. But I think YCombinator needs to take this problem seriously. The Sam Altmann piece made it completely clear he has no idea how some of the ideas he thinks are bad are promoted and grow on HN. The fact we can't talk about how ridiculous downvoted on that post are means it just keeps happening. I think there is complete blindness about how completely outrageous that is, and how it reinforces the negative stereotypes of silicon valley. I probably could argue the point better, but honestly I don't think arguing about it is the point. Really, all the moderation team here could read this article.
  - dang 9y ago
    I agree that this and many other things are more important than most topics on HN. Does that mean we should moderate HN with that in mind? Tempting as that sounds, I think it would do more harm than good. We need to remember what HN is and what it isn't. HN's mandate is the gratification of intellectual curiosity. Intellectual curiosity is playful, wants to learn, wants variety. It is more fond of trinkets than of causes. Its relationship to More Important Things is itself curious: it inevitably touches on larger topics, yet if we were to swap priorities and put More Important Things first and curiosity second, I think it would kill the site. Certainly it would turn it into a fundamentally different thing, and you can't do that to something alive without killing it. So the question is not "are there more important things in the world", it's "does a site dedicated to intellectual curiosity have a right to exist, even though there are more important things in the world". I think it does, because there is room for many kinds of website. This is just the kind of website that HN is. If you accept that, you need to understand that such a site, simply by being interesting, comes under constant pressure to be something else. From a moderation viewpoint it's our job to protect it for what it is and resist the pressure to turn it into something else, even when we know the something elses are more important. Often we personally agree with the very things we moderate. That sounds weird if not impossible, but sheer quantity has a way of changing a person and you get pretty good at it after a while. When you say "some of the ideas he thinks are bad are promoted and grow on HN" I don't know what ideas you're talking about. But if you mean ideological ideas, yes, they certainly crop up here, but I don't believe they're growing here. I think they're floating around everywhere. They also damage intellectual curiosity and so there's an easy argument that they don't belong on HN. Convincing everyone else of that of course is not so easy.

gt_ 9y ago

Most it this made sense to me, but the section on ‘bias’ seemed to leave the ground a little more than I think is healthy. The problem is correctly identified. The solutions offered are OK, but the examples and concluded causes appear unreasoned. Probably the clearest example is in the section on stereotypes. Again, the problems are solidly identified but the understandings of their causes are lacking. The implication of surprise that an algorithm with real-world data would associate being a doctor with being a male and being a nurse with being a female is highly suspicious. There’s plenty of reason to assume a social bias feedback loop as discussed in the beginning of the article, but the argument that this is a result of bias rsearch is unsound. This correlation is consistent in the statistical data, in all societies and cultures on Earth. Again, there’s plenty of reason to argue the results correlate with societal gender bias, but not researcher gender bias. Big difference. The race/skin color correlations argues in the article do have sound arguments for researcher bias.

nl 9y ago
The whole point of this article is that good data science involve detecting and avoiding these biased feedback loops, because software can amplify them. It literally says: “It’s not that data can be biased. Data is biased.” Know how your data was generated and what biases it may contain. We are encoding and even amplifying societal biases in the algorithms we create.
- gt_ 9y ago
  ??? I just read the article... not sure what you're point is. My point is that the author "encoded and even amplified" bias that the data did not have. Most of the article concerns being aware of inherent bias in data. My point concerns bias that the author has ascribed to the data.
  - nl 9y ago
    Gender bias in Word2Vec data is a well known problem. The article provides references. It's an unsupervised algorithm, and the Google pretrained vectors are trained from news coverage, so not really a researcher issue (insofar as they select an unusual datasets or something). Edit: to clarify, your claim is that Word2Vec data isn't biased even though there is a link right there showing how it is? Why do you think that? If you use that data in a system then you reinforce that bias.