3 ms·

Statistical algorithms like in machine learning can reproduce human racism that is fed to them. That does not mean that the creator or user of such algorithms i

Statistical algorithms like in machine learning can reproduce human racism that is fed to them. That does not mean that the creator or user of such algorithms is racist, but it is their responsibility.

whatshisface 5y ago
Like airplane accidents, algorithms accidents are products of whole organizations, which include engineers, but are not only engineers. I don't condone blaming individual engineers for incidents when there is an entire system of quality control and management surrounding them that is supposed to proactively prevent these kinds of things. Nonetheless, it is possible for the whole system to be negligent, and it's also possible for the whole system to, if not be racist (because organizations don't have sentiments), to act collectively in a way that aligns with what a racist would do, if they were an organization. I admit that it's a stretched analogy and borders on anthropomorphizing organizations, but at the same time it could be a useful conceptual tool. For a concrete example, imagine that a company was prioritizing investigation of moderator actions based on the average emotional reaction of twelve or so people on an accountability board, when presented with each case. If the accountability board has any biases that don't average to zero across the dozen, the diverse stream of injustices will get corrected more in one direction than another, and the result will be injustice even in the absence of any individual with bad intentions.
krona 5y ago
Point taken, but to be clear: bias doesn't imply racism, no matter how hard you might want it to.
- guerrilla 5y ago
  I don't understand what you mean. It reproduces the racism of the authors, no? It's just an extension of the producers. Like I guess you could say the hammer in the hand of a racist hitting you isn't itself racist but I think the point of people saying some tool is racist is to say that it is in some way designed to reproduce their own racism (even if completely accidentally and unintentionally.) But is something designed so like a hammer or is it something more specific?
  - whatshisface 5y ago
    Compare it to industrial accidents. If a lathe hurts a black person, that's not racism on the part of the machine (obviously) or on the part of the engineers who designed the lathe (obviously). However, if the machine shop company pays less attention, though through no conscious choice, to safety in their branch on MLK street, that's systemic racism on the part of the company.

whatshisface 5y ago

Like airplane accidents, algorithms accidents are products of whole organizations, which include engineers, but are not only engineers. I don't condone blaming individual engineers for incidents when there is an entire system of quality control and management surrounding them that is supposed to proactively prevent these kinds of things. Nonetheless, it is possible for the whole system to be negligent, and it's also possible for the whole system to, if not be racist (because organizations don't have sentiments), to act collectively in a way that aligns with what a racist would do, if they were an organization. I admit that it's a stretched analogy and borders on anthropomorphizing organizations, but at the same time it could be a useful conceptual tool. For a concrete example, imagine that a company was prioritizing investigation of moderator actions based on the average emotional reaction of twelve or so people on an accountability board, when presented with each case. If the accountability board has any biases that don't average to zero across the dozen, the diverse stream of injustices will get corrected more in one direction than another, and the result will be injustice even in the absence of any individual with bad intentions.

krona 5y ago

Point taken, but to be clear: bias doesn't imply racism, no matter how hard you might want it to.

guerrilla 5y ago
I don't understand what you mean. It reproduces the racism of the authors, no? It's just an extension of the producers. Like I guess you could say the hammer in the hand of a racist hitting you isn't itself racist but I think the point of people saying some tool is racist is to say that it is in some way designed to reproduce their own racism (even if completely accidentally and unintentionally.) But is something designed so like a hammer or is it something more specific?
- whatshisface 5y ago
  Compare it to industrial accidents. If a lathe hurts a black person, that's not racism on the part of the machine (obviously) or on the part of the engineers who designed the lathe (obviously). However, if the machine shop company pays less attention, though through no conscious choice, to safety in their branch on MLK street, that's systemic racism on the part of the company.