3 ms·

Is there a way to prove that it is not caused by systemic racism? Having some unfalsifiable quality is pretty useless, and if it is the sort of thing that's nev

by sprayk 6y ago

Is there a way to prove that it is not caused by systemic racism? Having some unfalsifiable quality is pretty useless, and if it is the sort of thing that's never meant to end then that feels more like religion.

nerdponx 6y ago
It would depend on the details of the particular situation (was there a specific news story associated with the GP?), but you'd have to show that people with the skin color in question signed off on the product. It's one thing if you produce a new model and present it at a conference, and one of the conclusion of your paper is that "this model does great on light-skinned faces but performs poorly on dark-skinned faces, we need to make improvements in optics and low-light sensing to bring up the accuracy on dark-skinned people." That's not racist, that's doing science. It's another if you are a for-profit image recognition company, or if you otherwise deploy that model in production without so much as checking to see if your model generalized well to dark-skinned faces, or without attempting to add some post-hoc bias correction, etc. We are currently in a world where the latter happens quite often. It's not just a race thing; there all kinds of inadequate "AI" systems being pushed on people that don't really work as advertised, to the detriment of some group or another (sometimes to everyone). But when it becomes a race thing, it's especially distressing to a lot of people, because Europe and the USA have long and grim histories of racism that extend up to and including the present day.
- sprayk 6y ago
  > ... people with the skin color in question signed off on the product. > ... if you otherwise deploy that model in production without so much as checking to see if your model generalized well to dark-skinned faces, or without attempting to add some post-hoc bias correction, etc Neither of these would rule out systemic racism in either case, though, since, as you said, the problem is that "the outcome is biased against a certain group of people". Regardless of whether there were BIPOC people signing off or whether you had a perfect training set/etc, the only thing that matters is the outcome.
  - nerdponx 6y ago
    That's right. My example stated two very different outcomes. One outcome: an academic result is published. Other outcomes: something is put into production.
    - sprayk 6y ago
      so you agree that attributing an outcome to systemic racism is unfalsifiable?
      - nerdponx 6y ago
        I see what you're saying. Mind you, I'm not a sociologist, so anything I say here is going to be a suboptimal paraphrasing. Systemic racism is not a cause of something, but a description of a state of existence. Systemic racism is itself the existence of inequitable outcomes in the context of a society that was literally founded on overt racism. That something "is" systemic racism is falsifiable by demonstrating that one or both of those criteria is not met. So this particular outcome will always be definitionally systemic racism until the outcome is no longer inequitable, or the USA has moved meaningfully past its white supremacist origins to the point where individual cases of accidental bias are truly individual cases and not part of a pattern. Personally, I am okay with that. Ceteris non paribus sunt.
themolecularman 6y ago
Agreed, but I've found that when discussing these things logic isn't often taken into account anymore due to matters of sensitivity and bias. It's frustrating.

by sprayk 6y ago

nerdponx 6y ago

It would depend on the details of the particular situation (was there a specific news story associated with the GP?), but you'd have to show that people with the skin color in question signed off on the product. It's one thing if you produce a new model and present it at a conference, and one of the conclusion of your paper is that "this model does great on light-skinned faces but performs poorly on dark-skinned faces, we need to make improvements in optics and low-light sensing to bring up the accuracy on dark-skinned people." That's not racist, that's doing science. It's another if you are a for-profit image recognition company, or if you otherwise deploy that model in production without so much as checking to see if your model generalized well to dark-skinned faces, or without attempting to add some post-hoc bias correction, etc. We are currently in a world where the latter happens quite often. It's not just a race thing; there all kinds of inadequate "AI" systems being pushed on people that don't really work as advertised, to the detriment of some group or another (sometimes to everyone). But when it becomes a race thing, it's especially distressing to a lot of people, because Europe and the USA have long and grim histories of racism that extend up to and including the present day.

sprayk 6y ago
> ... people with the skin color in question signed off on the product. > ... if you otherwise deploy that model in production without so much as checking to see if your model generalized well to dark-skinned faces, or without attempting to add some post-hoc bias correction, etc Neither of these would rule out systemic racism in either case, though, since, as you said, the problem is that "the outcome is biased against a certain group of people". Regardless of whether there were BIPOC people signing off or whether you had a perfect training set/etc, the only thing that matters is the outcome.
- nerdponx 6y ago
  That's right. My example stated two very different outcomes. One outcome: an academic result is published. Other outcomes: something is put into production.
  - sprayk 6y ago
    so you agree that attributing an outcome to systemic racism is unfalsifiable?
    - nerdponx 6y ago
      I see what you're saying. Mind you, I'm not a sociologist, so anything I say here is going to be a suboptimal paraphrasing. Systemic racism is not a cause of something, but a description of a state of existence. Systemic racism is itself the existence of inequitable outcomes in the context of a society that was literally founded on overt racism. That something "is" systemic racism is falsifiable by demonstrating that one or both of those criteria is not met. So this particular outcome will always be definitionally systemic racism until the outcome is no longer inequitable, or the USA has moved meaningfully past its white supremacist origins to the point where individual cases of accidental bias are truly individual cases and not part of a pattern. Personally, I am okay with that. Ceteris non paribus sunt.

themolecularman 6y ago

Agreed, but I've found that when discussing these things logic isn't often taken into account anymore due to matters of sensitivity and bias. It's frustrating.