3 ms·

That's very plausible. But then that means there is an inherent bias against people with darker skin. So the more interesting question is: why wasn't that bias

by jxf 3y ago

That's very plausible. But then that means there is an inherent bias against people with darker skin. So the more interesting question is: why wasn't that bias anticipated and caught, and if it was known, why was it allowed to go to production where it affected the lives and liberty of one group of people disproportionately?

jurimasa 3y ago
That's easy: racism.
vuln 3y ago
Because the people selling the software reported the false positive rate a x per y but just accidentally forget to report that ALL of the false positives are of one single group. I would love to be in a sales meeting with a company that develops this type of software and the police that buy it.
mapt 3y ago
Because nobody cared. No consequences, profit-driven company, metric-driven agencies, nobody cared. Constructing a modern-day criminal justice system from the ashes of the Bad Old Days is going to be a project for our generation or the next or the next after that, but it probably requires burning things to the ground. In the meantime: Most of the liberal democratic ideals you and I were raised on with regard to criminal justice were from academia musing on what should be, or from copaganda television shows lying about what is. They live in our heads - if you want them to be real, you have to make them real.
- 6D794163636F756 3y ago
  There's also the chance that they didn't care to build their training set to match the expected population. Another industry dealing with this is automotives. Until recently tests were done exclusively on male crash test dummies. This resulted in higher mortality rates in women as things like seatbelts and airbags were tested on models not reflecting their bodies. This would all be because either they didn't care or they didn't realize they should care. Which is to say, I agree with you and the problem you touched on is far more pervasive and effects more people than just the legal system and we need to start doing a better job addressing it.
  - Tozen 3y ago
    Good example.

by jxf 3y ago

jurimasa 3y ago

That's easy: racism.

vuln 3y ago

Because the people selling the software reported the false positive rate a x per y but just accidentally forget to report that ALL of the false positives are of one single group. I would love to be in a sales meeting with a company that develops this type of software and the police that buy it.

mapt 3y ago

Because nobody cared. No consequences, profit-driven company, metric-driven agencies, nobody cared. Constructing a modern-day criminal justice system from the ashes of the Bad Old Days is going to be a project for our generation or the next or the next after that, but it probably requires burning things to the ground. In the meantime: Most of the liberal democratic ideals you and I were raised on with regard to criminal justice were from academia musing on what should be, or from copaganda television shows lying about what is. They live in our heads - if you want them to be real, you have to make them real.

6D794163636F756 3y ago
There's also the chance that they didn't care to build their training set to match the expected population. Another industry dealing with this is automotives. Until recently tests were done exclusively on male crash test dummies. This resulted in higher mortality rates in women as things like seatbelts and airbags were tested on models not reflecting their bodies. This would all be because either they didn't care or they didn't realize they should care. Which is to say, I agree with you and the problem you touched on is far more pervasive and effects more people than just the legal system and we need to start doing a better job addressing it.
- Tozen 3y ago
  Good example.