3 ms·

If we're using that example of inadvertent discrimination, I think it's certainly feasible to have non-political discussion around improving it. What you descr

If we're using that example of inadvertent discrimination, I think it's certainly feasible to have non-political discussion around improving it. What you described sounds like a product flaw. Customers/users won't want to use a product that delivers sub-optimal results. Any internal employee saying "hey, we have an error rate of X% for this Y segment, and they represent Z% of the user base" isn't engaging in controversial political discussion (in my opinion). However, if an individual chose to describe this flaw in more loaded language, it could easily turn political and combative for the team.

evan_ 5y ago
> What you described sounds like a product flaw. It would never have shipped with a product flaw that frequently caused it to classify white people as something insulting.
- FFRefresh 5y ago
  'Never' is a strong word and doing a lot of work here, and I don't think the counterfactual can be proved one way or another. I do think your comment here, with an unprovable statement about an immutable characteristic, stated with absolute certainty, would invite toxic political discussions. Speaking with humility, and honestly trying to improve processes to yield better results, is the type of communication that I advocate for on teams I'm involved with.
  - jjeaff 5y ago
    There is speaking with humility, then there is just being downright, purposefully naive. Of course they never would have shipped it if it didn't recognize white faces. And the big isn't in the software, the bug is that the companies producing this stuff don't have a single solitary person of color either working on the product or testing it that would have certainly noticed that it doesn't work on them or said they were an animal, etc.
scarface74 5y ago
If the “customers” were law enforcement - who already racially profile on the flimsiest of excuses - that might be seen as a feature not a bug. If AI/ML says someone “fits the description” what better feature than being able to blame it on the computer? If law enforcement was willing to deal with the flaw, and if tech companies were willing to sell it to them, what’s to stop an unscrupulous company from continuing to sell it if they didn’t get push back?

evan_ 5y ago

> What you described sounds like a product flaw. It would never have shipped with a product flaw that frequently caused it to classify white people as something insulting.

FFRefresh 5y ago
'Never' is a strong word and doing a lot of work here, and I don't think the counterfactual can be proved one way or another. I do think your comment here, with an unprovable statement about an immutable characteristic, stated with absolute certainty, would invite toxic political discussions. Speaking with humility, and honestly trying to improve processes to yield better results, is the type of communication that I advocate for on teams I'm involved with.
- jjeaff 5y ago
  There is speaking with humility, then there is just being downright, purposefully naive. Of course they never would have shipped it if it didn't recognize white faces. And the big isn't in the software, the bug is that the companies producing this stuff don't have a single solitary person of color either working on the product or testing it that would have certainly noticed that it doesn't work on them or said they were an animal, etc.

scarface74 5y ago

If the “customers” were law enforcement - who already racially profile on the flimsiest of excuses - that might be seen as a feature not a bug. If AI/ML says someone “fits the description” what better feature than being able to blame it on the computer? If law enforcement was willing to deal with the flaw, and if tech companies were willing to sell it to them, what’s to stop an unscrupulous company from continuing to sell it if they didn’t get push back?