3 ms·
That would be the obvious first step and an absolute minimal requirement. The problem is that as these systems get more black-box-y there is little to stop them
by matthewmacleod 7y ago
That would be the obvious first step and an absolute minimal requirement. The problem is that as these systems get more black-box-y there is little to stop them making predictions that are affected by inferred hidden characteristics. Like, “we don’t take race or ethnicity into account, but our insurance algorithm says that if you have black hair and brown eyes and a name commonly found in black communities and you live in an area with a large black population then you are more of a risk sorry but it’s nothing to do with race honest”.
- MBCook 7y agoThe chances there is a line in the algorithm that says ‘women get less’ is basically 0%. It’s got to be inferences like you said. And those are so much harder to spot.
- pacala 7y agoTo the extent there exists a correlation between some characteristic and the outcome, models that exhibit the correlation will be more accurate than models that don't, irrespective whether the characteristic is used as a model feature or not. In the context of risk assessment, "more accurate" means "more likely to survive in the market place".