3 ms·
I'm not following why cross validation helps here. One group (sunglasses) is simply much rarer so my algorithm may globally perform better on the overall set by
by usaar333 5y ago
I'm not following why cross validation helps here. One group (sunglasses) is simply much rarer so my algorithm may globally perform better on the overall set by accepting lower recall on the rarer subgroup (allowing for higher precision on the much larger group).
This isn't a hypothetical example; many years back I improved the F-score of a face detector I was working on by using eye detection and skin color detection in the classification pipeline. I assume automated systems could produce similar effects, tanking recall on a small subgroup to buy better precision on the larger one.
- ekianjo 5y ago> One group (sunglasses) is simply much rarer so my algorithm may globally perform better on the overall set by accepting lower recall on the rarer subgroup (allowing for higher precision on the much larger group). I see what you mean. Well then my answer is still the same. If you have an unbalanced input data, this is what is causing the bias at the classifier level - not the classifier model/algorithm in itself.
- usaar333 5y agoI'm still confused on the semantics here. My data in this case is broadly representative of the real world input - so what's wrong? This is where I'm confused on the general idea of "algorithmic bias". Any maximally optimized classifier over the population may perform worse on any minority group (broadly defined). Representative data is necessary to achieve highest efficiency (as otherwise you will underperform in the real world), but it doesn't solve the problem of say relatively reduced recall of a minority group (which is often discussed in these posts). In fact, given that groupings are arbitrary, I don't think this is a solvable problem.