4 ms·
I am sorry to hear that statistics are not politically correct. For example, assume that the population is roughly 50% white, 25% hispanic, and 25% black. If y
by tasubotadas 7y ago
I am sorry to hear that statistics are not politically correct. For example, assume that the population is roughly 50% white, 25% hispanic, and 25% black.
If you sample this distribution without adding any bias, you will get a sample distribution that will have roughly the same distribution as the original population.
Now if you do machine learning on that, you will obviously give the algorithm twice as many of white faces to learn from as compared to the other two, and the error rate will be lower for the white faces.
Since people are upset that the error rate is lower (let's say 2% vs 2.6% - a 30% higher error rate), the solution here would seem to be to increase the error rate for whites by dropping their samples, as you can't really conjure data for the other two out of nowhere. So basically you have to make your system worse, so some people could be happy. Not nice.
Obviously, you could pay extra to acquire those samples, but given the way the current data collection systems work (data is just collected en-masse from the data that was uploaded by the people), you can't really expect companies to double their machine learning budgets (or whatever it's gonna cost them to collect that data) to justify this.
Also, it could be that the author of the article has some secret obvious solution to this, but they are just not sharing that with the rest of us :).
- deleted 7y ago[deleted]
- AstralStorm 7y agoSparse and reweighed learning. (With binned data.) These algorithms are probably slightly worse but they're useful for inference when number of samples is very small. If you apply them to dense data, the results are actually not bad. Not that much worse, either, and much stronger at generalization.