2 ms·
Well no shit, and it is a problem that people don't know this. In the research world we are still working on how to get broad optimizations and are treating pr
by godelski 6y ago
Well no shit, and it is a problem that people don't know this.
In the research world we are still working on how to get broad optimizations and are treating problems like they are short tailed. Real world (especially in medicine, which is what Ng works on a lot) datasets are long tailed. You look at common datasets like CelebA and you'll find a ton of light skinned people and other attributes that are limited (picking because obvious, but most datasets have these attribute problems and bias doesn't necessarily have to do with race or even human attributes. It can be words or even just types of events. But race is easy to understand and easier to talk about). For the most part we don't really care about that in the research world because we're still trying to optimize performance on these small world datasets (there are people working on long tail problems. This work is getting more popular in generative face models because we've gotten good at the restricted problem: see StyleGAN2-ADA). But there seems to be plenty of people that think they can just apply a CelebA pretrained model and use it for production in the real world. The data is already bad to begin with. And since we haven't tested on better datasets who knows if our model itself is going to exaggerate those biases or not, even if we did have better data. (There's many places bias can appear in the pipeline)
The problem is that in the ML community we're just at the stage where we're bad at performance and so limiting how complex our environment is (e.g. if you operate on mostly white people your world is less complex that has a more diversified spectrum of skin tones). We're getting there though. That's what this talk about algorithmic bias is often about. But if you just take a model from a small world you shouldn't expect similar results in anything but the exact same small world that the researchers tested on.
TLDR: research is small world, production is large world. Don't expect a small world model to work as effectively in the large world.