3 ms·

If you understood the scientific method, you would understand that you always need to account for bias. Just because something comes from data doesn't mean it's

by rspeer 8y ago

If you understood the scientific method, you would understand that you always need to account for bias. Just because something comes from data doesn't mean it's objective.

avinium 8y ago
Hold up - you also have to tread very carefully there. I read your blog post above about "racist" NLP models and I agree it makes sense to debias inputs for certain purposes. However, many people in this thread aren't talking about adjusting for statistical bias. They're really talking about adjusting for socio-political over/under-representation, which is getting onto pretty shaky ground. I'd hazard a guess that a model trained on Google News giving "more negative" sentiment for black names is a function of those names appearing more frequently in crime articles, matching the overrepresentation in crime rates. Likewise for Arab/Muslim names, which are presumably disproportionately present in articles on terrorism. Now, that's obviously statistical sampling bias if you're (somehow) modelling "names I should call my kid". If you're looking at "names more associated with crime", however, then we shouldn't be eliminating any racial imbalances simply because they make us feel queasy. That's intellectually dishonest, and does everybody a disservice by trying to sweep uncomfortable realities under the rug. Same goes for gender bias in word embeddings from classical fiction - that is (I assume) a very accurate depiction of social gender imbalance from that period of history. That may or may not be relevant to the question you're asking - but that doesn't make any bias inherently "wrong". I think the underlying message should be "models only reflect the data they're given and may need to be adjusted for bias depending on the question being asked", not "race and gender is always irrelevant and should always be normalized".

by rspeer 8y ago

If you understood the scientific method, you would understand that you always need to account for bias. Just because something comes from data doesn't mean it's objective.

avinium 8y ago

Hold up - you also have to tread very carefully there. I read your blog post above about "racist" NLP models and I agree it makes sense to debias inputs for certain purposes. However, many people in this thread aren't talking about adjusting for statistical bias. They're really talking about adjusting for socio-political over/under-representation, which is getting onto pretty shaky ground. I'd hazard a guess that a model trained on Google News giving "more negative" sentiment for black names is a function of those names appearing more frequently in crime articles, matching the overrepresentation in crime rates. Likewise for Arab/Muslim names, which are presumably disproportionately present in articles on terrorism. Now, that's obviously statistical sampling bias if you're (somehow) modelling "names I should call my kid". If you're looking at "names more associated with crime", however, then we shouldn't be eliminating any racial imbalances simply because they make us feel queasy. That's intellectually dishonest, and does everybody a disservice by trying to sweep uncomfortable realities under the rug. Same goes for gender bias in word embeddings from classical fiction - that is (I assume) a very accurate depiction of social gender imbalance from that period of history. That may or may not be relevant to the question you're asking - but that doesn't make any bias inherently "wrong". I think the underlying message should be "models only reflect the data they're given and may need to be adjusted for bias depending on the question being asked", not "race and gender is always irrelevant and should always be normalized".