3 ms·
Ya, the overloading of 'bias' is confusing. > The "algorithm" doesnt capture the semantics of the data, and so introduces moral bias regardless of any statisti
by darawk 5y ago
Ya, the overloading of 'bias' is confusing.
> The "algorithm" doesnt capture the semantics of the data, and so introduces moral bias regardless of any statistical bias within the dataset.
> Eg., consider training an NLP system on black rights literature. Negative words co-occur with racial terms -- the semantic association is one of "black people opposing hatred" NOT "black people COMEWITH hatred".
I agree that certain NLP models might learn an association like that, but that's a consequence of the pairing of model and data. The model itself does not encode a racial prejudice, it's simply poorly suited to the dataset in this context.
- mjburgess 5y ago"Poorly suited" is the heart of the issue. Nothing here is morally biased: neither the algorithm or the data. But when people say the "data is biased" they are suggesting that this is where the moral bias enters. Not at all. The moral bias IS entering at the level of the algorithm. In this sense the algorithm is morally baised. Of course it is really the human operator who selects this algorithm which causes the issue -- but by saying "the data is biased" we are obscuring this reality.
- random314 5y agoThat's a long winded way of stating that correlation coefficients are morally biased. It is kinda hilarious.
- mjburgess 5y agoThe moral bias enters when a person takes correlation coefficients to indicate meaning. It doesnt matter what value they have.
- wizzwizz4 5y agoWe're humans. We do that even when we're generating those coefficients in our own brains; it's irresponsible to teach a computer how to find lots of correlations, show the results to humans, then try to say "but we didn't mean to imply that the correlation was meaningful!".
- random314 5y ago> The moral bias IS entering at the level of the algorithm. In this sense the algorithm is morally baised. > The moral bias enters when a person takes correlation coefficients to indicate meaning. Changing your stance within 2 replies is not a good look.
- djoldman 5y ago> Nothing here is morally biased: neither the algorithm or the data. I think the issue here is the following progression: 1. Someone chooses and collects a subset of all data that exists. 2. They then choose an algorithm/generic-framework-of-a-model with the intent to apply it to the data chosen in step 1. 3. They then train the chosen algorithm from step 2, yielding a model. What @darawk is saying, I believe, is that the algorithm/generic-framework-of-a-model chosen in step 2 is not morally/ethically biased/racist/etc. A human may consider the outputted model from step 3 to be so, however. If the above is true, then the source of moral/ethical bias is the data chosen in step 1. What untrained model/algorithm is morally or ethically biased? I know of no model listed in scikit learn, tensorflow, scipy, etc. that has hardcoded any sense of anything about humans at all...
- seoaeu 5y agoI don't see how that follows. Surely you could select hyper-parameters that cause a model to perform differently across different races, ethnicities, etc. Like, shouldn't you be able to get a model to perform poorly for minority groups simply by specifying a model size that's too small to learn the full data set, with the knowledge that most training samples won't be describing those groups?
- djoldman 5y agoHrm, I think I understand what you're saying. Perhaps a better way to formulate it is: let's say we take a model and add a regularization term that penalizes/rewards some part of the data, which when trained makes for a morally/ethically biased model. I think you're exactly correct, the source of moral/ethical bias here is in the model. Good point.