This is not very surprising - Machine Learning algorithms trained on biased datasets tend to pick up the hidden biases in the training data. It’s important that

This is not very surprising - Machine Learning algorithms trained on biased datasets tend to pick up the hidden biases in the training data. It’s important that we be transparent about the training data that we are using, and are looking for hidden biases in it, otherwise we are building biased systems. Fortunately, there are open source tools out there that help audit machine learning models for bias, such as Audit AI, released by pymetrics - https://github.com/pymetrics/audit-ai https://github.com/pymetrics/audit-ai