But it does mean that AI models trained on data might end up just perpetuating racial correlations even when you don't think it's aware of race. It's another ex

But it does mean that AI models trained on data might end up just perpetuating racial correlations even when you don't think it's aware of race. It's another example of why interpretability is important: the model might end up depending on correlations you don't mean it to.