If that second scenario were to happen, then I think we should take a serious look at why that correlation is occurring rather than just throwing out the data b

If that second scenario were to happen, then I think we should take a serious look at why that correlation is occurring rather than just throwing out the data because it's "racist". That we removed the classification and then it was re-discovered by other correlations really should suggest something. On the assumption that it wasn't engineered to be biased and was naturally arrived at by the algorithm itself, then that actually seems like an important data point, and could even be a nice litmus test of how we're addressing racial differences if the models evolve to be more positive over time.