consider danah's example in which external studies show that blacks are convicted for drug crimes at a rate disproportionate to the actual crime incidence rate.

consider danah's example in which external studies show that blacks are convicted for drug crimes at a rate disproportionate to the actual crime incidence rate. the fact that the easily-available training data (court stats) is unrepresentative of the underlying process (crime incidence) is a problem that has to be corrected or predictive output will also over-represent them. the point is that this prior or correction is also part of the "data available" but you have to actually know something about the problem domain to know about it; you can't just grab a court data dump and pour it into a model and say, "welp, that was the data available to me, it's not biased."