It IS the responsibility of a data engineer / scientist to process the data, but this leads to a questions - to what lengths should one go, to ensure that the d

It IS the responsibility of a data engineer / scientist to process the data, but this leads to a questions - to what lengths should one go, to ensure that the dataset is balanced? For example - in the small town I'm from, there are practically no people of color. It's an extremely homogeneous town, in terms of ethnicity. Everyone's white - the only thing that seems to differentiate people, are their socioeconomic backgrounds. So in a way, ethnicity becomes a irrelevant feature, if we were to use it to say, profile and predict criminals. And what's more - if we were to transfer that model, which is trained on data from my small homogeneous place, it would probably generalize very poorly in areas with more diversity. On the flip side - if we were to make a dataset out of criminals in some large and diverse area, we'd need to get our sampling methodology right. Maybe it just so happens that the law enforcement tends to pour all their resources in policing poor areas, where some certain ethnicity is very over represented? As you can see, the further down we go, the more this problem moves from models -> datasets -> sampling -> policies, and so on. And as for our imagined crime profiling model, you can see that the scientists are oftentimes forced to work with wildly different datasets, that may look very different from one another. You get a bunch of different models with great local optimization, but which fail to generalize on the population. And what's more, those locally optimized models may work perfectly well for their intended tasks, so there will be no complaints - until one of them are deployed to cover more general cases. Yes - the ML engineers and scientists are responsible for building good generalized models. Yes - the data engineers and scientists are responsible for building good datasets But alas, there's only so much one can do about the above. In the end, the data actually stems from something real and tangible, and if there's a systemic bias in how that data is created, then that's going to be the greatest factor for everything which comes after. It's a very deep and complex topic, which goes way beyond this. But sadly there are real-life consequences, when models are being deployed in the real world.