3 ms·

No. It’s easy to fix bias in the data for a model developer, but one can’t control other people’s dietary choices.

atq2119 6y ago
If it's so easy to fix the bias, then why isn't it fixed, always? That should be the expectation. More likely, the reason that bias isn't routinely fixed is that it isn't easy, and these kinds of biases do make it into production systems. Which makes it a net positive in my view that the occasional shitstorm reminds society of this fact. Can that be unpleasant for AI researchers? Sure... but if it bothers then, then perhaps they could focus their research on trying to fix the problem? Physicists had and have unpleasant conversations about their moral responsibility for nuclear weapons. Other fields of research should take their moral responsibilities seriously as well (not just AI research, by the way).
kennywinker 6y ago
But can you control other model developers dataset choices? ML tools are developed, and filter down into the general developer population where they are used without fully comprehending the biases they contain or can contain if used incorrectly. I'm pretty partial to the idea that ML is a systemic bias footgun. VERY hard to not use incorrectly, and with potentially HUGE social repercussions. E.g. the youtube funnel towards extremism that was well documented a few years ago - visitors start on some innocuous video, and get recommended more and more extreme things, funnelling traffic towards the alt-right in an unhealthy way.
- TrackerFF 6y ago
  It IS the responsibility of a data engineer / scientist to process the data, but this leads to a questions - to what lengths should one go, to ensure that the dataset is balanced? For example - in the small town I'm from, there are practically no people of color. It's an extremely homogeneous town, in terms of ethnicity. Everyone's white - the only thing that seems to differentiate people, are their socioeconomic backgrounds. So in a way, ethnicity becomes a irrelevant feature, if we were to use it to say, profile and predict criminals. And what's more - if we were to transfer that model, which is trained on data from my small homogeneous place, it would probably generalize very poorly in areas with more diversity. On the flip side - if we were to make a dataset out of criminals in some large and diverse area, we'd need to get our sampling methodology right. Maybe it just so happens that the law enforcement tends to pour all their resources in policing poor areas, where some certain ethnicity is very over represented? As you can see, the further down we go, the more this problem moves from models -> datasets -> sampling -> policies, and so on. And as for our imagined crime profiling model, you can see that the scientists are oftentimes forced to work with wildly different datasets, that may look very different from one another. You get a bunch of different models with great local optimization, but which fail to generalize on the population. And what's more, those locally optimized models may work perfectly well for their intended tasks, so there will be no complaints - until one of them are deployed to cover more general cases. Yes - the ML engineers and scientists are responsible for building good generalized models. Yes - the data engineers and scientists are responsible for building good datasets But alas, there's only so much one can do about the above. In the end, the data actually stems from something real and tangible, and if there's a systemic bias in how that data is created, then that's going to be the greatest factor for everything which comes after. It's a very deep and complex topic, which goes way beyond this. But sadly there are real-life consequences, when models are being deployed in the real world.

If it's so easy to fix the bias, then why isn't it fixed, always? That should be the expectation. More likely, the reason that bias isn't routinely fixed is that it isn't easy, and these kinds of biases do make it into production systems. Which makes it a net positive in my view that the occasional shitstorm reminds society of this fact. Can that be unpleasant for AI researchers? Sure... but if it bothers then, then perhaps they could focus their research on trying to fix the problem? Physicists had and have unpleasant conversations about their moral responsibility for nuclear weapons. Other fields of research should take their moral responsibilities seriously as well (not just AI research, by the way).

kennywinker 6y ago

But can you control other model developers dataset choices? ML tools are developed, and filter down into the general developer population where they are used without fully comprehending the biases they contain or can contain if used incorrectly. I'm pretty partial to the idea that ML is a systemic bias footgun. VERY hard to not use incorrectly, and with potentially HUGE social repercussions. E.g. the youtube funnel towards extremism that was well documented a few years ago - visitors start on some innocuous video, and get recommended more and more extreme things, funnelling traffic towards the alt-right in an unhealthy way.

TrackerFF 6y ago
It IS the responsibility of a data engineer / scientist to process the data, but this leads to a questions - to what lengths should one go, to ensure that the dataset is balanced? For example - in the small town I'm from, there are practically no people of color. It's an extremely homogeneous town, in terms of ethnicity. Everyone's white - the only thing that seems to differentiate people, are their socioeconomic backgrounds. So in a way, ethnicity becomes a irrelevant feature, if we were to use it to say, profile and predict criminals. And what's more - if we were to transfer that model, which is trained on data from my small homogeneous place, it would probably generalize very poorly in areas with more diversity. On the flip side - if we were to make a dataset out of criminals in some large and diverse area, we'd need to get our sampling methodology right. Maybe it just so happens that the law enforcement tends to pour all their resources in policing poor areas, where some certain ethnicity is very over represented? As you can see, the further down we go, the more this problem moves from models -> datasets -> sampling -> policies, and so on. And as for our imagined crime profiling model, you can see that the scientists are oftentimes forced to work with wildly different datasets, that may look very different from one another. You get a bunch of different models with great local optimization, but which fail to generalize on the population. And what's more, those locally optimized models may work perfectly well for their intended tasks, so there will be no complaints - until one of them are deployed to cover more general cases. Yes - the ML engineers and scientists are responsible for building good generalized models. Yes - the data engineers and scientists are responsible for building good datasets But alas, there's only so much one can do about the above. In the end, the data actually stems from something real and tangible, and if there's a systemic bias in how that data is created, then that's going to be the greatest factor for everything which comes after. It's a very deep and complex topic, which goes way beyond this. But sadly there are real-life consequences, when models are being deployed in the real world.