4 ms·

The algorithms being only as good as we are has many levels to it. Encoding existing biases is one. But nowadays it’s not existing algorithms but rather data dr

The algorithms being only as good as we are has many levels to it. Encoding existing biases is one. But nowadays it’s not existing algorithms but rather data driving the decisions. So the biases in how data might be collected or processed is at least the second and much more difficult to discern layer. Finally, what if we find the data to be correct, the AI cost function impeccable and still find a bias? I.e. what if we find there’s a bias in the larger system of society? At some point the entire exercise becomes futile, as the passionless machine cannot be made to fit perfectly into an imperfect society where we’d like to make decisions not because they make sense but because we feel them to be right. Let’s keep the machines where they belong, shall we?

andreskytt 5y ago
Also, come to think of it, a machine cannot (at least for now) be taught about detecting and avoiding feedback loops. Data might suggest an ethnicity A to be more likely to be criminal which leads them to be filtered out for spot checks more frequently which enforces the algorithm.
- e1ghtSpace 5y ago
  I can't see how this would cause a feedback loop. You can still check the data to see what percentage of each ethnicity has been checked and found to be breaking the law (compared to checked and not breaking the law) and use that as a guide on which one is more criminal. Of course, you shouldn't be able to set an ethnicity to zero otherwise they would never be checked again.
  - Nasrudith 5y ago
    To give a trivial but excessively stupid example of a feedback loop imagine a delusion "Redheads are murderous bandits - kill them on sight!". Under that barbaric policy held as dogma the only redheads likely to survive would either avoid them entirely (and not be factored into the data) or in turn start killing in the crazed non-redheads in self defense, and being vastly outresourced looting as a matter of survival. The idiotic prophecy turned self-fuffilling as reflected by the data. The entire premise of checking by ethnicity is also incredibly dumb - as an enforcer you shouldn't care what they look like you care if they are breaking the fucking law or not! If you lack any causology in your examinations what you have is automated superstition.
- naresh_xai 5y ago
  I think it can be done. Just maintain the data not in a standard tabular format but a time based graph instead. And put constraints to not look back. Constrained deep learning is a thing too.
  - mjburgess 5y ago
    The issue is that the machine has no causal model of how the predictions are leading to the data. You could, as you say, try to single-out some variables and rig how they're processed. Here I dont think that helps: so long as you predict area A has more crime, area A is more policed and thus always appears to have more crime. The issue, in my view, is not the data nor the algorithm -- no modification to either can fix the issue. The issue is the machine isn't embedded in the world, and esp. has no ability to acquire a rich understanding of the human social environment. A fundamental aspect of understanding X is knowing what is irrelevant to X, ie., what "data" it is permissible/essential to ignore. In the case of crime, one ought ignore data from over-policing -- but this is not an effect which is present in the data itself.
KarlKemp 5y ago
It is entirely obvious that biased results are the results of real-world biases, at least in most cases. That explains them, but it doesn't excuse very much. Reproducing, for example, incorrect hiring or credit decisions is wrong independent of the causality, and harms both those individuals and society (yes, banks lose money when they turn down perfectly good customers for irrelevant reasons) One misunderstanding in this debate is that AI researchers get very defensive and retreat to, basically, "I'm not racist". What they miss was that (in most cases) they were never accused of anything like that. Everybody understands how these biases arise from the data, and are sometimes amplified by it. What they can sometimes be accused of is not checking for the most obvious flaws in their models before publishing them. At this point, the problem is well-known and there's simply no excuse for not checking. And if that means your model cannot be released because you don't have the data, that's too bad, but your work just isn't up to purpose. Go find better data. We don't let people sell rotten steaks just because the stores are out of the fresh variety, and there's no reason to treat AI differently.
tchalla 5y ago
> But nowadays it’s not existing algorithms but rather data driving the decisions. There are different flavours of Machine Learning algorithms. You are referring to one of those where the learned representations are created from raw data. There are other flavours where algorithm designers manually input the important aspects of the representation. For example, a popular credit scoring agency where I live - uses the postcode as an input. The postcode in some cities is directly correlated to race and ethnicity. The broader point is that there exists various aspects that drive bias and that goes beyond data.

andreskytt 5y ago

Also, come to think of it, a machine cannot (at least for now) be taught about detecting and avoiding feedback loops. Data might suggest an ethnicity A to be more likely to be criminal which leads them to be filtered out for spot checks more frequently which enforces the algorithm.

e1ghtSpace 5y ago
I can't see how this would cause a feedback loop. You can still check the data to see what percentage of each ethnicity has been checked and found to be breaking the law (compared to checked and not breaking the law) and use that as a guide on which one is more criminal. Of course, you shouldn't be able to set an ethnicity to zero otherwise they would never be checked again.
- Nasrudith 5y ago
  To give a trivial but excessively stupid example of a feedback loop imagine a delusion "Redheads are murderous bandits - kill them on sight!". Under that barbaric policy held as dogma the only redheads likely to survive would either avoid them entirely (and not be factored into the data) or in turn start killing in the crazed non-redheads in self defense, and being vastly outresourced looting as a matter of survival. The idiotic prophecy turned self-fuffilling as reflected by the data. The entire premise of checking by ethnicity is also incredibly dumb - as an enforcer you shouldn't care what they look like you care if they are breaking the fucking law or not! If you lack any causology in your examinations what you have is automated superstition.
naresh_xai 5y ago
I think it can be done. Just maintain the data not in a standard tabular format but a time based graph instead. And put constraints to not look back. Constrained deep learning is a thing too.
- mjburgess 5y ago
  The issue is that the machine has no causal model of how the predictions are leading to the data. You could, as you say, try to single-out some variables and rig how they're processed. Here I dont think that helps: so long as you predict area A has more crime, area A is more policed and thus always appears to have more crime. The issue, in my view, is not the data nor the algorithm -- no modification to either can fix the issue. The issue is the machine isn't embedded in the world, and esp. has no ability to acquire a rich understanding of the human social environment. A fundamental aspect of understanding X is knowing what is irrelevant to X, ie., what "data" it is permissible/essential to ignore. In the case of crime, one ought ignore data from over-policing -- but this is not an effect which is present in the data itself.

KarlKemp 5y ago

It is entirely obvious that biased results are the results of real-world biases, at least in most cases. That explains them, but it doesn't excuse very much. Reproducing, for example, incorrect hiring or credit decisions is wrong independent of the causality, and harms both those individuals and society (yes, banks lose money when they turn down perfectly good customers for irrelevant reasons) One misunderstanding in this debate is that AI researchers get very defensive and retreat to, basically, "I'm not racist". What they miss was that (in most cases) they were never accused of anything like that. Everybody understands how these biases arise from the data, and are sometimes amplified by it. What they can sometimes be accused of is not checking for the most obvious flaws in their models before publishing them. At this point, the problem is well-known and there's simply no excuse for not checking. And if that means your model cannot be released because you don't have the data, that's too bad, but your work just isn't up to purpose. Go find better data. We don't let people sell rotten steaks just because the stores are out of the fresh variety, and there's no reason to treat AI differently.

tchalla 5y ago

> But nowadays it’s not existing algorithms but rather data driving the decisions. There are different flavours of Machine Learning algorithms. You are referring to one of those where the learned representations are created from raw data. There are other flavours where algorithm designers manually input the important aspects of the representation. For example, a popular credit scoring agency where I live - uses the postcode as an input. The postcode in some cities is directly correlated to race and ethnicity. The broader point is that there exists various aspects that drive bias and that goes beyond data.