4 ms·

Also, come to think of it, a machine cannot (at least for now) be taught about detecting and avoiding feedback loops. Data might suggest an ethnicity A to be mo

Also, come to think of it, a machine cannot (at least for now) be taught about detecting and avoiding feedback loops. Data might suggest an ethnicity A to be more likely to be criminal which leads them to be filtered out for spot checks more frequently which enforces the algorithm.

e1ghtSpace 5y ago
I can't see how this would cause a feedback loop. You can still check the data to see what percentage of each ethnicity has been checked and found to be breaking the law (compared to checked and not breaking the law) and use that as a guide on which one is more criminal. Of course, you shouldn't be able to set an ethnicity to zero otherwise they would never be checked again.
- Nasrudith 5y ago
  To give a trivial but excessively stupid example of a feedback loop imagine a delusion "Redheads are murderous bandits - kill them on sight!". Under that barbaric policy held as dogma the only redheads likely to survive would either avoid them entirely (and not be factored into the data) or in turn start killing in the crazed non-redheads in self defense, and being vastly outresourced looting as a matter of survival. The idiotic prophecy turned self-fuffilling as reflected by the data. The entire premise of checking by ethnicity is also incredibly dumb - as an enforcer you shouldn't care what they look like you care if they are breaking the fucking law or not! If you lack any causology in your examinations what you have is automated superstition.
naresh_xai 5y ago
I think it can be done. Just maintain the data not in a standard tabular format but a time based graph instead. And put constraints to not look back. Constrained deep learning is a thing too.
- mjburgess 5y ago
  The issue is that the machine has no causal model of how the predictions are leading to the data. You could, as you say, try to single-out some variables and rig how they're processed. Here I dont think that helps: so long as you predict area A has more crime, area A is more policed and thus always appears to have more crime. The issue, in my view, is not the data nor the algorithm -- no modification to either can fix the issue. The issue is the machine isn't embedded in the world, and esp. has no ability to acquire a rich understanding of the human social environment. A fundamental aspect of understanding X is knowing what is irrelevant to X, ie., what "data" it is permissible/essential to ignore. In the case of crime, one ought ignore data from over-policing -- but this is not an effect which is present in the data itself.

e1ghtSpace 5y ago

I can't see how this would cause a feedback loop. You can still check the data to see what percentage of each ethnicity has been checked and found to be breaking the law (compared to checked and not breaking the law) and use that as a guide on which one is more criminal. Of course, you shouldn't be able to set an ethnicity to zero otherwise they would never be checked again.

Nasrudith 5y ago
To give a trivial but excessively stupid example of a feedback loop imagine a delusion "Redheads are murderous bandits - kill them on sight!". Under that barbaric policy held as dogma the only redheads likely to survive would either avoid them entirely (and not be factored into the data) or in turn start killing in the crazed non-redheads in self defense, and being vastly outresourced looting as a matter of survival. The idiotic prophecy turned self-fuffilling as reflected by the data. The entire premise of checking by ethnicity is also incredibly dumb - as an enforcer you shouldn't care what they look like you care if they are breaking the fucking law or not! If you lack any causology in your examinations what you have is automated superstition.

naresh_xai 5y ago

I think it can be done. Just maintain the data not in a standard tabular format but a time based graph instead. And put constraints to not look back. Constrained deep learning is a thing too.

mjburgess 5y ago
The issue is that the machine has no causal model of how the predictions are leading to the data. You could, as you say, try to single-out some variables and rig how they're processed. Here I dont think that helps: so long as you predict area A has more crime, area A is more policed and thus always appears to have more crime. The issue, in my view, is not the data nor the algorithm -- no modification to either can fix the issue. The issue is the machine isn't embedded in the world, and esp. has no ability to acquire a rich understanding of the human social environment. A fundamental aspect of understanding X is knowing what is irrelevant to X, ie., what "data" it is permissible/essential to ignore. In the case of crime, one ought ignore data from over-policing -- but this is not an effect which is present in the data itself.