3 ms·
The problem is that the system isn’t falsifiable. If the model says someone won’t turn up to court they are locked in cells until their trial, how will you know
by dan-robertson 6y ago
The problem is that the system isn’t falsifiable. If the model says someone won’t turn up to court they are locked in cells until their trial, how will you know whether it was right or not? If you randomly release people as a test of the model then this is firstly unfair but secondarily won’t necessarily reveal small scale problems with the model that only affect certain groups of people because the sample will be small.
This also ignores the large impact that going to jail tends to have on people’s lives. I don’t think it is reasonable to simply dismiss the likelihood of bias.
- AnHonestComment 6y agoThere’s certainly parts that are hard to test, but it’s incorrect to say it’s unfalsifiable as a whole. For example, court attendance isn’t a Boolean: we create a confidence score and decide the “yes/no” question via some threshold. Even if you can’t test the model on the people you decide to keep in custody, you can test it on the people you release. The percentage who do appear for trial vs running should match your confidence score in that cohort — ie, if you guess they show up 95% of the time, then they should show up 95% of the time.
- darawk 6y agoFor any one instance? You can't. But in aggregate, we do it all the time: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_diver...