4 ms·

This is an interesting paper on a subject that has provoked debate on HN in the past. I would be interested to read criticism of the methods from those who have

by mrow84 9y ago

This is an interesting paper on a subject that has provoked debate on HN in the past. I would be interested to read criticism of the methods from those who have expressed support for these kinds of automated systems. The most obvious criticism is that the uninspiring performance of the COMPAS model does not provide any evidence of the weakness of automated systems in general, which is surely correct. This kind of argument would not, however, address more general concerns about how and why these kinds of systems are adopted - why is this system being used when its performance is not obviously competitive? (The answer to that is presumably "corruption", in a generalised sense, but that is hardly an adequate response.) Even more broadly, as the complexity of human society increases, it seems likely that the difficulty of making constructive interventions will also increase, and possibly more rapidly ("The Collapse of Complex Societies" by Joseph Tainter investigates this hypothesis). It does seem risky to just plough on, hoping that we can invent our way out of any problems that might arise as a result of what we are doing.

sinxoveretothex 9y ago
One question that would be interesting to explore is: what is the predictive score of judges/juries/etc? Is it more or less variable than the COMPAS score? In other words, do judges perform better than COMPAS? And slightly less important: is the variance lower? If it isn't, then there is no point arguing whether COMPAS is risky: it would be less so than the alternative. Although I suppose that the 'crowd of non-experts' is essentially what a jury is and they had about the same performance. At any rate, it sounds like much less of a hassle to input characteristics into COMPAS and get a probability than getting a jury together. Besides, I think I'd be much more worried that a jury could influence each other and bias its decision than about am algorithm making unfair predictions.
stult 9y ago
Well hold on. COMPAS performs as well as people but costs less than getting a jury together or paying a professional. So I wouldn't say that's uninspiring. It also performs as well as their simpler model. But they only evaluated the simple model on data from a single county. There's not enough evidence to determine whether that model would perform as well on other data sets. I would presume, perhaps wrongly, that more work has been done to ensure the generalizability of COMPAS, the results of which work militated against the adoption of a simpler algorithm. Possibly because the algorithm has to avoid racial disparities that might creep up in counties with a different demographic mix. Even if the simpler model performs equally well across many jurisdictions, that merely suggests that a simpler non-proprietary algorithm would be better. But we're still choosing between algorithms, not discrediting the idea of algorithmic sentencing altogether.
- mrow84 9y ago
  As long as you are vaguely positivist then we are surely only "choosing between algorithms" - the question is rather how those choices are made. When we consider computerised solutions we are nearly always forced to radically simplify our frame, dispensing with "judgment". This can be an advantage, because it reduces the scope for individual bias, but it can also be a disadvantage, because it reduces the scope for mitigating systemic biases. It is much more difficult to design a perfect system than it is to apply corrections to a merely good one - capturing what those corrections are, however, is very difficult. As an aside, do you have a source for the cost comparison, because I didn't see anything in the paper?
  - stult 9y ago
    In the interests of disclosure and not as an assertion of authority: I must admit that I am a lawyer and my perspective is shaped by my dissatisfaction with the profession's lack of scientific rigor both in the academic and practical spheres. And as an initial matter, I would point out that there is no judicial procedure which relies on algorithmic sentencing or bail decisions without a judge reviewing the decision. COMPAS is merely used to produce a recommendation for a bail amount which a judge then has to review and approve. Defendants are still afforded a hearing where they can object and raise any extenuating circumstances. So to the extent that judicial discretion can address systemic biases, it already does. And I certainly recognize the value of judicial discretion. Mandatory minimums alone demonstrate how prioritizing the punitive and retributive goals of the judicial system over simple human mercy can amplify the negative effects of racial and other social biases. However, my experience has been that the legal field has a strong systemic bias against empirical or statistical techniques. Essentially anything that involves math. There's a running, tired joke at every law school that students pursue JDs because they couldn't get a good score on the math GREs (the LSATs in contrast involve no math). And to clarify since I am at risk of confusing what we mean by the term bias: I am not denying that the legal profession is susceptible to the social biases you mention, but rather I would argue those biases are impossible to address until the deeper, methodological biases in how the legal profession pursues objective truth are addressed. Algorithmic sentencing and bail setting software can assist there in two ways. First, it ensures the collection of extensive, objective, and standardized data that can be used to evaluate the potential systemic biases you are concerned about. A single, consolidated, structured database with enormous amounts of information about each individual defendant is a gold mine for researchers. Simply put, we cannot even begin to resolve our biases without first identifying them. Software like COMPAS can help (though I'm not sure if it does in this case because I don't think they provide public access to the data). Second, the introduction of this software gives lawyers a chance to get used to machine learning algorithms and other advanced statistical tools. They're powerful techniques and need to become a larger part of the standard lawyer's tool kit. I don't think COMPAS is the terminal destination for sentencing software. In fact, I really hope otherwise. But, as a profession, I think we need to start taking steps to modernize our practices, one little step at a time. Each individual step may be imperfect at first, but so long as we maintain judicial discretion and careful oversight, these tools will be a boon for American justice. And in response to your last question, I don't have a citation on hand. I've read it somewhere before. Basically they replace a professional clerk or low level prosecutor in the DA's office who prepares a bunch of data and biographical information for the bail recommendation to the judge. It ends up being about one FTE a year and the software costs less than that to license.
fvdessen 9y ago
The COMPAS system seems to have the same or better performance than the others methods they compared it to (including humans), so it seems to work pretty well. I find it weird that an algorithm doing better or same than humans is called 'uninspiring'. Maybe people have too high expectations of what AI can do ?
- mrow84 9y ago
  The discussion in the paper describes COMPAS as "no more accurate or fair than the predictions of people with little to no criminal justice expertise who responded to an online survey", an assessment that doesn't seem unreasonable given the results presented. So, COMPAS is uninspiring because it provides no demonstrable benefits over even a crude analog of existing methods. Furthermore, it is not competitive against a two-feature linear regression model which only considers age and the number of previous convictions. As it happens, however, I agree with you - people do have unrealistic expectations of what can be achieved with AI. Replacing humans is, in many domains, a more difficult prospect than might be imagined, because so much of what we do is implicit. Suggesting that a particular problem is "solved" because a system achieves non-significant parity on a handful of measures is short-sighted to say the least.

by mrow84 9y ago

sinxoveretothex 9y ago

One question that would be interesting to explore is: what is the predictive score of judges/juries/etc? Is it more or less variable than the COMPAS score? In other words, do judges perform better than COMPAS? And slightly less important: is the variance lower? If it isn't, then there is no point arguing whether COMPAS is risky: it would be less so than the alternative. Although I suppose that the 'crowd of non-experts' is essentially what a jury is and they had about the same performance. At any rate, it sounds like much less of a hassle to input characteristics into COMPAS and get a probability than getting a jury together. Besides, I think I'd be much more worried that a jury could influence each other and bias its decision than about am algorithm making unfair predictions.

stult 9y ago

Well hold on. COMPAS performs as well as people but costs less than getting a jury together or paying a professional. So I wouldn't say that's uninspiring. It also performs as well as their simpler model. But they only evaluated the simple model on data from a single county. There's not enough evidence to determine whether that model would perform as well on other data sets. I would presume, perhaps wrongly, that more work has been done to ensure the generalizability of COMPAS, the results of which work militated against the adoption of a simpler algorithm. Possibly because the algorithm has to avoid racial disparities that might creep up in counties with a different demographic mix. Even if the simpler model performs equally well across many jurisdictions, that merely suggests that a simpler non-proprietary algorithm would be better. But we're still choosing between algorithms, not discrediting the idea of algorithmic sentencing altogether.

mrow84 9y ago
As long as you are vaguely positivist then we are surely only "choosing between algorithms" - the question is rather how those choices are made. When we consider computerised solutions we are nearly always forced to radically simplify our frame, dispensing with "judgment". This can be an advantage, because it reduces the scope for individual bias, but it can also be a disadvantage, because it reduces the scope for mitigating systemic biases. It is much more difficult to design a perfect system than it is to apply corrections to a merely good one - capturing what those corrections are, however, is very difficult. As an aside, do you have a source for the cost comparison, because I didn't see anything in the paper?
- stult 9y ago
  In the interests of disclosure and not as an assertion of authority: I must admit that I am a lawyer and my perspective is shaped by my dissatisfaction with the profession's lack of scientific rigor both in the academic and practical spheres. And as an initial matter, I would point out that there is no judicial procedure which relies on algorithmic sentencing or bail decisions without a judge reviewing the decision. COMPAS is merely used to produce a recommendation for a bail amount which a judge then has to review and approve. Defendants are still afforded a hearing where they can object and raise any extenuating circumstances. So to the extent that judicial discretion can address systemic biases, it already does. And I certainly recognize the value of judicial discretion. Mandatory minimums alone demonstrate how prioritizing the punitive and retributive goals of the judicial system over simple human mercy can amplify the negative effects of racial and other social biases. However, my experience has been that the legal field has a strong systemic bias against empirical or statistical techniques. Essentially anything that involves math. There's a running, tired joke at every law school that students pursue JDs because they couldn't get a good score on the math GREs (the LSATs in contrast involve no math). And to clarify since I am at risk of confusing what we mean by the term bias: I am not denying that the legal profession is susceptible to the social biases you mention, but rather I would argue those biases are impossible to address until the deeper, methodological biases in how the legal profession pursues objective truth are addressed. Algorithmic sentencing and bail setting software can assist there in two ways. First, it ensures the collection of extensive, objective, and standardized data that can be used to evaluate the potential systemic biases you are concerned about. A single, consolidated, structured database with enormous amounts of information about each individual defendant is a gold mine for researchers. Simply put, we cannot even begin to resolve our biases without first identifying them. Software like COMPAS can help (though I'm not sure if it does in this case because I don't think they provide public access to the data). Second, the introduction of this software gives lawyers a chance to get used to machine learning algorithms and other advanced statistical tools. They're powerful techniques and need to become a larger part of the standard lawyer's tool kit. I don't think COMPAS is the terminal destination for sentencing software. In fact, I really hope otherwise. But, as a profession, I think we need to start taking steps to modernize our practices, one little step at a time. Each individual step may be imperfect at first, but so long as we maintain judicial discretion and careful oversight, these tools will be a boon for American justice. And in response to your last question, I don't have a citation on hand. I've read it somewhere before. Basically they replace a professional clerk or low level prosecutor in the DA's office who prepares a bunch of data and biographical information for the bail recommendation to the judge. It ends up being about one FTE a year and the software costs less than that to license.

fvdessen 9y ago

The COMPAS system seems to have the same or better performance than the others methods they compared it to (including humans), so it seems to work pretty well. I find it weird that an algorithm doing better or same than humans is called 'uninspiring'. Maybe people have too high expectations of what AI can do ?

mrow84 9y ago
The discussion in the paper describes COMPAS as "no more accurate or fair than the predictions of people with little to no criminal justice expertise who responded to an online survey", an assessment that doesn't seem unreasonable given the results presented. So, COMPAS is uninspiring because it provides no demonstrable benefits over even a crude analog of existing methods. Furthermore, it is not competitive against a two-feature linear regression model which only considers age and the number of previous convictions. As it happens, however, I agree with you - people do have unrealistic expectations of what can be achieved with AI. Replacing humans is, in many domains, a more difficult prospect than might be imagined, because so much of what we do is implicit. Suggesting that a particular problem is "solved" because a system achieves non-significant parity on a handful of measures is short-sighted to say the least.