3 ms·

It has been shown many times that if you know the human making the decisions, you can game it by knowing the person, or anything from having a preexisting relat

It has been shown many times that if you know the human making the decisions, you can game it by knowing the person, or anything from having a preexisting relationship with them, being famous, dependence (e.g. they're a tenant of you, or a landlord), planting ideas, ... Most of these are not illegal, or even possible to avoid. And then, of course, there's racism, nepotism and outright bribery. Humans, under the best of circumstances, are known to be unfair. Can an algorithm really be that much worse ?

notahacker 5y ago
You miss the point. We don't have a computer process that establishes factors such as the level of remorse expressed by the offender or their claimed motive, whether their misinterpretation of the relevant law is plausible or not, or the magnitude of the harm caused to the victim or whether the plaintiff's claim of monetary losses suffered is plausible or not so the "algorithm" is a thin veneer over a human determining which factors are and aren't relevant... so you get exactly the same level of bias before plus noise from the human judge making these determinations not knowing exactly how the algorithm works and a false claim of greater impartiality. It's probably even worse if you try to pull the human out the loop altogether and throw an ML black box at the testimony, which doesn't understand the actual situation the defendant describes at all but does pick up the statistical association between African American vernacular words used and higher sentences in its training corpus. Much easier to train a model that picks up the prejudice in sentencing factors than the situational nuance...
- stickfigure 5y ago
  I think the point is that we are currently using opaque, poorly trained and erratic ML (Meat Learning) algorithms. Oh sure the meatbox says that it's allocating N years for this, M years for that, but empirical studies have proven that this readout is nonsense. Which kind of ML is easier to train to be non-racist? I don't know, but it's not obvious to me that meat is the winner here.

notahacker 5y ago

You miss the point. We don't have a computer process that establishes factors such as the level of remorse expressed by the offender or their claimed motive, whether their misinterpretation of the relevant law is plausible or not, or the magnitude of the harm caused to the victim or whether the plaintiff's claim of monetary losses suffered is plausible or not so the "algorithm" is a thin veneer over a human determining which factors are and aren't relevant... so you get exactly the same level of bias before plus noise from the human judge making these determinations not knowing exactly how the algorithm works and a false claim of greater impartiality. It's probably even worse if you try to pull the human out the loop altogether and throw an ML black box at the testimony, which doesn't understand the actual situation the defendant describes at all but does pick up the statistical association between African American vernacular words used and higher sentences in its training corpus. Much easier to train a model that picks up the prejudice in sentencing factors than the situational nuance...

stickfigure 5y ago
I think the point is that we are currently using opaque, poorly trained and erratic ML (Meat Learning) algorithms. Oh sure the meatbox says that it's allocating N years for this, M years for that, but empirical studies have proven that this readout is nonsense. Which kind of ML is easier to train to be non-racist? I don't know, but it's not obvious to me that meat is the winner here.