3 ms·

> You should trust humans far, far less than you do. Which is why I don't trust the people picking the algorithm. You still have human bias, but now they are e

by pdkl95 10y ago

> You should trust humans far, far less than you do. Which is why I don't trust the people picking the algorithm. You still have human bias, but now they are easier to hide behind complicated algorithms and unreliable data. edit: removed original editing error edit2: You say I should trust the algorithm, but y9u seem to be going out of your way to ignore that the algorithm itself has to be created by someone. You haven't reduced the amount of bias; trusting an algorithm simply codifies the author's bias.

Houshalter 10y ago
You should trust the "complicated" algorithm far more than you trust the complicated human brain of the judge, who was also trained on unreliable data. Look it's easy to verify whether parole officers are better at predicting recidivism than an algorithm. If the algorithm is objectively better than it should be used.
- kragen 10y ago
  Given an unbiased algorithm that does better at predicting recidivism, it would be easy to deliberately construct an algorithm that does almost as much better, but is egregiously biased. For example, if you had been wronged by somebody named Thiel, you could persuade it to never recommend parole for anybody named Thiel. There aren't enough people named Thiel for this to substantially worsen its measured performance. Given that it's easy to construct an example of how you could deliberately do this, and it's so easy to accidentally overfit machine-learning algorithms, we should be very concerned about people accidentally doing this. An easy way would be to try a few thousand different algorithm variants and have a biased group of people eyeball the results to see which ones look good. If those people are racist, for example, they could subconsciously give undue weight to type 1 errors for white prisoners and type 2 errors for black prisoners, or vice versa. The outcome of the process would be an algorithm that is "objectively better" by whatever measure you optimize it for, but still unjustly worsens the situation for some group of people. A potential advantage of algorithms and other rules is that, unlike the brain of the judge, they can be publicly analyzed, and the analyses debated. This is the basis for the idea of "rule of law". Aside from historical precedents, though, the exploitation of the publicly-analyzed DAO algorithms should give us pause on this count. Deeper rule of law may help, but it could easily make the situation worse. We must be skeptical, and we must do better.

by pdkl95 10y ago

Houshalter 10y ago

You should trust the "complicated" algorithm far more than you trust the complicated human brain of the judge, who was also trained on unreliable data. Look it's easy to verify whether parole officers are better at predicting recidivism than an algorithm. If the algorithm is objectively better than it should be used.

kragen 10y ago
Given an unbiased algorithm that does better at predicting recidivism, it would be easy to deliberately construct an algorithm that does almost as much better, but is egregiously biased. For example, if you had been wronged by somebody named Thiel, you could persuade it to never recommend parole for anybody named Thiel. There aren't enough people named Thiel for this to substantially worsen its measured performance. Given that it's easy to construct an example of how you could deliberately do this, and it's so easy to accidentally overfit machine-learning algorithms, we should be very concerned about people accidentally doing this. An easy way would be to try a few thousand different algorithm variants and have a biased group of people eyeball the results to see which ones look good. If those people are racist, for example, they could subconsciously give undue weight to type 1 errors for white prisoners and type 2 errors for black prisoners, or vice versa. The outcome of the process would be an algorithm that is "objectively better" by whatever measure you optimize it for, but still unjustly worsens the situation for some group of people. A potential advantage of algorithms and other rules is that, unlike the brain of the judge, they can be publicly analyzed, and the analyses debated. This is the basis for the idea of "rule of law". Aside from historical precedents, though, the exploitation of the publicly-analyzed DAO algorithms should give us pause on this count. Deeper rule of law may help, but it could easily make the situation worse. We must be skeptical, and we must do better.