5 ms·

Stating that method A is problematic does not automatically mean method B is better. > The reality is humans are far worse Citation needed - especially when c

by Smerity 10y ago

Stating that method A is problematic does not automatically mean method B is better. > The reality is humans are far worse Citation needed - especially when comparing against a specific instantiation of a machine learning model. Papers published by the statistician in the article used only 516 data points. Most data scientists running an A/B test wouldn't change their homepage with only 516 data points. There's no guarantee the methods he is using for the parole model involve better datasets or models without deep flaws. An algorithm or machine learning model is not magically less biased than the process it is replacing. Indeed, if it's trained on biased data, as you believe by stating "never ever trust human judges", then the models are inherently biased in the exact same way. If you give a machine learning model a dataset where one feature appears entirely indicative (remember: correlation is not causation), it can overfit to that, even if that does not reflect reality. I highly recommend reading "How big data is unfair: understanding unintended sources of unfairness in data driven decision making"[1], by Moritz Hardt, a Google machine learning researcher who has published on the topic (see: Fairness, Accountability, Transparency). It is a non-technical and general introduction to some of the many issues that can result in bias and prejudice in machine learning models. To summarize, "machine learning is not, by default, fair or just in any meaningful way". Algorithms and machine learning models _can_ be biased, for many reasons. Without proper analysis, we don't know whether it's a good or bad model, full stop. [1]: https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de#.ykd1vedz1 https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de...

zamalek 10y ago
Additionally, before we can approach AI justice we'd need artificial intuition. If you know what the local minima are, you could game the model to convict or defend anyone. The AI would need a way to determine that it is being gamed. For example: taking pictures of bullet casings framed in a specific way might guarantee convictions. Sometimes there is a minority report.
Houshalter 10y ago
>Citation needed - especially when comparing against a specific instantiation of a machine learning model. Papers published by the statistician in the article used only 516 data points. Most data scientists running an A/B test wouldn't change their homepage with only 516 data points. There's no guarantee the methods he is using for the parole model involve better datasets or models without deep flaws. 516 is more than enough to fit a simple model. As long as you use cross validation and hold out tests to make sure you aren't fitting. 516 data points is more than a person needs to see to be called an "expert". Many of the algorithms I referenced used fewer data points, or even totally unoptimized weights, and still beat human experts. >An algorithm or machine learning model is not magically less biased than the process it is replacing. Indeed, if it's trained on biased data, as you believe by stating "never ever trust human judges", then the models are inherently biased in the exact same way. We have ground truth though. Whether someone will be convicted is a fairly objective measure. Even if it's slightly biased, it's still the best possible indicator we have of whether or not someone should be released. If you had a time machine that could go into the future and see who would be convicted, would you still argue against using that information, because it might be biased? Leaving people to rot in prison, even if all the statistics point to them being very low risk, is just wrong. >"machine learning is not, by default, fair or just in any meaningful way" Humans are not, by default, fair or just in any meaningful way. Nor are they accurate at prediction. Any argument you can possibly use against algorithms applies even more to humans. That's my entire point. You should trust humans far, far less than you do.
- pdkl95 10y ago
  > You should trust humans far, far less than you do. Which is why I don't trust the people picking the algorithm. You still have human bias, but now they are easier to hide behind complicated algorithms and unreliable data. edit: removed original editing error edit2: You say I should trust the algorithm, but y9u seem to be going out of your way to ignore that the algorithm itself has to be created by someone. You haven't reduced the amount of bias; trusting an algorithm simply codifies the author's bias.
  - Houshalter 10y ago
    You should trust the "complicated" algorithm far more than you trust the complicated human brain of the judge, who was also trained on unreliable data. Look it's easy to verify whether parole officers are better at predicting recidivism than an algorithm. If the algorithm is objectively better than it should be used.
    - kragen 10y ago
      Given an unbiased algorithm that does better at predicting recidivism, it would be easy to deliberately construct an algorithm that does almost as much better, but is egregiously biased. For example, if you had been wronged by somebody named Thiel, you could persuade it to never recommend parole for anybody named Thiel. There aren't enough people named Thiel for this to substantially worsen its measured performance. Given that it's easy to construct an example of how you could deliberately do this, and it's so easy to accidentally overfit machine-learning algorithms, we should be very concerned about people accidentally doing this. An easy way would be to try a few thousand different algorithm variants and have a biased group of people eyeball the results to see which ones look good. If those people are racist, for example, they could subconsciously give undue weight to type 1 errors for white prisoners and type 2 errors for black prisoners, or vice versa. The outcome of the process would be an algorithm that is "objectively better" by whatever measure you optimize it for, but still unjustly worsens the situation for some group of people. A potential advantage of algorithms and other rules is that, unlike the brain of the judge, they can be publicly analyzed, and the analyses debated. This is the basis for the idea of "rule of law". Aside from historical precedents, though, the exploitation of the publicly-analyzed DAO algorithms should give us pause on this count. Deeper rule of law may help, but it could easily make the situation worse. We must be skeptical, and we must do better.
apathy 10y ago
Re: most data scientists wouldn't change their home page based on 516 data points: Out of curiosity, if you find that in 240 out of 261 small round blue cell tumors, your algorithm can make a better treatment decision than the pathologist, do you wait for more, or get the algorithm into trials ASAP? Careful now. I didn't mention how long you might be waiting. This is a domain specific trick question. Furthermore, we have an appeals process to address sentencing mistakes. It isn't easy, but if the risk scoring algorithm turns out to be materially wrong and contributes to a harsher sentence than warranted, that can be considered a factual error and the basis for resentencing. It's almost as if our justice system is geared towards a reinforcement learning approach, due to the consequences of both action and inaction...

by Smerity 10y ago

zamalek 10y ago

Additionally, before we can approach AI justice we'd need artificial intuition. If you know what the local minima are, you could game the model to convict or defend anyone. The AI would need a way to determine that it is being gamed. For example: taking pictures of bullet casings framed in a specific way might guarantee convictions. Sometimes there is a minority report.

Houshalter 10y ago

>Citation needed - especially when comparing against a specific instantiation of a machine learning model. Papers published by the statistician in the article used only 516 data points. Most data scientists running an A/B test wouldn't change their homepage with only 516 data points. There's no guarantee the methods he is using for the parole model involve better datasets or models without deep flaws. 516 is more than enough to fit a simple model. As long as you use cross validation and hold out tests to make sure you aren't fitting. 516 data points is more than a person needs to see to be called an "expert". Many of the algorithms I referenced used fewer data points, or even totally unoptimized weights, and still beat human experts. >An algorithm or machine learning model is not magically less biased than the process it is replacing. Indeed, if it's trained on biased data, as you believe by stating "never ever trust human judges", then the models are inherently biased in the exact same way. We have ground truth though. Whether someone will be convicted is a fairly objective measure. Even if it's slightly biased, it's still the best possible indicator we have of whether or not someone should be released. If you had a time machine that could go into the future and see who would be convicted, would you still argue against using that information, because it might be biased? Leaving people to rot in prison, even if all the statistics point to them being very low risk, is just wrong. >"machine learning is not, by default, fair or just in any meaningful way" Humans are not, by default, fair or just in any meaningful way. Nor are they accurate at prediction. Any argument you can possibly use against algorithms applies even more to humans. That's my entire point. You should trust humans far, far less than you do.

pdkl95 10y ago
> You should trust humans far, far less than you do. Which is why I don't trust the people picking the algorithm. You still have human bias, but now they are easier to hide behind complicated algorithms and unreliable data. edit: removed original editing error edit2: You say I should trust the algorithm, but y9u seem to be going out of your way to ignore that the algorithm itself has to be created by someone. You haven't reduced the amount of bias; trusting an algorithm simply codifies the author's bias.
- Houshalter 10y ago
  You should trust the "complicated" algorithm far more than you trust the complicated human brain of the judge, who was also trained on unreliable data. Look it's easy to verify whether parole officers are better at predicting recidivism than an algorithm. If the algorithm is objectively better than it should be used.
  - kragen 10y ago
    Given an unbiased algorithm that does better at predicting recidivism, it would be easy to deliberately construct an algorithm that does almost as much better, but is egregiously biased. For example, if you had been wronged by somebody named Thiel, you could persuade it to never recommend parole for anybody named Thiel. There aren't enough people named Thiel for this to substantially worsen its measured performance. Given that it's easy to construct an example of how you could deliberately do this, and it's so easy to accidentally overfit machine-learning algorithms, we should be very concerned about people accidentally doing this. An easy way would be to try a few thousand different algorithm variants and have a biased group of people eyeball the results to see which ones look good. If those people are racist, for example, they could subconsciously give undue weight to type 1 errors for white prisoners and type 2 errors for black prisoners, or vice versa. The outcome of the process would be an algorithm that is "objectively better" by whatever measure you optimize it for, but still unjustly worsens the situation for some group of people. A potential advantage of algorithms and other rules is that, unlike the brain of the judge, they can be publicly analyzed, and the analyses debated. This is the basis for the idea of "rule of law". Aside from historical precedents, though, the exploitation of the publicly-analyzed DAO algorithms should give us pause on this count. Deeper rule of law may help, but it could easily make the situation worse. We must be skeptical, and we must do better.

apathy 10y ago

Re: most data scientists wouldn't change their home page based on 516 data points: Out of curiosity, if you find that in 240 out of 261 small round blue cell tumors, your algorithm can make a better treatment decision than the pathologist, do you wait for more, or get the algorithm into trials ASAP? Careful now. I didn't mention how long you might be waiting. This is a domain specific trick question. Furthermore, we have an appeals process to address sentencing mistakes. It isn't easy, but if the risk scoring algorithm turns out to be materially wrong and contributes to a harsher sentence than warranted, that can be considered a factual error and the basis for resentencing. It's almost as if our justice system is geared towards a reinforcement learning approach, due to the consequences of both action and inaction...