6 ms·

“AI may quickly point out a corrupt official, but it is not very good at explaining the process it has gone through to reach such a conclusion,” the researcher

by Renaud 8y ago

“AI may quickly point out a corrupt official, but it is not very good at explaining the process it has gone through to reach such a conclusion,” the researcher said. “Although it gets it right in most cases, you need a human to work closely with it.” And that's the crux of the issue with AI being used in any law enforcement situation. If we allow decisions and conclusions being drawn by the AI without a clear explanation of how it got there, we're just creating a monster that will advise -maybe replace- the judgment of law enforcement professionals who won't have the means to question these decisions. Catching corrupt officials is a laudable goal anywhere and I'd like to see it applied to highlight potential irregularities that may require a second look but the danger here is that an unprovable AI be used to make claims or start being used as sufficient evidence to ruin people's lives.

newsbinator 8y ago
I wonder if it gets it right in most cases because officials are corrupt in most cases. It's like building an AI that guesses when it's going to be a rainy day in Seattle.
- WalterBright 8y ago
  My Seattle rain-forecasting algorithm is very good. It forecasts rain tomorrow if it is raining today, and not-rain if it is not raining today.
- dsfyu404ed 8y ago
  >I wonder if it gets it right in most cases because officials are corrupt in most cases. This with a large helping of "the laws were intentionally written to be broad" on the side. With a suitably inclusive definition of "corrupt" every official is corrupt.
jlarocco 8y ago
> If we allow decisions and conclusions being drawn by the AI without a clear explanation of how it got there The funny thing is expert systems were doing that 30 years ago. Seems like this would be a perfect use of them but it's not a buzzword any more. It's crazy how many wheels get poorly reinvented in the software world.
- soraki_soladead 8y ago
  Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with recidivism at all since expert systems were simply built to emulate (human) expert decisions. While it's very easy to do machine learning _incorrectly_ it is also possible to reasonably attribute factors to outcomes based on large quantities of data. You can also look a LIME/SHAP and other factor contribution metrics. This seems like a significant improvement over expert systems.
  - jlarocco 8y ago
    > Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with recidivism at all since expert systems were simply built to emulate (human) expert decisions. Re-reading the article now, it may very well be an expert system. The "big data" mentioned isn't training data, but pre-existing databases like bank account info, salaries, contracts, property ownership, etc. FTA: > Disciplinary officials need to help scientists train the machine with their experience and knowledge accumulated from previous cases. For instance, disciplinary officials spent many hours manually tagging unusual phenomenon in various types of data sets to teach the machine what to look for. As far as explaining the results, it's hard to draw any conclusion without knowing exactly what "not very good at explaining the process it has gone through " means.
narrator 8y ago
This reminds me of Klaatu's speech in "The Day The Earth Stood Still." They created a race of robots to prevent aggression. "The test of any such higher authority is, of course, the police force that supports it. For our policemen, we created a race of robots. Their function is to patrol the planets—in space ships like this one—and preserve the peace. In matters of aggression, we have given them absolute power over us; this power can not be revoked. At the ?rst sign of violence, they act automatically against the aggressor. The penalty for provoking their action is too terrible to risk. The result is that we live in peace, without arms or armies, secure in the knowledge that we are free from aggression and war—free to pursue more pro?table enterprises. Now, we do not pretend to have achieved perfection, but we do have a system, and it works."[1] [1] https://www.rottentomatoes.com/m/1005371_day_the_earth_stood_still/quotes https://www.rottentomatoes.com/m/1005371_day_the_earth_stood...
- NeedMoreTea 8y ago
  Fabulous movie. The 1950s one anyway. We'll not speak of the 00's awfulness. If we continue to blindly stumble towards that future we are likely to set 21st century (USA) biases in stone. Racial, national, sexual and corporate biases etc will be baked in. We've seen they usually are in the systems we have already. Yet it will be done mostly innocently, and unaware from accidental bias in training data. Makes for an interesting future.
- WalterBright 8y ago
  I doubt such a scheme would work. People would just engage in forms of aggression the robots would not recognize. It's like today's popular notion that one can control the thoughts of others by banning certain words.
  - TheOtherHobbes 8y ago
    There's a rather obvious gap between calling someone names and throwing nuclear weapons at them. As for controlling thoughts - that's the easiest thing in the world today. There are entire industries devoted to making sure people think as they're told to, and act as they're told to. Of course they don't work on everyone. But if you can fool more than half of the people when you need to, you can do more or less anything you want.
darkpuma 8y ago
They're solving the wrong problem. Instead of writing "an AI that determines guilt" (e.g. a classifier) they should be trying to write "an AI that compiles a list of evidence against the accused". This list of evidence would then be subjected to the same scrutiny that a list of evidence from a department of human police officers would be subjected to. If the AI cannot make the case against somebody in a way that's persuasive to a human prosecutor, then the case goes nowhere. If the AI cannot provide rational clear and understandable justification for it's assertion, then it's not ready to replace humans who can be expected to provide justifications for their beliefs. Police officers are expected to write police reports, and so should any AI that's meant to do their job. Right now they've just got the AI equivalent of a cop who says "I feel in my gut that he's guilty, and I'm not going to explain why." But because it's "AI" I guess some politicians thought that was good enough. Of course doing it right is probably harder.
- AnthonyMouse 8y ago
  > Instead of writing "an AI that determines guilt" (e.g. a classifier) they should be trying to write "an AI that compiles a list of evidence against the accused". The problem there is selection bias. If you have a list of fifteen things, three make it slightly more likely they've committed a crime. Seven make it slightly less likely. Two make it significantly more likely and three make it significantly less likely. If you consider them all, the probability they've committed a crime is in the "probably not" range. But if you take the list and prune everything that makes it less likely, leaving only the factors that make it more likely, you get a distorted result. This is what prosecutors do for juries in general, and it's problematic. Now suppose instead of a list of fifteen you start with a list of fifteen million and do the same thing.
  - darkpuma 8y ago
    That's no different from how regular law enforcement works. It's the standard to which any autonomous law enforcement should be implemented. If you want to build better prosecutors that understand statistics, that should be considered a separate project. One AI that assembles the best case against the accused, and another AI that critically examines that case and discards it if it fails to meet some threshold for statistical relevance, or passes it on to human prosecutors if it does.
sonnyblarney 8y ago
"If we allow decisions and conclusions being drawn by the AI" Im doubtful that anyone is getting charted without specifics. Moreover, with this amount of corruption, it seems clear that just some basic oversight is all that is necessary.
kumarvvr 8y ago
> Although it gets it right in most cases, you need a human to work closely with it Is it just me or does that simply show that there is a lot of casual corruption in China?

by Renaud 8y ago

newsbinator 8y ago

I wonder if it gets it right in most cases because officials are corrupt in most cases. It's like building an AI that guesses when it's going to be a rainy day in Seattle.

WalterBright 8y ago
My Seattle rain-forecasting algorithm is very good. It forecasts rain tomorrow if it is raining today, and not-rain if it is not raining today.
dsfyu404ed 8y ago
>I wonder if it gets it right in most cases because officials are corrupt in most cases. This with a large helping of "the laws were intentionally written to be broad" on the side. With a suitably inclusive definition of "corrupt" every official is corrupt.

jlarocco 8y ago

> If we allow decisions and conclusions being drawn by the AI without a clear explanation of how it got there The funny thing is expert systems were doing that 30 years ago. Seems like this would be a perfect use of them but it's not a buzzword any more. It's crazy how many wheels get poorly reinvented in the software world.

soraki_soladead 8y ago
Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with recidivism at all since expert systems were simply built to emulate (human) expert decisions. While it's very easy to do machine learning _incorrectly_ it is also possible to reasonably attribute factors to outcomes based on large quantities of data. You can also look a LIME/SHAP and other factor contribution metrics. This seems like a significant improvement over expert systems.
- jlarocco 8y ago
  > Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with recidivism at all since expert systems were simply built to emulate (human) expert decisions. Re-reading the article now, it may very well be an expert system. The "big data" mentioned isn't training data, but pre-existing databases like bank account info, salaries, contracts, property ownership, etc. FTA: > Disciplinary officials need to help scientists train the machine with their experience and knowledge accumulated from previous cases. For instance, disciplinary officials spent many hours manually tagging unusual phenomenon in various types of data sets to teach the machine what to look for. As far as explaining the results, it's hard to draw any conclusion without knowing exactly what "not very good at explaining the process it has gone through " means.

narrator 8y ago

This reminds me of Klaatu's speech in "The Day The Earth Stood Still." They created a race of robots to prevent aggression. "The test of any such higher authority is, of course, the police force that supports it. For our policemen, we created a race of robots. Their function is to patrol the planets—in space ships like this one—and preserve the peace. In matters of aggression, we have given them absolute power over us; this power can not be revoked. At the ?rst sign of violence, they act automatically against the aggressor. The penalty for provoking their action is too terrible to risk. The result is that we live in peace, without arms or armies, secure in the knowledge that we are free from aggression and war—free to pursue more pro?table enterprises. Now, we do not pretend to have achieved perfection, but we do have a system, and it works."[1] [1] https://www.rottentomatoes.com/m/1005371_day_the_earth_stood_still/quotes https://www.rottentomatoes.com/m/1005371_day_the_earth_stood...

NeedMoreTea 8y ago
Fabulous movie. The 1950s one anyway. We'll not speak of the 00's awfulness. If we continue to blindly stumble towards that future we are likely to set 21st century (USA) biases in stone. Racial, national, sexual and corporate biases etc will be baked in. We've seen they usually are in the systems we have already. Yet it will be done mostly innocently, and unaware from accidental bias in training data. Makes for an interesting future.
WalterBright 8y ago
I doubt such a scheme would work. People would just engage in forms of aggression the robots would not recognize. It's like today's popular notion that one can control the thoughts of others by banning certain words.
- TheOtherHobbes 8y ago
  There's a rather obvious gap between calling someone names and throwing nuclear weapons at them. As for controlling thoughts - that's the easiest thing in the world today. There are entire industries devoted to making sure people think as they're told to, and act as they're told to. Of course they don't work on everyone. But if you can fool more than half of the people when you need to, you can do more or less anything you want.

darkpuma 8y ago

They're solving the wrong problem. Instead of writing "an AI that determines guilt" (e.g. a classifier) they should be trying to write "an AI that compiles a list of evidence against the accused". This list of evidence would then be subjected to the same scrutiny that a list of evidence from a department of human police officers would be subjected to. If the AI cannot make the case against somebody in a way that's persuasive to a human prosecutor, then the case goes nowhere. If the AI cannot provide rational clear and understandable justification for it's assertion, then it's not ready to replace humans who can be expected to provide justifications for their beliefs. Police officers are expected to write police reports, and so should any AI that's meant to do their job. Right now they've just got the AI equivalent of a cop who says "I feel in my gut that he's guilty, and I'm not going to explain why." But because it's "AI" I guess some politicians thought that was good enough. Of course doing it right is probably harder.

AnthonyMouse 8y ago
> Instead of writing "an AI that determines guilt" (e.g. a classifier) they should be trying to write "an AI that compiles a list of evidence against the accused". The problem there is selection bias. If you have a list of fifteen things, three make it slightly more likely they've committed a crime. Seven make it slightly less likely. Two make it significantly more likely and three make it significantly less likely. If you consider them all, the probability they've committed a crime is in the "probably not" range. But if you take the list and prune everything that makes it less likely, leaving only the factors that make it more likely, you get a distorted result. This is what prosecutors do for juries in general, and it's problematic. Now suppose instead of a list of fifteen you start with a list of fifteen million and do the same thing.
- darkpuma 8y ago
  That's no different from how regular law enforcement works. It's the standard to which any autonomous law enforcement should be implemented. If you want to build better prosecutors that understand statistics, that should be considered a separate project. One AI that assembles the best case against the accused, and another AI that critically examines that case and discards it if it fails to meet some threshold for statistical relevance, or passes it on to human prosecutors if it does.

sonnyblarney 8y ago

"If we allow decisions and conclusions being drawn by the AI" Im doubtful that anyone is getting charted without specifics. Moreover, with this amount of corruption, it seems clear that just some basic oversight is all that is necessary.

kumarvvr 8y ago

> Although it gets it right in most cases, you need a human to work closely with it Is it just me or does that simply show that there is a lot of casual corruption in China?