4 ms·

The subtitle is also important: Non-binding resolution also asks for AI-based predictive policing ban. AI, by its complexity and opacity should not be used to

The subtitle is also important: Non-binding resolution also asks for AI-based predictive policing ban. AI, by its complexity and opacity should not be used to profile people. Just because it could be so easily manipulated.

hprotagonist 5y ago
Why are opacity and complexity so often invoked as a reason here? I don’t want to be mass-profiling populations with totally clear and simple technologies, either.
- capableweb 5y ago
  My guess is that "opacity" is referring to implementations, while "complexity" is referring to the effects. "Totally clear and simple technologies" is referring to just the implementation. We don't want to get involved with AI because of the complex issues it opens, not because the current implementations are complex.
- lmkg 5y ago
  Opacity (of which complexity is a form) is an impediment to accountability. I agree there are many reasons that I don't want mass surveillance and profiling, but I think a lack of accountability of a system is a very large force-multiplier for the malignant effects its ends up having.
  - raxxorrax 5y ago
    What do you think about police having to fulfill certain quotas in their policing?
- mrweasel 5y ago
  Opacity because if you are detained by the police they should be able to tell you why. Many, if not all, AI solution based on a training sets will have a hard time telling you exactly why you where flagged. Also even if they can tell you that the AI believe that the combination of your ears and nose makes you look like some wanted person, what are you suppose to do about it?
  - atoav 5y ago
    Maybe we can develop AI that decides who has a criminal look based on the shape of the skull? /s As someone living in Germany all these AI approaches have a distinct fascist flavour to them and remind me of my Nazi grandfather who claimed he "could just tell" our bosnian neighbours have "something criminal" in their blood.
- vagrantJin 5y ago
  Even if the below commentor is correct about the lack of accountability - which I assume is important in a democracy - and the more insidious danger is the obvious moral and ethical issues.
  - avianlyric 5y ago
    Could you be a little clearer on what the “obvious moral and ethical issues” are? Personally I’m not sure what you’re referencing, and I also don’t think the many of the moral and ethical issues are obvious. If they were, then I suspect countries like the UK wouldn’t already be trialing many of these technologies.
    - raxxorrax 5y ago
      AI today is on a level of an illiterate toddler with a talent for stochastic at least. Using it to classify people is unethical because it puts suspicion on them without evidence or sense.
- raxxorrax 5y ago
  Exactly, it is not that necessity would demand it anyway. Security today is often emotional and if there is no real threat, it will be made up.
toss1 5y ago
>>AI, by its complexity and opacity should not be used to profile people. Just because it could be so easily manipulated. True, but I'd add in a touch of Hanlon's Razor — don't attribute to malice what can be attributed to mere incompetence — far too much of the time, the technology is just bad and wrong. It has been over-promised and under-delivered. It is so often wrong that there are numerous lawsuits, including one today about Uber being sued because the so-called "AI Facial Recognition" system cannot even recognize it's own employees [1]. Great idea to verify everyone automatically, but if the technology sucks, it is worse than nothing. As if falsely losing your employment is bad enough, the consequences of broad adoption of technology that is not fit for purpose is far worse than merely wasting money and time of law enforcement, courts, etc - it will literally falsely steal people's freedom, while letting actual criminals run loose. The EU is wise to call a halt to this over-hyped tech. (And frankly, calling it "AI" is ridiculous — it is just a fancy pattern-match search. This is what the industry gets for over-selling it's new stuff before it is actually ready - and deservedly so. It'd be an entirely different story if they'd fully developed and tested it until it was six+-sigma reliable). [1] https://techcrunch.com/2021/10/05/uber-faces-legal-action-over-racially-discriminatory-facial-recognition-id-checks/ https://techcrunch.com/2021/10/05/uber-faces-legal-action-ov...
- rsynnott 5y ago
  > don't attribute to malice what can be attributed to mere incompetence Police use of this nonsense is either malice or _negligence_; it's not mere incompetence. If you're the enforcement arm of the state, you largely lose the right to plead incompetence; if you don't do the research; that's negligent.
  - potatoman22 5y ago
    I don't think police using ML is negligent in and of itself. Overly relying on a model, however, would certainly be negligent.
    - toss1 5y ago
      hmm, considering the seriously deficient state of these systems, I'd have to agree with rsynnott that using it without a very serious overlay of human checking before taking any action - probably enough to outweigh the benefits of the "AI" itself - would constitute negligence. The PDs either already know that the "AI" FR systems are not fit for purpose and are using them anyway without proper checking, or they don't know because they couldn't be bothered to do their due diligence. I want to catch the crooks as much as anyone, but catching the wrong people does no one any good, and has a hugely corrosive effect on society.
cmrdporcupine 5y ago
I agree with the principle but the problem here is the word "AI" or "ML"; what we are talking about is really just fancy statistics. Are we forbidding statistical modeling of any kind? How would we even describe or enforce that? The ML courses I took started with just running a linear regression on a large data set, and those were considered a form of ML. This is tricky. And the problem is not just the techniques, but can we trust our institutions and practitioners -- what kind of systemic and unconscious biases are driving the misuses or potential misuses of ML?

hprotagonist 5y ago

Why are opacity and complexity so often invoked as a reason here? I don’t want to be mass-profiling populations with totally clear and simple technologies, either.

capableweb 5y ago
My guess is that "opacity" is referring to implementations, while "complexity" is referring to the effects. "Totally clear and simple technologies" is referring to just the implementation. We don't want to get involved with AI because of the complex issues it opens, not because the current implementations are complex.
lmkg 5y ago
Opacity (of which complexity is a form) is an impediment to accountability. I agree there are many reasons that I don't want mass surveillance and profiling, but I think a lack of accountability of a system is a very large force-multiplier for the malignant effects its ends up having.
- raxxorrax 5y ago
  What do you think about police having to fulfill certain quotas in their policing?
mrweasel 5y ago
Opacity because if you are detained by the police they should be able to tell you why. Many, if not all, AI solution based on a training sets will have a hard time telling you exactly why you where flagged. Also even if they can tell you that the AI believe that the combination of your ears and nose makes you look like some wanted person, what are you suppose to do about it?
- atoav 5y ago
  Maybe we can develop AI that decides who has a criminal look based on the shape of the skull? /s As someone living in Germany all these AI approaches have a distinct fascist flavour to them and remind me of my Nazi grandfather who claimed he "could just tell" our bosnian neighbours have "something criminal" in their blood.
vagrantJin 5y ago
Even if the below commentor is correct about the lack of accountability - which I assume is important in a democracy - and the more insidious danger is the obvious moral and ethical issues.
- avianlyric 5y ago
  Could you be a little clearer on what the “obvious moral and ethical issues” are? Personally I’m not sure what you’re referencing, and I also don’t think the many of the moral and ethical issues are obvious. If they were, then I suspect countries like the UK wouldn’t already be trialing many of these technologies.
  - raxxorrax 5y ago
    AI today is on a level of an illiterate toddler with a talent for stochastic at least. Using it to classify people is unethical because it puts suspicion on them without evidence or sense.
raxxorrax 5y ago
Exactly, it is not that necessity would demand it anyway. Security today is often emotional and if there is no real threat, it will be made up.

toss1 5y ago

>>AI, by its complexity and opacity should not be used to profile people. Just because it could be so easily manipulated. True, but I'd add in a touch of Hanlon's Razor — don't attribute to malice what can be attributed to mere incompetence — far too much of the time, the technology is just bad and wrong. It has been over-promised and under-delivered. It is so often wrong that there are numerous lawsuits, including one today about Uber being sued because the so-called "AI Facial Recognition" system cannot even recognize it's own employees [1]. Great idea to verify everyone automatically, but if the technology sucks, it is worse than nothing. As if falsely losing your employment is bad enough, the consequences of broad adoption of technology that is not fit for purpose is far worse than merely wasting money and time of law enforcement, courts, etc - it will literally falsely steal people's freedom, while letting actual criminals run loose. The EU is wise to call a halt to this over-hyped tech. (And frankly, calling it "AI" is ridiculous — it is just a fancy pattern-match search. This is what the industry gets for over-selling it's new stuff before it is actually ready - and deservedly so. It'd be an entirely different story if they'd fully developed and tested it until it was six+-sigma reliable). [1] https://techcrunch.com/2021/10/05/uber-faces-legal-action-over-racially-discriminatory-facial-recognition-id-checks/ https://techcrunch.com/2021/10/05/uber-faces-legal-action-ov...

rsynnott 5y ago
> don't attribute to malice what can be attributed to mere incompetence Police use of this nonsense is either malice or _negligence_; it's not mere incompetence. If you're the enforcement arm of the state, you largely lose the right to plead incompetence; if you don't do the research; that's negligent.
- potatoman22 5y ago
  I don't think police using ML is negligent in and of itself. Overly relying on a model, however, would certainly be negligent.
  - toss1 5y ago
    hmm, considering the seriously deficient state of these systems, I'd have to agree with rsynnott that using it without a very serious overlay of human checking before taking any action - probably enough to outweigh the benefits of the "AI" itself - would constitute negligence. The PDs either already know that the "AI" FR systems are not fit for purpose and are using them anyway without proper checking, or they don't know because they couldn't be bothered to do their due diligence. I want to catch the crooks as much as anyone, but catching the wrong people does no one any good, and has a hugely corrosive effect on society.

cmrdporcupine 5y ago

I agree with the principle but the problem here is the word "AI" or "ML"; what we are talking about is really just fancy statistics. Are we forbidding statistical modeling of any kind? How would we even describe or enforce that? The ML courses I took started with just running a linear regression on a large data set, and those were considered a form of ML. This is tricky. And the problem is not just the techniques, but can we trust our institutions and practitioners -- what kind of systemic and unconscious biases are driving the misuses or potential misuses of ML?