5 ms·

>>AI, by its complexity and opacity should not be used to profile people. Just because it could be so easily manipulated. True, but I'd add in a touch of Hanlo

by toss1 5y ago

>>AI, by its complexity and opacity should not be used to profile people. Just because it could be so easily manipulated. True, but I'd add in a touch of Hanlon's Razor — don't attribute to malice what can be attributed to mere incompetence — far too much of the time, the technology is just bad and wrong. It has been over-promised and under-delivered. It is so often wrong that there are numerous lawsuits, including one today about Uber being sued because the so-called "AI Facial Recognition" system cannot even recognize it's own employees [1]. Great idea to verify everyone automatically, but if the technology sucks, it is worse than nothing. As if falsely losing your employment is bad enough, the consequences of broad adoption of technology that is not fit for purpose is far worse than merely wasting money and time of law enforcement, courts, etc - it will literally falsely steal people's freedom, while letting actual criminals run loose. The EU is wise to call a halt to this over-hyped tech. (And frankly, calling it "AI" is ridiculous — it is just a fancy pattern-match search. This is what the industry gets for over-selling it's new stuff before it is actually ready - and deservedly so. It'd be an entirely different story if they'd fully developed and tested it until it was six+-sigma reliable). [1] https://techcrunch.com/2021/10/05/uber-faces-legal-action-over-racially-discriminatory-facial-recognition-id-checks/ https://techcrunch.com/2021/10/05/uber-faces-legal-action-ov...

rsynnott 5y ago
> don't attribute to malice what can be attributed to mere incompetence Police use of this nonsense is either malice or _negligence_; it's not mere incompetence. If you're the enforcement arm of the state, you largely lose the right to plead incompetence; if you don't do the research; that's negligent.
- potatoman22 5y ago
  I don't think police using ML is negligent in and of itself. Overly relying on a model, however, would certainly be negligent.
  - toss1 5y ago
    hmm, considering the seriously deficient state of these systems, I'd have to agree with rsynnott that using it without a very serious overlay of human checking before taking any action - probably enough to outweigh the benefits of the "AI" itself - would constitute negligence. The PDs either already know that the "AI" FR systems are not fit for purpose and are using them anyway without proper checking, or they don't know because they couldn't be bothered to do their due diligence. I want to catch the crooks as much as anyone, but catching the wrong people does no one any good, and has a hugely corrosive effect on society.

by toss1 5y ago

rsynnott 5y ago

> don't attribute to malice what can be attributed to mere incompetence Police use of this nonsense is either malice or _negligence_; it's not mere incompetence. If you're the enforcement arm of the state, you largely lose the right to plead incompetence; if you don't do the research; that's negligent.

potatoman22 5y ago
I don't think police using ML is negligent in and of itself. Overly relying on a model, however, would certainly be negligent.
- toss1 5y ago
  hmm, considering the seriously deficient state of these systems, I'd have to agree with rsynnott that using it without a very serious overlay of human checking before taking any action - probably enough to outweigh the benefits of the "AI" itself - would constitute negligence. The PDs either already know that the "AI" FR systems are not fit for purpose and are using them anyway without proper checking, or they don't know because they couldn't be bothered to do their due diligence. I want to catch the crooks as much as anyone, but catching the wrong people does no one any good, and has a hugely corrosive effect on society.