6 ms·

There is a human review (ugh.) as well. But still, your example is very concerning.

by throw03172019 5y ago

falcolas 5y ago
From Apple's announcements, there is a human comparison of the visual hashes. They don't appear to be forcing humans to view the images themselves.
- y7 5y ago
  The technical assessments speak about "visual derivatives". I'm guessing they mean a low resolution thumbnail.
foxyv 5y ago
They say there is a human review, but fail to explain the parameters of the review. How many photos is a reviewer expected to review in an hour? What are their incentives and performance metrics? Do they have a 3 party voting system to confirm? For positive human review is there additional scrutiny? Is data anonymized properly? Does law enforcement receive a copy? Is this going to be limited to child exploitation? Without this info I just have to assume it's some guy in a box expected to review 1000 pictures an hour and gets scrutinized if he doesn't click at least 10% positive. So he's just counting 1 positive 9 negatives.
- throw03172019 5y ago
  I would hope they aren’t incentivized to incriminate innocent people.
  - foxyv 5y ago
    I would hope so as well. However, my hopes for these policing algorithms have been dashed too many times. The potential for abuse is just too great without effective laws and appropriate procedures. We've seen the results at YouTube, Pasco County Florida, the UK, and Facebook. There is a reason that a key part of justice is being able to face your accusers. Secret algorithms are being used to abrogate this right. You can't cross examine an algorithm. Expert witnesses to contradict its expertise are too expensive for most defendants. Then the developers of the algorithm can claim with plausible deniability that the algorithm is 1 in a trillion accurate. It's essentially the new bite mark analysis and polygraph pseudoscience used to elicit false admissions of guilt or convince gullible juries. "We have the hash of your photo George, we KNOW you are a pedophile. Just admit it and you can go home. We'll go easy on you. We promise!" https://www.forbes.com/sites/nicksibilla/2021/04/26/lawsuit-florida-county-uses-predictive-policing-to-arrest-residents-for-petty-code-violations/ https://www.forbes.com/sites/nicksibilla/2021/04/26/lawsuit-...
- gjsman-1000 5y ago
  Remember they said the odds of an account being falsely reported were "1 in a trillion." We have no math to back that, some have called it BS. However... that would mean that one person could easily review only a few photos a week even if it was way off.

There is a human review (ugh.) as well. But still, your example is very concerning.

From Apple's announcements, there is a human comparison of the visual hashes. They don't appear to be forcing humans to view the images themselves.

y7 5y ago
The technical assessments speak about "visual derivatives". I'm guessing they mean a low resolution thumbnail.

foxyv 5y ago

They say there is a human review, but fail to explain the parameters of the review. How many photos is a reviewer expected to review in an hour? What are their incentives and performance metrics? Do they have a 3 party voting system to confirm? For positive human review is there additional scrutiny? Is data anonymized properly? Does law enforcement receive a copy? Is this going to be limited to child exploitation? Without this info I just have to assume it's some guy in a box expected to review 1000 pictures an hour and gets scrutinized if he doesn't click at least 10% positive. So he's just counting 1 positive 9 negatives.

throw03172019 5y ago
I would hope they aren’t incentivized to incriminate innocent people.
- foxyv 5y ago
  I would hope so as well. However, my hopes for these policing algorithms have been dashed too many times. The potential for abuse is just too great without effective laws and appropriate procedures. We've seen the results at YouTube, Pasco County Florida, the UK, and Facebook. There is a reason that a key part of justice is being able to face your accusers. Secret algorithms are being used to abrogate this right. You can't cross examine an algorithm. Expert witnesses to contradict its expertise are too expensive for most defendants. Then the developers of the algorithm can claim with plausible deniability that the algorithm is 1 in a trillion accurate. It's essentially the new bite mark analysis and polygraph pseudoscience used to elicit false admissions of guilt or convince gullible juries. "We have the hash of your photo George, we KNOW you are a pedophile. Just admit it and you can go home. We'll go easy on you. We promise!" https://www.forbes.com/sites/nicksibilla/2021/04/26/lawsuit-florida-county-uses-predictive-policing-to-arrest-residents-for-petty-code-violations/ https://www.forbes.com/sites/nicksibilla/2021/04/26/lawsuit-...
gjsman-1000 5y ago
Remember they said the odds of an account being falsely reported were "1 in a trillion." We have no math to back that, some have called it BS. However... that would mean that one person could easily review only a few photos a week even if it was way off.