7 ms·

How likely is it that you will have enough colliding images in your photo library to even trigger a review? I'm guessing you need at least 5 images, perhaps mu

How likely is it that you will have enough colliding images in your photo library to even trigger a review? I'm guessing you need at least 5 images, perhaps much more, to trigger it. In any case, 1 image is definitely not enough.

sandworm101 5y ago
For a normal random person, a grey man. In the real world, a single collision could be enough for the police to acquire further access to people they are already looking at. If the police want access to your phone for other reasons (drugs, taxes, illegal speech) they can use that one collision to get a warrant which will give them greater access. It is akin to cops wanting to search a car. They don't need a warrant. They only need to follow the car until it breaks any number of traffic laws. Then the resulting traffic stops lets them talk to the driver, who seems evasive, which gives them probable cause, which gets them a warrant to perform a full search. The accidental collision, week evidence of an offending image on the phone, opens the door to whatever other investigation they want to do.
- Grustaf 5y ago
  I'm not sure what you mean. If we are talking about Apple's new feature, they will not even be alerted until you have enough matches. And the police probably won't be using any neural hashes if they have access to your device, so I'm not sure what you are talking about.
  - sandworm101 5y ago
    That is an Apple current policy, not any sort of law. If the FBI wants to know if there are any collisions on a particular phone, that will be shared. If the FBI wants to know of all single collisions, that too must be shared. A corporation's internal policy decisions are nothing when faced with a government official carrying a warrant. If they have data indicating possible crimes, no matter how small, they can be forced to divulge it to investigators. Unlike Lavabit, I don't think Apple will ever close up shop to protect customer data.
    - Grustaf 5y ago
      Well of course if you believe that FBI can ask Apple to share anything and they will agree, and not tell us, then what is even the point of this discussion? Then they already have access to every photo, every email, every text. Because Apple controls those apps on your phone.
      - fsflover 5y ago
        The difference is that now they confirmed that they do search your private images.
        Grustaf 5y ago
        Why would that make it any likelier that they are also secretly searching your files, with another method? Seems completely orthogonal to me.
        fsflover 5y ago
        They showed that they are willing to do it in one case. Why not in other cases?
    - jasamer 5y ago
      I thought the system is designed such that the threshold must be met to check the matches, i.e. it’s not possible to follow the FBIs request without changing the entire system.
      - brokenmachine 5y ago
        int matchThreshold = 30; if (numMatches > matchThreshold) { snitch(customer) && customer.life.ruin() }
- merpnderp 5y ago
  Isn't Apple seeding their database with fake hits so that a court can't order them to turn over everyone who's just a few shy of Apple's threshold? So when this database leaks, won't there be a lot of people accused of horrific crimes simply because Apple seeded their database with random hits? Why would people pay a company to falsely accuse them of horrific crimes?
- JKCalhoun 5y ago
  Since, as I have read, Microsoft and Google already do this, where are all the illicit cop take-downs? I have not heard of any.
  - sandworm101 5y ago
    They are scanning images on online accounts, stuff that is stored on their systems and system files on machines (files they own). They are not, from what I have read, scanning customer-owned images stored on customer-owned devices.
    - snowwrestler 5y ago
      Every AV (antivirus) software system scans customer-owned files on customer-owned devices. So the question is the same: if it’s easy for law enforcement to deputize such a system, where is the flood of cases built off of evidence gathered this way?
      - sandworm101 5y ago
        >> Every AV (antivirus) software system Mine doesn't. If Ubuntu or ClamAV is scanning all my photos and reporting the results to Canonical against my will then I will soon be having words with Mr. Linus.
        snowwrestler 5y ago
        If law enforcement can force software companies to adapt their software to serve law enforcement purposes, why aren’t Ubuntu or ClamAV doing so already? What makes them better at resisting requests from law enforcement than Apple?
        MichaelGroves 5y ago
        Canonical and ClamAV aren't trying to curry favor with the US government to stave off antitrust action.
dhosek 5y ago
Apple has publicly stated 30. Plus, there would be a manual review of the images before forwarding to law enforcement. I don't see this collision attack leading to innocent people being handed over to the police. The worst-case (best-case?) scenario is that Apple gives up on the whole hashing system and then there will be zero chance of there ever being end-to-end encryption of photos in iCloud.
gorbachev 5y ago
For individual users the risk is very small, but when you have a billion users the chances of innocent people being caught up is pretty much 100%. Platform owners like Google, Apple, Twitter and Facebook should really keep stuff like that in mind when they deploy algorithmic solutions like this. Like dhosek said, the manual review step should reduce the risk quite a bit, though.
- Grustaf 5y ago
  They did keep it in mind, that's why they require 30 matches, which gets the false positive rate down to 1 in a trillion per year, so on average it will happen about 1 per millennium, given a billion users. So that's per photo library, not per photo. The rate per photo in their testing was 3 in 100 million, then they added a 30x safety margin and assumed it's actually 1 in a million. https://www.zdnet.com/article/apple-to-tune-csam-system-to-keep-one-in-a-trillion-false-positive-deactivation-threshold/ https://www.zdnet.com/article/apple-to-tune-csam-system-to-k...
  - deleted 5y ago
    [deleted]
JKCalhoun 5y ago
Not sure why the downvotes. You are absolutely correct that a threshold in the number of positives (false or otherwise) must be met. To be fair though we do not know what the threshold is. But I would guess even higher than 5 — I would presume 12 or more. I'm no criminologist (IANAC) but when you read about someone getting busted with child pornography they have hundreds or thousand of images — not one, not five. They're "trading cards" for these creeps.
- cyanite 5y ago
  Apple says they expect to use 30 as the threshold.
floatingatoll 5y ago
I saw 30 mentioned somewhere, and I think that’s a more likely guess than 5. When men have a porn collection, it rarely has less than a hundred images, and so the threshold for CSAM detections can be set quite high in quantity, and enjoy a near-zero false positive rate aside from malicious hackers. (And yes, 90% of CSAM abusers are men, so y’all will have to find some other way to weaken my argument.)

sandworm101 5y ago

For a normal random person, a grey man. In the real world, a single collision could be enough for the police to acquire further access to people they are already looking at. If the police want access to your phone for other reasons (drugs, taxes, illegal speech) they can use that one collision to get a warrant which will give them greater access. It is akin to cops wanting to search a car. They don't need a warrant. They only need to follow the car until it breaks any number of traffic laws. Then the resulting traffic stops lets them talk to the driver, who seems evasive, which gives them probable cause, which gets them a warrant to perform a full search. The accidental collision, week evidence of an offending image on the phone, opens the door to whatever other investigation they want to do.

Grustaf 5y ago
I'm not sure what you mean. If we are talking about Apple's new feature, they will not even be alerted until you have enough matches. And the police probably won't be using any neural hashes if they have access to your device, so I'm not sure what you are talking about.
- sandworm101 5y ago
  That is an Apple current policy, not any sort of law. If the FBI wants to know if there are any collisions on a particular phone, that will be shared. If the FBI wants to know of all single collisions, that too must be shared. A corporation's internal policy decisions are nothing when faced with a government official carrying a warrant. If they have data indicating possible crimes, no matter how small, they can be forced to divulge it to investigators. Unlike Lavabit, I don't think Apple will ever close up shop to protect customer data.
  - Grustaf 5y ago
    Well of course if you believe that FBI can ask Apple to share anything and they will agree, and not tell us, then what is even the point of this discussion? Then they already have access to every photo, every email, every text. Because Apple controls those apps on your phone.
    - fsflover 5y ago
      The difference is that now they confirmed that they do search your private images.
      - Grustaf 5y ago
        Why would that make it any likelier that they are also secretly searching your files, with another method? Seems completely orthogonal to me.
        fsflover 5y ago
        They showed that they are willing to do it in one case. Why not in other cases?
  - jasamer 5y ago
    I thought the system is designed such that the threshold must be met to check the matches, i.e. it’s not possible to follow the FBIs request without changing the entire system.
    - brokenmachine 5y ago
      int matchThreshold = 30; if (numMatches > matchThreshold) { snitch(customer) && customer.life.ruin() }
merpnderp 5y ago
Isn't Apple seeding their database with fake hits so that a court can't order them to turn over everyone who's just a few shy of Apple's threshold? So when this database leaks, won't there be a lot of people accused of horrific crimes simply because Apple seeded their database with random hits? Why would people pay a company to falsely accuse them of horrific crimes?
JKCalhoun 5y ago
Since, as I have read, Microsoft and Google already do this, where are all the illicit cop take-downs? I have not heard of any.
- sandworm101 5y ago
  They are scanning images on online accounts, stuff that is stored on their systems and system files on machines (files they own). They are not, from what I have read, scanning customer-owned images stored on customer-owned devices.
  - snowwrestler 5y ago
    Every AV (antivirus) software system scans customer-owned files on customer-owned devices. So the question is the same: if it’s easy for law enforcement to deputize such a system, where is the flood of cases built off of evidence gathered this way?
    - sandworm101 5y ago
      >> Every AV (antivirus) software system Mine doesn't. If Ubuntu or ClamAV is scanning all my photos and reporting the results to Canonical against my will then I will soon be having words with Mr. Linus.
      - snowwrestler 5y ago
        If law enforcement can force software companies to adapt their software to serve law enforcement purposes, why aren’t Ubuntu or ClamAV doing so already? What makes them better at resisting requests from law enforcement than Apple?
        MichaelGroves 5y ago
        Canonical and ClamAV aren't trying to curry favor with the US government to stave off antitrust action.

dhosek 5y ago

Apple has publicly stated 30. Plus, there would be a manual review of the images before forwarding to law enforcement. I don't see this collision attack leading to innocent people being handed over to the police. The worst-case (best-case?) scenario is that Apple gives up on the whole hashing system and then there will be zero chance of there ever being end-to-end encryption of photos in iCloud.

gorbachev 5y ago

For individual users the risk is very small, but when you have a billion users the chances of innocent people being caught up is pretty much 100%. Platform owners like Google, Apple, Twitter and Facebook should really keep stuff like that in mind when they deploy algorithmic solutions like this. Like dhosek said, the manual review step should reduce the risk quite a bit, though.

Grustaf 5y ago
They did keep it in mind, that's why they require 30 matches, which gets the false positive rate down to 1 in a trillion per year, so on average it will happen about 1 per millennium, given a billion users. So that's per photo library, not per photo. The rate per photo in their testing was 3 in 100 million, then they added a 30x safety margin and assumed it's actually 1 in a million. https://www.zdnet.com/article/apple-to-tune-csam-system-to-keep-one-in-a-trillion-false-positive-deactivation-threshold/ https://www.zdnet.com/article/apple-to-tune-csam-system-to-k...
- deleted 5y ago
  [deleted]

JKCalhoun 5y ago

Not sure why the downvotes. You are absolutely correct that a threshold in the number of positives (false or otherwise) must be met. To be fair though we do not know what the threshold is. But I would guess even higher than 5 — I would presume 12 or more. I'm no criminologist (IANAC) but when you read about someone getting busted with child pornography they have hundreds or thousand of images — not one, not five. They're "trading cards" for these creeps.

cyanite 5y ago
Apple says they expect to use 30 as the threshold.

floatingatoll 5y ago

I saw 30 mentioned somewhere, and I think that’s a more likely guess than 5. When men have a porn collection, it rarely has less than a hundred images, and so the threshold for CSAM detections can be set quite high in quantity, and enjoy a near-zero false positive rate aside from malicious hackers. (And yes, 90% of CSAM abusers are men, so y’all will have to find some other way to weaken my argument.)