8 ms·

Unless those pictures are also in the NCMEC database, there won’t be a match.* * As addressed in the comments below, this isn’t entirely true: the hash looks f

Unless those pictures are also in the NCMEC database, there won’t be a match.* * As addressed in the comments below, this isn’t entirely true: the hash looks for visually similar picture and there may be false positives.

gambiting 5y ago
Absolutely not true. Apple is using a similarity based hash, so if the NCMEC database contains a picture that's similar to one that you have, it could produce a match even if it's not the same. Apple says this isn't an issue, because a person will look at your picture(yes, a random person somewhere will look at the pictures of your newborn) and judge whether they are pictures of child abuse or not. If this unknown person thinks your picture shows child abuse, you will be reported to NCMEC and then what happens is unknown - but likely that it would result in some legal action against you.
- lawkwok 5y ago
  Keep in mind, this manual review only happens after Apple’s system detects multiple occurrences of matches. Until that point, no human is alerted of matches nor does anyone see how many matches there have been. In a TechCrunch interview Apple said that they are going after larger targets that are worth NCMEC’s time.
  - zionic 5y ago
    Parents take a lot of photos of their kid. Like, lots.
    - simondotau 5y ago
      How many of them include erect adult penises and active participation in sex acts? Apple's on-device list of hashes only includes images which have been classified "A1" under the CSAM categorisation scale. If any other photographs are accidental hash collisions to these images, it's going to be pretty damn obvious to the human reviewer.
      - gambiting 5y ago
        The problem with all of this, is that it's about to trust. We're are supposed to trust apple's algorithm to avoid false positives, we're supposed to trust apple that even with false positives there's some threshold to cross, and then supposed to trust apple that their employees will do a good job verifying the pictures, and then(most imporantly) trust that a giant american corporation won't honor secretive state requests to start scanning our phones for other data. I just don't have that trust. Obviously I think that protecting children is an incredibly important thing to be doing, but I don't trust apple to be running such a system(I do trust them maybe a tiny bit more than I'd trust google in this case, but ultimately I'd rather this system didn't exist at all).
        simondotau 5y ago
        I trust that Apple knows that an actual, real world false accusation will make this week's media challenges look like a fleabite.
  - jamesu 5y ago
    "Multiple occurrences of matches" could have definitely been an issue for a friend of mine. When they took a picture, they'd often go for the "blitz the subject with the camera" approach, then never ended up deleting all the bad pictures because they had hoarding tendencies.
- Spooky23 5y ago
  Where’s your evidence on this? The NCMEC database and this hashing have been around for like 15 years. I’m curious as to how you know this.
  - gambiting 5y ago
    Literally Apple said in their own FAQ that they are using a perceptual(similarity based) hash and that their employees will review images when flagged. If that's not good enough(somehow) then even the New York Times article about it says the same thing. What other evidence do you need?
  - mattigames 5y ago
    False positives have been found, not because the photo hold any similarities but because the hashes match: https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
    - Spooky23 5y ago
      Ok. Who has been arrested for a picture of a tomato that generated a hash collision?
- vxNsr 5y ago
  > Apple is using a similarity based hash, so if the NCMEC database contains a picture that's similar to one that you have, it could produce a match even if it's not the same It's actually worse than this, if the hashes are similar then they'll get sent for review. Your picture could be a picture of an abstract painting[0] which has no visual similarity to anything in the db, but through the magic of crypto is similar and it too will be flagged. [0] The reason I use this example is because someone posted a bunch of abstract art that was flagged by the algo.
- samename 5y ago
  Good point, thanks, updated my comment.
- FabHK 5y ago
  > because a person will look at your picture(yes, a random person somewhere will look at the pictures of your newborn) No. If the number of matches to known CSAM in your library exceeds a threshold, then a person will look at a "visual derivative" of only those pictures whose perceptual hatch match that of known CSAM. Note that, if I understand correctly, pictures that Android users sync to Google have already been scanned for some time. Where are all those false positives?
  - gambiting 5y ago
    >>the number of matches to known CSAM in your library exceeds a threshold, then a person will look at a "visual derivative" of only those pictures whose perceptual hatch match that of known CSAM Apple very specifically said that their employees will look at the suspected picture before sending it through to authorities. Where do you see the bit about visual derivatives? What would that even mean or look like? Also what is this threshold? As others have pointed out, us parents have literally hundreds of pictures of our newborns, toddlers and kids - having to trigger some detector "multiple" times doesn't give me any peace of mind at all. >>Where are all those false positives? Google doesn't use perceptual hashing, or at least haven't said they do.
    - FabHK 5y ago
      > Where do you see the bit about visual derivatives? In Apple's white paper about the proposed feature: https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... See the links at the bottom here for more: https://www.apple.com/child-safety/ https://www.apple.com/child-safety/ > Also what is this threshold? The "perceptual hash" is supposed to match a specific image (though possibly cropped, or otherwise altered a bit, such as through a filter), not "toddlers" per se. > Google doesn't use perceptual hashing, or at least haven't said they do. I don't know what the other cloud providers are doing, but I'd be very surprised if they use (trivially circumventable) cryptographic hashes.
    - deleted 5y ago
      [deleted]
josefx 5y ago
As far as I understand they use some kind of hash. I suspect their paper on avoiding hash collisions is right next to the Nobel price wining description of the worlds first working perpetuum mobile.
- potatoman22 5y ago
  https://en.wikipedia.org/wiki/Perceptual_hashing https://en.wikipedia.org/wiki/Perceptual_hashing
- FabHK 5y ago
  They have accounted for the possibility of false positives.
Dah00n 5y ago
That is not true. If you read what experts who actually do something to stop CP (unlike Apple) say there are proven false positives. https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
- samename 5y ago
  Thanks, I updated my comment.
- FabHK 5y ago
  Yes, and you can put a number on the probability of that happening, say a fixed p << 1. And then you can choose the number of required matches before flagging, say N. And then (assuming independence [1]) you have an overall probability of p^N, which you can make arbitrarily small by making N sufficiently large. (I'm pretty sure that's how Apple came up with their "1 in a trillion chance per year".) And then you still have manual review. [1] you could "help" independence by requiring a certain distance between images you simultaneously flag.

gambiting 5y ago

Absolutely not true. Apple is using a similarity based hash, so if the NCMEC database contains a picture that's similar to one that you have, it could produce a match even if it's not the same. Apple says this isn't an issue, because a person will look at your picture(yes, a random person somewhere will look at the pictures of your newborn) and judge whether they are pictures of child abuse or not. If this unknown person thinks your picture shows child abuse, you will be reported to NCMEC and then what happens is unknown - but likely that it would result in some legal action against you.

lawkwok 5y ago
Keep in mind, this manual review only happens after Apple’s system detects multiple occurrences of matches. Until that point, no human is alerted of matches nor does anyone see how many matches there have been. In a TechCrunch interview Apple said that they are going after larger targets that are worth NCMEC’s time.
- zionic 5y ago
  Parents take a lot of photos of their kid. Like, lots.
  - simondotau 5y ago
    How many of them include erect adult penises and active participation in sex acts? Apple's on-device list of hashes only includes images which have been classified "A1" under the CSAM categorisation scale. If any other photographs are accidental hash collisions to these images, it's going to be pretty damn obvious to the human reviewer.
    - gambiting 5y ago
      The problem with all of this, is that it's about to trust. We're are supposed to trust apple's algorithm to avoid false positives, we're supposed to trust apple that even with false positives there's some threshold to cross, and then supposed to trust apple that their employees will do a good job verifying the pictures, and then(most imporantly) trust that a giant american corporation won't honor secretive state requests to start scanning our phones for other data. I just don't have that trust. Obviously I think that protecting children is an incredibly important thing to be doing, but I don't trust apple to be running such a system(I do trust them maybe a tiny bit more than I'd trust google in this case, but ultimately I'd rather this system didn't exist at all).
      - simondotau 5y ago
        I trust that Apple knows that an actual, real world false accusation will make this week's media challenges look like a fleabite.
- jamesu 5y ago
  "Multiple occurrences of matches" could have definitely been an issue for a friend of mine. When they took a picture, they'd often go for the "blitz the subject with the camera" approach, then never ended up deleting all the bad pictures because they had hoarding tendencies.
Spooky23 5y ago
Where’s your evidence on this? The NCMEC database and this hashing have been around for like 15 years. I’m curious as to how you know this.
- gambiting 5y ago
  Literally Apple said in their own FAQ that they are using a perceptual(similarity based) hash and that their employees will review images when flagged. If that's not good enough(somehow) then even the New York Times article about it says the same thing. What other evidence do you need?
- mattigames 5y ago
  False positives have been found, not because the photo hold any similarities but because the hashes match: https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
  - Spooky23 5y ago
    Ok. Who has been arrested for a picture of a tomato that generated a hash collision?
vxNsr 5y ago
> Apple is using a similarity based hash, so if the NCMEC database contains a picture that's similar to one that you have, it could produce a match even if it's not the same It's actually worse than this, if the hashes are similar then they'll get sent for review. Your picture could be a picture of an abstract painting[0] which has no visual similarity to anything in the db, but through the magic of crypto is similar and it too will be flagged. [0] The reason I use this example is because someone posted a bunch of abstract art that was flagged by the algo.
samename 5y ago
Good point, thanks, updated my comment.
FabHK 5y ago
> because a person will look at your picture(yes, a random person somewhere will look at the pictures of your newborn) No. If the number of matches to known CSAM in your library exceeds a threshold, then a person will look at a "visual derivative" of only those pictures whose perceptual hatch match that of known CSAM. Note that, if I understand correctly, pictures that Android users sync to Google have already been scanned for some time. Where are all those false positives?
- gambiting 5y ago
  >>the number of matches to known CSAM in your library exceeds a threshold, then a person will look at a "visual derivative" of only those pictures whose perceptual hatch match that of known CSAM Apple very specifically said that their employees will look at the suspected picture before sending it through to authorities. Where do you see the bit about visual derivatives? What would that even mean or look like? Also what is this threshold? As others have pointed out, us parents have literally hundreds of pictures of our newborns, toddlers and kids - having to trigger some detector "multiple" times doesn't give me any peace of mind at all. >>Where are all those false positives? Google doesn't use perceptual hashing, or at least haven't said they do.
  - FabHK 5y ago
    > Where do you see the bit about visual derivatives? In Apple's white paper about the proposed feature: https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... See the links at the bottom here for more: https://www.apple.com/child-safety/ https://www.apple.com/child-safety/ > Also what is this threshold? The "perceptual hash" is supposed to match a specific image (though possibly cropped, or otherwise altered a bit, such as through a filter), not "toddlers" per se. > Google doesn't use perceptual hashing, or at least haven't said they do. I don't know what the other cloud providers are doing, but I'd be very surprised if they use (trivially circumventable) cryptographic hashes.
  - deleted 5y ago
    [deleted]

josefx 5y ago

As far as I understand they use some kind of hash. I suspect their paper on avoiding hash collisions is right next to the Nobel price wining description of the worlds first working perpetuum mobile.

potatoman22 5y ago
https://en.wikipedia.org/wiki/Perceptual_hashing https://en.wikipedia.org/wiki/Perceptual_hashing
FabHK 5y ago
They have accounted for the possibility of false positives.

Dah00n 5y ago

That is not true. If you read what experts who actually do something to stop CP (unlike Apple) say there are proven false positives. https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...

samename 5y ago
Thanks, I updated my comment.
FabHK 5y ago
Yes, and you can put a number on the probability of that happening, say a fixed p << 1. And then you can choose the number of required matches before flagging, say N. And then (assuming independence [1]) you have an overall probability of p^N, which you can make arbitrarily small by making N sufficiently large. (I'm pretty sure that's how Apple came up with their "1 in a trillion chance per year".) And then you still have manual review. [1] you could "help" independence by requiring a certain distance between images you simultaneously flag.