5 ms·

Something to note here is that in the hash collision that was discovered, the two images look nothing alike. One is a picture of a dog, the other is blobby grey

Something to note here is that in the hash collision that was discovered, the two images look nothing alike. One is a picture of a dog, the other is blobby grey static.

smoldesu 5y ago
Furthermore, it may well be possible to combine that blobby grey static with another image, manipulating it's visual hash to create a "sleeper" positive. If this was possible in a week, then it's going to be very interesting to watch the technology change/evolve over the next few years.
- floatingatoll 5y ago
  Doing so would create a positive that still doesn't pass Apple's human visual check against the (blurred) CSAM content associated with that checksum, and if it somehow did, it would still then also have to occur at qty.30 or more, and they'd have to pass a human visual check against the (unblurred) CSAM content by one of the agencies in possession of it. It's not possible to spoof that final test unless you possess real CSAM content, at which point you don't need to spoof that final test and you'll end up arrested for possession.
  - deleted 5y ago
    [deleted]
  - laverya 5y ago
    And if you make the second image be actual, 18+ porn? Will Apple be able to tell the difference between that and CSAM after the blur is applied? Bonus points if you match poses, coloration, background, etc.
    - floatingatoll 5y ago
      The only way to match poses, coloration, background, etc is to possess illegal CSAM content. If you do so successfully, you will result in your crafted image passing the blur check and reaching the agency that possesses the original image for final verification, where it will immediately fail because it is obviously a replica. You will then trigger that agency leading law enforcement to find the creator of the image, so that they can arrest whoever possesses CSAM content in order to model replicas after it. I think that's a perfectly acceptable outcome, since anyone with the hubris to both possess CSAM content and create replicas of it especially deserves to be arrested. Do you see a more problematic outcome here?
      - laverya 5y ago
        This assumes that it's impossible to reverse the perceptual hashes used here in such a way that you could determine poses and coloration, for one. And in retrospect, you don't need to match that - you just need it to appear obviously pornographic after the blur is applied in order to get past Apple's reviewers. After that, the lucky individual's life is in the hands of the police/prosecutors. (I have to imagine that both real and faked cases will look pretty much like "Your honor/members of the jury, this person's device contained numerous photos matching known CSAM. No, we won't be showing you the pictures. No, the defence can't see them either." Can you imagine a "tough on crime" prosecutor taking the faked case to trial too? Would the police and prosecutors even know it was faked?)
        floatingatoll 5y ago
        Apple's reviewers are comparing blurred originals to blurred matches. It needs to look like the blurred original associated with the checksum that matched. It is irrelevant whether the blurred match looks pornographic or not.
        ultrarunner 5y ago
        This has already been demonstrated. https://twitter.com/SarahJamieLewis/status/1428083744280256512 https://twitter.com/SarahJamieLewis/status/14280837442802565...
        floatingatoll 5y ago
        At which point the image will be handed to the relevant CSAM group unblurred, who will do a visual comparison and find out immediately that it’s not a match, and then reject it without invoking law enforcement.
      - ultrarunner 5y ago
        Cops no-knock raid the wrong houses all the time; what makes you think that the procedure you outlined above will be followed without error? Especially against suspected child abusers, from an anti-child-abuse agency? What if the visual check gets accidentally signed off on, or even gets fraudulently marked as positive by a burned out/lazy/competent-but-distracted employee? There are just so many failure modes for this dragnet that all end with innocent people suffering a very difficult process, even if they don't eventually land in prison. I don't think it's unreasonable to not want to be volunteered to participate.
  - treesprite82 5y ago
    There are concerns without the images needing to make it all the way through their CSAM process. Apple is a US company with obligations to report certain crimes they become aware of, and NCMEC is a US government-tied organisation that dismissed privacy concerns as "the screeching voices of the minority". Consider if the honeypot images (manipulated to match CSAM hashes) are terrorist recruitment material for example.
  - int_19h 5y ago
    At the point where it is submitted for a human visual check, your privacy has already been violated.
    - floatingatoll 5y ago
      My privacy is violated every time I leave my home. Anyone can take a photo of me and look up the FBI Most Wanted and see if I’m there. If they think someone is me and they’re wrong, they can still summon law enforcement, and I’ll still be mistreated for their poor judgement. Is this just as unacceptable as the CSAM scanning? Should all public photography be banned, in order to reduce the risk of false positive identifications of innocent people as criminals to zero? Or is that an acceptable degree of privacy impingement for the good of society? Is Apple’s implementation an acceptable trade of impingement and risk, for good for society? We do live in a society, and so zero impingement upon privacies is never going to be acceptable (sorry, free/tech/libs). But instead of discussing whether Apple’s approach violates privacy minimally or not in order to get the job done, these discussions here just keep circling the drain of “putting my privacy at risk by any degree is never acceptable”, when that drain is cemented shut by the existence of society and will never lead to a valid outcome.
      - int_19h 5y ago
        By leaving your home and going out in public, you are implicitly consenting to being observed. So, no, that's not a violation of your privacy. Apple's implementation is more like a stranger sneaking into your house if they suspect you're up to something bad. That would be equally unacceptable - we have warrants for a reason.
bawolff 5y ago
I dont really think it'd be a valid second pre-image for this type of hash if they did look similar.
__blockcipher__ 5y ago
They actually do look alike to my eye, but in a “the way the algorithm sees it” kind of way. I can see the obvious similarity. But to your point it’s not like it’s two very slightly different photos of dogs.
- TechBro8615 5y ago
  You must be a champion CAPTCHA solver.
0x5f3759df-i 5y ago
Take a look at this collision https://twitter.com/SarahJamieLewis/status/1428083744280256512 https://twitter.com/SarahJamieLewis/status/14280837442802565...
- legostormtroopr 5y ago
  Those are two almost identical images based off each other. I know that that "think of the children" is a meme, but I think this illustrates the point for apple. If you have a croped or modified image of CSA the system will identify it. As long as your image is different enough from CSA, you are safe. The point here is that Apple is specifically looking for matches against known CSA material. If someone can demonstrate that a legal NSFW image (eg. regular old-fashioned pornography), can be collided with a legal, completely 100% SFW image then I'll be concerned. But until then, this looks like a reasonable and supportable way for finding CSAM in real time.
  - hartator 5y ago
    > If someone can demonstrate that a legal NSFW image (eg. regular old-fashioned pornography), can be collided with a legal, completely 100% SFW image then I'll be concerned. Look at this other collision: https://twitter.com/SarahJamieLewis/status/1428206088118149123 https://twitter.com/SarahJamieLewis/status/14282060881181491... An attacker can send you an innocent looking picture that embbed some CSA material and you get swatted the next day.
    - legostormtroopr 5y ago
      I think you are getting downvoted because thats the same image I replied to. Looking at those two images its plain to see why they are identical. And if someone is sending you CSA material, isn't that the point of this process. Apple identifies it as CSA, can give you a warning its sensitive, and identify they authorities that people are sending CSA. Again - this seems like a win. If Apple can automatically identify CSA material, en-masse thats good. edit: It looks like they are different URLs, but Twitter only allows people to see replies if they are logged in, so I can't see that example. edit 2: On further thought, if someone can use CSA material and produce an innocuous image with a similar hash, if they send that image to you, its still proof that the sender had CSA material. Again, its still good.
kbelder 5y ago
I'd be more interested in whether you could take an image of, say, an underage dog, and tweak a vaguely similar image of a grown dog so that it was a match. That's what will get innocent people thrown in jail.

Something to note here is that in the hash collision that was discovered, the two images look nothing alike. One is a picture of a dog, the other is blobby grey static.

smoldesu 5y ago

Furthermore, it may well be possible to combine that blobby grey static with another image, manipulating it's visual hash to create a "sleeper" positive. If this was possible in a week, then it's going to be very interesting to watch the technology change/evolve over the next few years.

floatingatoll 5y ago
Doing so would create a positive that still doesn't pass Apple's human visual check against the (blurred) CSAM content associated with that checksum, and if it somehow did, it would still then also have to occur at qty.30 or more, and they'd have to pass a human visual check against the (unblurred) CSAM content by one of the agencies in possession of it. It's not possible to spoof that final test unless you possess real CSAM content, at which point you don't need to spoof that final test and you'll end up arrested for possession.
- deleted 5y ago
  [deleted]
- laverya 5y ago
  And if you make the second image be actual, 18+ porn? Will Apple be able to tell the difference between that and CSAM after the blur is applied? Bonus points if you match poses, coloration, background, etc.
  - floatingatoll 5y ago
    The only way to match poses, coloration, background, etc is to possess illegal CSAM content. If you do so successfully, you will result in your crafted image passing the blur check and reaching the agency that possesses the original image for final verification, where it will immediately fail because it is obviously a replica. You will then trigger that agency leading law enforcement to find the creator of the image, so that they can arrest whoever possesses CSAM content in order to model replicas after it. I think that's a perfectly acceptable outcome, since anyone with the hubris to both possess CSAM content and create replicas of it especially deserves to be arrested. Do you see a more problematic outcome here?
    - laverya 5y ago
      This assumes that it's impossible to reverse the perceptual hashes used here in such a way that you could determine poses and coloration, for one. And in retrospect, you don't need to match that - you just need it to appear obviously pornographic after the blur is applied in order to get past Apple's reviewers. After that, the lucky individual's life is in the hands of the police/prosecutors. (I have to imagine that both real and faked cases will look pretty much like "Your honor/members of the jury, this person's device contained numerous photos matching known CSAM. No, we won't be showing you the pictures. No, the defence can't see them either." Can you imagine a "tough on crime" prosecutor taking the faked case to trial too? Would the police and prosecutors even know it was faked?)
      - floatingatoll 5y ago
        Apple's reviewers are comparing blurred originals to blurred matches. It needs to look like the blurred original associated with the checksum that matched. It is irrelevant whether the blurred match looks pornographic or not.
        ultrarunner 5y ago
        This has already been demonstrated. https://twitter.com/SarahJamieLewis/status/1428083744280256512 https://twitter.com/SarahJamieLewis/status/14280837442802565...
        floatingatoll 5y ago
        At which point the image will be handed to the relevant CSAM group unblurred, who will do a visual comparison and find out immediately that it’s not a match, and then reject it without invoking law enforcement.
    - ultrarunner 5y ago
      Cops no-knock raid the wrong houses all the time; what makes you think that the procedure you outlined above will be followed without error? Especially against suspected child abusers, from an anti-child-abuse agency? What if the visual check gets accidentally signed off on, or even gets fraudulently marked as positive by a burned out/lazy/competent-but-distracted employee? There are just so many failure modes for this dragnet that all end with innocent people suffering a very difficult process, even if they don't eventually land in prison. I don't think it's unreasonable to not want to be volunteered to participate.
- treesprite82 5y ago
  There are concerns without the images needing to make it all the way through their CSAM process. Apple is a US company with obligations to report certain crimes they become aware of, and NCMEC is a US government-tied organisation that dismissed privacy concerns as "the screeching voices of the minority". Consider if the honeypot images (manipulated to match CSAM hashes) are terrorist recruitment material for example.
- int_19h 5y ago
  At the point where it is submitted for a human visual check, your privacy has already been violated.
  - floatingatoll 5y ago
    My privacy is violated every time I leave my home. Anyone can take a photo of me and look up the FBI Most Wanted and see if I’m there. If they think someone is me and they’re wrong, they can still summon law enforcement, and I’ll still be mistreated for their poor judgement. Is this just as unacceptable as the CSAM scanning? Should all public photography be banned, in order to reduce the risk of false positive identifications of innocent people as criminals to zero? Or is that an acceptable degree of privacy impingement for the good of society? Is Apple’s implementation an acceptable trade of impingement and risk, for good for society? We do live in a society, and so zero impingement upon privacies is never going to be acceptable (sorry, free/tech/libs). But instead of discussing whether Apple’s approach violates privacy minimally or not in order to get the job done, these discussions here just keep circling the drain of “putting my privacy at risk by any degree is never acceptable”, when that drain is cemented shut by the existence of society and will never lead to a valid outcome.
    - int_19h 5y ago
      By leaving your home and going out in public, you are implicitly consenting to being observed. So, no, that's not a violation of your privacy. Apple's implementation is more like a stranger sneaking into your house if they suspect you're up to something bad. That would be equally unacceptable - we have warrants for a reason.

bawolff 5y ago

I dont really think it'd be a valid second pre-image for this type of hash if they did look similar.

__blockcipher__ 5y ago

They actually do look alike to my eye, but in a “the way the algorithm sees it” kind of way. I can see the obvious similarity. But to your point it’s not like it’s two very slightly different photos of dogs.

TechBro8615 5y ago
You must be a champion CAPTCHA solver.

0x5f3759df-i 5y ago

Take a look at this collision https://twitter.com/SarahJamieLewis/status/1428083744280256512 https://twitter.com/SarahJamieLewis/status/14280837442802565...

legostormtroopr 5y ago
Those are two almost identical images based off each other. I know that that "think of the children" is a meme, but I think this illustrates the point for apple. If you have a croped or modified image of CSA the system will identify it. As long as your image is different enough from CSA, you are safe. The point here is that Apple is specifically looking for matches against known CSA material. If someone can demonstrate that a legal NSFW image (eg. regular old-fashioned pornography), can be collided with a legal, completely 100% SFW image then I'll be concerned. But until then, this looks like a reasonable and supportable way for finding CSAM in real time.
- hartator 5y ago
  > If someone can demonstrate that a legal NSFW image (eg. regular old-fashioned pornography), can be collided with a legal, completely 100% SFW image then I'll be concerned. Look at this other collision: https://twitter.com/SarahJamieLewis/status/1428206088118149123 https://twitter.com/SarahJamieLewis/status/14282060881181491... An attacker can send you an innocent looking picture that embbed some CSA material and you get swatted the next day.
  - legostormtroopr 5y ago
    I think you are getting downvoted because thats the same image I replied to. Looking at those two images its plain to see why they are identical. And if someone is sending you CSA material, isn't that the point of this process. Apple identifies it as CSA, can give you a warning its sensitive, and identify they authorities that people are sending CSA. Again - this seems like a win. If Apple can automatically identify CSA material, en-masse thats good. edit: It looks like they are different URLs, but Twitter only allows people to see replies if they are logged in, so I can't see that example. edit 2: On further thought, if someone can use CSA material and produce an innocuous image with a similar hash, if they send that image to you, its still proof that the sender had CSA material. Again, its still good.

kbelder 5y ago

I'd be more interested in whether you could take an image of, say, an underage dog, and tweak a vaguely similar image of a grown dog so that it was a match. That's what will get innocent people thrown in jail.