31 ms·

The Problem with Perceptual Hashes

• asimpletune 5y ago
  “ Even at a Hamming Distance threshold of 0, that is, when both hashes are identical, I don’t see how Apple can avoid tons of collisions, given the large number of pictures taken every year (1.4 trillion in 2021, now break this down by iPhone market share and country, the number for US iPhone users will still be extremely big).” Is this true? I’d imagine you could generate billions a second without having a collision, although I don’t know much about how these hashes are produced. It would be cool for an expert to weigh in here.
• klodolph 5y ago
  > Even at a Hamming Distance threshold of 0, that is, when both hashes are identical, I don’t see how Apple can avoid tons of collisions... You'd want to look at the particular perceptual hash implementation. There is no reason to expect, without knowing the hash function, that you would end up with tons of collisions at distance 0.
  • mirker 5y ago
    If images have cardinality N and hashes M and N > M, then yes, by pigeonhole principle you will have collisions regardless of hash function, f: N -> M. N is usually much bigger than M, since you have the combinatorial pixel explosion. Say images are 8 bit RGB 256x256, then you have 2^(8x256x256x3) bit combinations. If you have a 256-bit hash, then that’s only 2^256. So there is a factor of 2^(8x256x3) difference between N and M if I did my math right, which is a factor I cannot even calculate without numeric overflow.
    • klodolph 5y ago
      The number of possible different images doesn't matter, it's only the number of actually different images encountered in the world. This number cannot be anywhere near 2^256, that would be physically impossible.
      • mirker 5y ago
        But you cannot know that a-priori so it’s either an attack vector for image manipulation or straight up false positives. Assume we had this perfect hash knowledge. I’d create a compression algorithm to uniquely map between images and the 256 bit hash space, which we probably agree is similarly improbable. It’s on the order of 1000x to 10000x more efficient than JPEG and isn’t even lossy.
        • klodolph 5y ago
          You’re going to have to explain that—what is an attack vector for image manipulation? What is an attack vector for false positives? > Assume we had this perfect hash knowledge. It’s not a perfect hash. Nobody’s saying it’s a perfect hash. It’s not. It’s a perceptual hash. It is specifically designed to map similar images to similar hashes, for the “right” notion of similar.
          • mirker 5y ago
            It’s an attack vector because, either directly or indirectly, people will get access to the hash function. Therefore, they will have a testbed to launch attacks against the hash. Once they learn an attack, they will hash your harmless pictures into being classified as CP by applying a potentially perceptually invisible modification to your images. Imagine every image on reddit’s frontpage being classified as CP. The distribution of natural images is irrelevant as long as people can modify images arbitrarily. The hashing isn’t really so relevant apart from pigeonhole arguments. It’s a machine learning problem of classification between CP and not, and hashing is an implementation detail. The way I would attack this reading a few papers would be to approximate any non-differentiable parts of the hashing with a smooth proxy function, then use an off-the-shelf gradient-based attack such as Fast Sign Gradient Method. The hashing guarantees that even at hashing distance 0, you have a huge amount of collisions, so that is blind spot 1. Blind spot 2 is the CNN is not-robust to mild-perturbations, so you can “squeeze” inputs together in hash space by modifying them. You can likely attack both simultaneously by doing the attack I said above.
            • 5y ago
• jbmsf 5y ago
  I am fairly ignorant if this space. Do any of the standard methods use multiple hash functions vs just one?
  • heavyset_go 5y ago
    I've built products that utilize different phash algorithms at once, and it's entirely possible, and quite common, to get false positives across hashing algorithms.
  • jdavis703 5y ago
    Yes, I worked on such a product. Users had several hashing algorithms they could chose from, and the ability to create custom ones if they wanted.
• lordnacho 5y ago
  Why wouldn't the algo check that one image has a face while the other doesn't? That would remove this particular false positive, though I'm not sure what it might cause of new ones.
  • PUSH_AX 5y ago
    Because where do you draw the line with classifying arbitrary features in the images? The concept is it should work with an image of anything.
• ivalm 5y ago
  I am not exactly buying the premise here, if you train a CNN on useful semantic categories then the representations they generate will be semantically meaningful (so the error shown in blog wouldn’t occur). I dislike the general idea of iCloud having back doors but I don’t think the criticism in this blog is entirely valid. Edit: it was pointed out apple doesn’t have semantically meaningful classifier so the blog post’s criticism is valid.
  • jeffbee 5y ago
    I agree the article is a straw-man argument and is not addressing the system that Apple actually describes.
  • SpicyLemonZest 5y ago
    Apple's description of the training process (https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...) sounds like they're just training it to recognize some representative perturbations, not useful semantic categories.
    • ivalm 5y ago
      Ok, good point, thanks.
• at_a_remove 5y ago
  I do not know as much about perceptual hashing as I would like, but have considered it for a little project of my own. Still, I know it has been floating around in the wild. I recently came across it on Discord when I attempted to push an ancient image, from the 4chan of old, to a friend, which mysteriously wouldn't send. Saved it as a PNG, no dice. This got me interested. I stripped the EXIF data off of the original JPEG. I resized it slightly. I trimmed some edges. I adjusted colors. I did a one degree rotation. Only after a reasonably complete combination of those factors would the image make it through. How interesting! I just don't know how well this little venture of Apple's will scale, and I wonder if it won't even up being easy enough to bypass in a variety of ways. I think the tradeoff will do very little, as stated, but is probably a glorious apportunity for black-suited goons of state agencies across the globe. We're going to find out in a big big way soon. * The image is of the back half of a Sphynx cat atop a CRT. From the angle of the dangle, the presumably cold, man-made feline is draping his unexpectedly large testicles across the similarly man-made device to warm them, suggesting that people create problems and also their solutions, or that, in the Gibsonian sense, the street finds its own uses for things. I assume that the image was blacklisted, although I will allow for the somewhat baffling concept of a highly-specialized scrotal matching neural-net that overreached a bit or a byte on species, genus, family, and order.
  • judge2020 5y ago
    AFAIK Discord's NSFW filter is not a perceptual hash nor uses the NCMEC database (although that might indeed be in the pipeline elsewhere) but instead uses a ML classifier (I'm certain it doesn't use perceptual hashes as Discord doesn't have a catalogue of NSFW image hashes to compare against). I've guessed it's either open_nsfw[0] or Google's Cloud Vision since the rest of Discord's infrastructure uses Google Cloud VMs. There's a web demo available of this api[1], Discord probably pulls the safe search classifications for determining NSFW. 0: https://github.com/yahoo/open_nsfw https://github.com/yahoo/open_nsfw 1: https://cloud.google.com/vision#section-2 https://cloud.google.com/vision#section-2
  • a_t48 5y ago
    Adding your friend as a "friend" on discord should disable the filter.
• ttul 5y ago
  Apple would not be so naive as to roll out a solution to child abuse images that has a high false positive rate. They do test things prior to release…
  • celeritascelery 5y ago
    Test it… how exactly? This is detecting illegal material that they can’t use to test against.
    • ben_w 5y ago
      While I don’t have any inside knowledge at all, I would expect a company as big as Apple to be able to ask law enforcement to run Apple’s algorithm on data sets Apple themselves don’t have access to and report the result. No idea if they did (or will), but I do expect it’s possible.
      • zimpenfish 5y ago
        > ask law enforcement to run Apple’s algorithm on data sets Apple themselves don’t have access to Sounds like that's what they did since they say they're matching against hashes provided by NCMEC generated from their 200k CSAM corpus. [edit: Ah, in the PDF someone else linked, "First, Apple receives the NeuralHashes corresponding to known CSAM from the above child-safety organizations."]
    • bryanrasmussen 5y ago
      Not knowing anything about it but I suppose various governmental agencies maintain corpora of nasty stuff and that you can say to them - hey we want to roll out anti-nasty stuff functionality in our service therefore we need access to corpora to test at which point there is probably a pretty involved process that requires governmental access also to make sure things work and are not misused otherwise - how does anyone ever actually fight the nasty stuff? This problem structure of how do I catch examples of A if examples of A are illegal must apply in many places and ways.
      • 5y ago
• stickfigure 5y ago
  I've also implemented perceptual hashing algorithms for use in the real world. Article is correct, there really is no way to eliminate false positives while still catching minor changes (say, resizing, cropping, or watermarking). I'm sure I'm not the only person with naked pictures of my wife. Do you really want a false positive to result in your intimate moments getting shared around some outsourced boiler room for laughs?
  • mjlee 5y ago
    > I'm sure I'm not the only person with naked pictures of my wife. I'm not completely convinced that says what you want it to.
    • enedil 5y ago
      Didn't she possibly have previous partners?
      • websites2023 5y ago
        Presumably she wasn’t his wife then. But also people have various arrangements so I’m not here to shame.
      • iratewizard 5y ago
        I don't even have nude photos of my wife. The only person who might would be the NSA contractor assigned to watch her.
        • x-shadowban 5y ago
          what function does the word "even" perform in this sentence?
          • iratewizard 5y ago
            It's used to emphasize the concept that if anyone would have nudes of my wife, it would be me, her husband. Here's another example of "even" used as an emphasizing word. >I don't know how to answer that. >Even I don't know how to answer that. Hope that helps you with your ESL tests!
• rustybolt 5y ago
  > an Apple employee will then look at your (flagged) pictures. This means that there will be people paid to look at child pornography and probably a lot of private nude pictures as well.
  • varjag 5y ago
    There are people who are paid to do that already, just generally not in corporate employment.
  • pkulak 5y ago
    Apple, with all those Apple == Privacy billboards plastered everywhere, is going to have a full-time staff of people with the job of looking through it's customers' private photos.
    • arvinsim 5y ago
      Sue them for false marketing.
  • emodendroket 5y ago
    And what do you think the content moderation teams employed by Facebook, YouTube, et al. do all day?
    • deleted 5y ago
      [deleted]
    • mattigames 5y ago
      Yeah, we obviously needed one more company doing it as well, and I'm sure having more positions in the job market which pretty much could be described as "Get paid to watch pedophilia all day long" will not backfire in any way.
      • emodendroket 5y ago
        You could say there are harmful effects of these jobs but probably not in the sense you're thinking. https://www.wired.com/2014/10/content-moderation/ https://www.wired.com/2014/10/content-moderation/
• karmakaze 5y ago
  It really all comes down to if Apple has and is willing to maintain the effort of human evaluations prior to taking action on the potentially false positives: > According to Apple, a low number of positives (false or not) will not trigger an account to be flagged. But again, at these numbers, I believe you will still get too many situations where an account has multiple photos triggered as a false positive. (Apple says that probability is “1 in 1 trillion” but it is unclear how they arrived at such an estimate.) These cases will be manually reviewed. At scale, even human classification which ought to be clear will fail, accidentally clicking 'not ok' when they saw something they thought was 'ok'. It will be interesting to see what happens then.
  • jdavis703 5y ago
    Then law enforcement, a prosecutor and a jury would get involved. Hopefully law enforcement would be the first and final stage if it was merely the case that a person pressed “ok” by accident.
    • karmakaze 5y ago
      This is exactly the kind of thing that is to be avoided: premature escalation, tying up resources, increasing costs, and raising the stakes and probability of bad outcomes.
    • gtyras2mrs 5y ago
      Do you think once you are charged with possessing child porn - will you still have your job, your friends, your family, your life as you know it? Will a court decision - months or years later - restore what you have lost?
• deleted 5y ago
  [deleted]
• IfOnlyYouKnew 5y ago
  Apple’s documents said they require multiple hits before anything happens, as the article notes. They can (and have) adjusted that number to any desired balance of false positive to negatives. How can they say it’s 1 in a trillion? You test the algorithm on a bunch of random negatives, see how many positives you get, and do one division and one multiplication. This isn’t rocket science. So, while there are many arguments against this program, this isn’t it. It’s also somewhat strange to believe the idea of collisions in hashes of far smaller size than the images they are run on somehow escaped Apple and/or really anyone mildly competent.
  • bt1a 5y ago
    That would not be a good way to arrive at an accurate estimate. Would you not need dozens of trillions of photos to begin with in order to get an accurate estimate when the occurrence rate is so small?
    • KarlKemp 5y ago
      What? No... Or, more accurately: if you need "dozens of trillions" that implies a false positive rate so low, it's practically of no concern. You'd want to look up the poisson distribution for this. But, to get at this intuitively: say you have a bunch of eggs, some of which may be spoiled. How many would you have to crack open, to get a meaningful idea of how many are still fine, and how many are not? The absolute number depends on the fraction that are off. But independent of that, you'd usually start trusting your sample when you've seen 5 to 10 spoiled ones. So Apple runs the hash algorithm on random photos. They find 20 false positives in the first ten million. Given that error rate, how many positives would it require for the average photo collection of 10,000 to be certain at at 1:a trillion level that it's not just coincidence? Throw it into, for example, https://keisan.casio.com/exec/system/1180573179 https://keisan.casio.com/exec/system/1180573179 with lambda = 0.2 (you're expecting one false positive for every 50,000 at the error rate we assumed, or 0.2 for 10,000), and n = 10 (we've found 10 positives in this photo library) to see the chances of that, 2.35x10^-14, or 2.35 / 100 trillion.
  • fogof 5y ago
    I was unhappy to find this comment so far down and even unhappier to see it downvoted. I'm not a fan of the decrease in privacy Apple is creating with this move but I think this forum has gotten its feelings for Apple caught up with its response to a completely valid criticism of an anti-Apple article. To explain things even further, let's say that the perceptual algorithm makes a false positive 1% of the time. That is, 1 in every 100 completely normal pictures are incorrectly matched with some picture in the child pornography database. There's no reason to think (at least none springs to mind, happy to hear suggestions) that a false positive in one image will make it any more likely to see a false positive in another image. Thus, if you have a phone with 1000 pictures on it, and it takes 40 trigger a match, there's less than a 1 in a trillion probability that this would happen if the pictures are all normal.
• marcinzm 5y ago
  > an Apple employee will then look at your (flagged) pictures. Always fun when unknown strangers get to look at your potentially sensitive photos with probably no notice given to you.
  • judge2020 5y ago
    They already do this for photodna-matched iCloud Photos (and Google Photos, Flickr, Imgur, etc), perceptual hashes do not change that.
    • version_five 5y ago
      I'm not familiar with iPhone picture storage. Are the pictures automatically sync'ed with cloud storage? I would assume (even if I don't like it) that cloud providers may be scanning my data. But I would not expect anyone to be able to see or scan what is stored on my phone. Incidentally, I work in computer vision and handle proprietary images. I would be violating client agreements if I let anyone else have access to them. This is a concern I've had in the past e.g. with Office365 (the gold standard in disregarding privacy) that defaults to sending pictures in word documents to Microsoft servers for captioning, etc. I use a Mac now for work, but if somehow this snooping applies to computers as well I can't keep doing so while respecting the privacy of my clients. I echo the comment on another post, Apple is an entertainment company, I don't know why we all started using their products for business applications.
      • Asdrubalini 5y ago
        You can disable automatic backups, this way your photos won’t ever be uploaded to iCloud.
      • abawany 5y ago
        By default it is enabled. One has to go through Settings to turn off the default iCloud upload, afaik.
        • judge2020 5y ago
• marcinzm 5y ago
  Given all the zero day exploits on iOS I wonder if it's now going to be viable to hack someone's phone and upload child porn to their account. Apple with happily flag the photos and then, likely, get those people arrested. Now they have to, in practice, prove they were hacked which might be impossible. Will either ruin their reputation or put them in jail for a long time. Given past witch hunts it could be decades before people get exonerated.
  • toxik 5y ago
    This is really a difficult problem to solve I think. However, I think most people who are prosecuted for CP distribution are hoarding it by the terabyte. It’s hard to claim that you were unaware of that. A couple of gigabytes though? Plausible. And that’s what this CSAM scanner thing is going to find on phones.
    • emodendroket 5y ago
      A couple gigabytes is a lot of photos... and they'd all be showing up in your camera roll. Maybe possible but stretching the bounds of plausibility.
      • danachow 5y ago
        A couple gigabytes is enough to ruin someone’s day but not a lot to surreptitiously transfer, it’s literally seconds. Just backdate them and they may very well go unnoticed.
        • tornato7 5y ago
          It's also possible to 'hide' photos from the reel in the photos app. Many people are unaware of that feature so an attacker could hide as many photos they want in your iCloud.
          • imwillofficial 5y ago
            Assmuning they have access
• drzoltar 5y ago
  The other issue with these hashes is non-robustness to adversarial attacks. Simply rotating the image by a few degrees, or slightly translating/shearing it will move the hash well outside the threshold. The only way to combat this would be to use a face bounding box algorithm to somehow manually realign the image.
  • foobarrio 5y ago
    In my admittedly limited experience in image hashing, typically you extract some basic feature and transform the image before hashing (eg darkest corner in the upper left or look for verticals/horizontals and align). You also take multiple hashes of the images to handle various crops, black and white vs color. This increases robustness a bit but overall yea you can always transform the image in such a way to come up with a different enough hash. One thing that would be hard to catch is if you do something like a swirl and then the consumers of that content will use a plugin or something to "deswirl" the image. There's also something like the Scale Invariant Feature Transform that would protect against all affine transformations (scale, rotate, translate, skew). I believe one thing that's done is whenever any CP is found, the hashes of all images in the "collection" is added to the DB whether or not they actually contain abuse. So if there are any common transforms of existing images then those also now have their hashes added to the db. The idea being that a high percent of hits from even the benign hashes means the presence of the same "collection".
    • megous 5y ago
      Huh, or you can just use encryption if you'll be using some SW based transformation anyway.
• deleted 5y ago
  [deleted]
• ajklsdhfniuwehf 5y ago
  whatsapp and other apps place pictures from groups chats in folders deep in your IOS gallery. Swatting will be a problem all over again.... wait, did it ever stop being a problem?
• starkd 5y ago
  The method Apple is using looks more like a cryptographic hash. That's entirely different (and more secure) than a perceptual hash. From https://www.apple.com/child-safety/ https://www.apple.com/child-safety/ "Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the known CSAM hashes. This matching process is powered by a cryptographic technology called private set intersection, which determines if there is a match without revealing the result. The device creates a cryptographic safety voucher that encodes the match result along with additional encrypted data about the image. This voucher is uploaded to iCloud Photos along with the image." Elsewhere, it does explain the use of neuralhashes which I take to be the perceptual hash part of it. I did some work on a similar attempt awhile back. I also have a way to store hashes and find similar images. Here's my blog post. I'm currently working on a full site. http://starkdg.github.io/posts/concise-image-descriptor http://starkdg.github.io/posts/concise-image-descriptor
  • dogma1138 5y ago
    The cryptography is most likely done at a higher level than the perception comparison and is quite likely done to protect the CSAM hashes than your privacy. My interpretation of this is that they still use some sort of a perception based matching algorithm they just encrypt the hashes and then use some “zero knowledge proof” when comparing the locally generated hashes against the list, the result of which would be just that X hashes marched but not which X. This way there would be no way to reverse engineer the CSAM hash list or bypass the process by altering key regions of the image.
    • visarga 5y ago
      > the result of which would be just that X hashes marched but not which X That means you can't prove an incriminating file was not deleted even if you're the victim of a false positive. So they will suspect you and put you through the whole police investigation routine.
      • dogma1138 5y ago
        Not necessarily it just means that you don’t know/prove until a certain threshold is reached, in guessing above a specific one that hashes and the photo is then uploaded to Apple for verification and preservation.
• yellow_lead 5y ago
  Regarding false positives re:Apple, the Ars Technica article claims > Apple offers technical details, claims 1-in-1 trillion chance of false positives. There are two ways to read this, but I'm assuming it means, for each scan, there is a 1-in-1 trillion chance of a false positive. Apple has over 1 billion devices. Assuming ten scans per device per day, you would reach one trillion scans in ~100 days. Okay, but not all the devices will be on the latest iOS, not all are active, etc, etc. But this is all under the assumption those numbers are accurate. I imagine reality will be much worse. And I don't think the police will be very understanding. Maybe you will get off, but you'll be in a huge debt from your legal defense. Or maybe, you'll be in jail, because the police threw the book at you.
  • KarlKemp 5y ago
    Do you really believe that if they scan your photo library at 10am and don't get any false positives, another scan five hours later, with no changes to the library, has the same chance of getting false positives as the first one, independent of that result?
    • burnished 5y ago
      If you take photos, then yes?
    • NoNotTheDuo 5y ago
      Even if the library doesn’t change, doesn’t the possibility of the list of “bad” hashes changing exist? I.e., in your example, a new hash is added to by Apple to the list at 11:30am, and then checked against your unchanged library.
      • IfOnlyYouKnew 5y ago
        Oh god have mercy on whatever has happened to these people…
  • nanidin 5y ago
    > Apple has over 1 billion devices. Assuming ten scans per device per day, you would reach one trillion scans in ~100 days. People like to complain about the energy wasted mining cryptocurrencies - I wonder how this works out in terms of energy waste? How many people will be caught and arrested by this? Hundreds or thousands? Does it make economic sense for the rest of us to pay an electric tax in the name of scanning other people's phones for this? Can we claim it as a deductible against other taxes?
• SavantIdiot 5y ago
  This article covers three methods, all of which just look for alterations of a source image to find a fast match (in fact, that's the paper referenced). It is still a "squint to see if it is similar" test. I was under the impression there were more sophisticated methods that looked for types of images, not just altered known images. Am I misunderstanding?
  • chipotle_coyote 5y ago
    Apple's proposed system compares against a database of known images. I can't think of a way to "look for types of images" other than trying to do it with machine learning, which strikes me as fraught with incredible fiasco potential. (The compare-to-a-known-database approach has its own issues, including the ones the article talks about, of course.)
    • SavantIdiot 5y ago
      Ok, that's what it is seeming like. Since a crypto hash by definition has to generate a huge hamming distance for a small change, everything i've read about perceptual hashes is just the opposite: they should be tolerant enough of a certain amount of difference.
• mrtksn 5y ago
  The technical challenges aside, I’m very disturbed that my device will be reporting me to the authorities. That’s very different from authorities taking a sneak peek into my stuff. That’s like the theological concept of always being watched. It starts with child pornography but the technology is indifferent towards it, it can be anything. It’s always about the children because we all want to save the children. Soon they will start asking you start saving your country. Depending on your location they will start checking against sins against religion, race, family values, political activities. I bet you, after the next election in the US your device will be reporting you for spreading far right or deep state lies, depending on who wins. I’m big Apple fanboy, but I’m not going to carry a snitch in my pocket. That’s “U2 Album in everyone’s iTunes library” blunder level creepy with the only difference that it’s actually truly creepy. In my case, my iPhone is going to be snitching me to Boris and Erdogan, in your case it could be Macron, Bolsonaro, Biden, Trump etc. That’s no go for me, you can decide for yourself.
  • baggy_trough 5y ago
    Totally agree. This is very sinister indeed. Horrible idea, Apple.
    • zionic 5y ago
      So what are we going to do about it? I have a large user base on iOS. Considering a blackout protest.
      • mrtksn 5y ago
        IMHO, Unless everything being E2E encrypted becomes the law we can’t do anything about it because that’s not Apple’s initiative but comes from people whose job is to know things and they cannot resist keeping their hands out of these data collecting devices. They promise politicians that all the troubles will go away if we do that. Child pornography, Terrorism? Solve it the old way. I don’t know why citizens are obligated to make their jobs easier. We survived the times when phone calls were not moderated, we survived the times when signal intelligence was not a thing.
• ris 5y ago
  I agree with the article in general except part of the final conclusion > The simple fact that image data is reduced to a small number of bits leads to collisions and therefore false positives Our experience with regular hashes suggests this is not the underlying problem. SHA256 hashes have 256 bits and still there are no known collisions, even with people deliberately trying to find them. SHA-1 only has only 160 bits to play with and it's still hard enough to find collisions. MD5 is easier to find collisions but at 128 bits, still people don't come across them by chance. I think the actual issue is that perceptual hashes tend to be used with this "nearest neighbour" comparison scheme which is clearly needed to compensate for the inexactness of the whole problem.
  • dogma1138 5y ago
    This isn’t due to the entropy of the hash but due to the entropy of the source data. These algos work by limiting the color space of the photo, usually to only black and white (not even grey scale) resizing it to a fraction of its original size and then chopping it into tiles using a fixed size grid. This increases the chances of collisions greatly because photos with a similar composition are likely to match on a sufficient number of tiles to flag the photo as a match. This is why the women image was matched to the butterfly image, if you turn the image to B&W resize it to something like 256x256 pixels and divide it into a grid of say 16 tiles all of a sudden a lot of these tiles can match.
  • giantrobot 5y ago
    Perceptual hashes don't involve diffusion and confusion steps like cryptographic hashes. Perceptual hashes don't want decorrelation like cryptographic hashes. In fact they want similar but not identical images to end up with similar hash values.
• brian_herman 5y ago
  Fortunately I have a cisco router and enough knowledge to block the 17.0.0.0/8 ip address range. This combined with an openvpn vpn will block all apple services from my devices. So basically my internet will look like this: Internet <---> CISCO <---> ASUS ROUTER with openvpn <-> Network The cisco router will block the 17.0.0.0/8 ip address range and I will use spotify on all my computers.
  • brian_herman 5y ago
    Disregard comment I don't want to edit it because I am lazy. You can do all of this inside the asus router underneath the routes page just put this inside the asus router: Ip address 17.0.0.0 Subnet 255.0.0.0 Destination 127.0.0.1
    • procinct 5y ago
      You don't plan to ever use 4G/5G again?
      • brian_herman 5y ago
        I have openvpn so the block will remain in effect. I don't plan to use apple services ever again but the hard ware is pretty good.
        • loser777 5y ago
          Does this mean you are attempting to use an IP range block to avoid this "service" while continuing to use Apple hardware? How does such a block deal with say, Apple software conveniently "routing-around" what appears to be an "authoritarian government's firewall?"
  • verygoodname 5y ago
    And then they switch to using Akamai or AWS IP space (like Microsoft does), so you start blocking those as well?
• lancemurdock 5y ago
  I am going to give this lineageOS on an android device a shot. This is one of the most egregious things Apple has ever done
• ezoe 5y ago
  The problem of hash or NN based matching is, the authority can avoid explaining the mismatch. Suppose the authority want to false-arrest you. They prepare a hash that matches to an innocent image they knew the target has in his Apple product. They hand that hash to the Apple, claiming it's a hash from a child abuse image and demand privacy-invasive searching for the greater good. Then, Apple report you have a file that match the hash to the authority. The authority use that report for a convenient reason to false-arrest you. Now what happens if you sue the authority for the intentional false-arrest? Demand the original intended file for the hash? "No. We won't reveal the original file because it's child abusing image, also we don't keep the original file for moral reason" But come to think of it, we already have tons of such bogus pseudo-science technology like the dogs which conveniently bark at police's secret hand sign, polygraph, and the drug test kit which detect illegal drugs from thin air.
  • thaumasiotes 5y ago
    > like the dogs which conveniently bark at police's secret hand sign This isn't necessary; the state of the art is for drug dogs to alert 100% of the time. They're graded on whether they ever miss drugs. It's easy to never miss.
    • exporectomy 5y ago
      Airport baggage drug dogs must obviously have far fewer false positives than that. So alerting on everything can't be the state of the art.
      • deleted 5y ago
        [deleted]
      • thaumasiotes 5y ago
        https://reason.com/2021/05/13/the-police-dog-who-cried-drugs-at-every-traffic-stop/ https://reason.com/2021/05/13/the-police-dog-who-cried-drugs... > Similar patterns abound nationwide, suggesting that Karma's career was not unusual. Lex, a drug detection dog in Illinois, alerted for narcotics 93 percent of the time during roadside sniffs, but was wrong in more than 40 percent of cases. Sella, a drug detection dog in Florida, gave false alerts 53 percent of the time. Bono, a drug detection dog in Virginia, incorrectly indicated the presence of drugs 74 percent of the time.
• Waterluvian 5y ago
  I’m rather fascinated by the false matches. Those two images are very different and yet beautifully similar. I want to see a lot more pairs like this!
• altitudinous 5y ago
  This article focusses too much on the individual case, and not enough on the fact that Apple will need multiple matches to report someone. Images would normally be distributed in sets I suspect, so it is going to be easy to detect when someone is holding an offending set because of multiple matches. I don't think Apple are going to be concerned with a single hit. Here in the news offenders are reported as holding many thousands of images.
  • trynumber9 5y ago
    Does it scan files within archives? If it does, you could download the wrong zip and instantaneously be over their threshold.
    • altitudinous 5y ago
      The scanning is to take place within iCloud Photos, which handles images / videos etc on an individual basis. It would be a pretty easy thing to do for Apple to calculate hashes on these. I'm not sure how iOS handles archives, but it doesn't matter - remember it isn't 100% or 0% with these things - say only 50% of those people store images in iCloud Photo, catching out only 50% of those folk is still a good result.
      • trynumber9 5y ago
        Yeah, I'm not sure. Just is a bit worrying to me. On my device iCloud Drive synchronizes anything in my downloads folder. If images contained within zips are treated as individual images, then I'm always just one wrong click from triggering their threshold.
• ngneer 5y ago
  What is the ratio of consumers of child pornography to the population of iPhone users? In order of magnitude, is it 1%, 0.1%, 0.001%, 0.0001%? With all the press around the announcement, this is not exactly stealth technology. Wouldn't such consumers switch platforms, rendering the system pointless?
  • aix1 5y ago
    It's clearly a marketing exercise aimed to sell products to parents and other concerned citizens. It doesn't actually need to be effective to achieve this goal. (I am not saying whether it will or won't be, just that it doesn't need to be.)
• bastawhiz 5y ago
  Correct me if I'm wrong, but nowhere in Apple's announcement do they mention "perceptual" hashing. I've searched through some of the PDFs they link as well, but those also don't seem to mention the word "perceptual". Can someone point out exactly where this is mentioned?
  • rcarback 5y ago
    "NeuralHash is a perceptual hashing function" https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
• cratermoon 5y ago
  If I'm reading this right? Apple is saying they are going to flag CSAM they find on their servers. This article talks about finding a match for photos by comparing a hash of a photo you're testing with a hash you have, from a photo you have. Does this mean Apple had/has CSAM available to generate the hashes?
  • deleted 5y ago
    [deleted]
  • aix1 5y ago
    For the purposes of this they only have the hashes, which they receive from third parties. > on-device matching using a database of known CSAM image hashes provided by NCMEC and other child safety organizations https://www.apple.com/child-safety/ https://www.apple.com/child-safety/ (Now, I do wonder how secure those third parties are.)
• jiggawatts 5y ago
  The world in the 1900s: Librarians: "It is unthinkable that we would ever share a patron's borrowing history!" Post office employees: "Letters are private, only those commie countries open the mail their citizens send!" Police officers: "A search warrant from a Judge or probable cause is required before we can search a premises or tap a single, specific phone line!" The census: "Do you agree to share the full details of your record after 99 years have elapsed?" The world in the 2000s: FAANGs: "We know everything about you. Where you go. What you buy. What you read. What you say and to whom. What specific type of taboo pornography you prefer. We'll happily share it with used car salesmen and the hucksters that sell WiFi radiation blockers and healing magnets. Also: Cambridge Analytica, the government, foreign governments, and anyone who asks and can pony up the cash, really. Shh now, I have a quarterly earnings report to finish." Device manufacturers: "We'll rifle through your photos on a weekly basis, just to see if you've got some banned propaganda. Did I say propaganda? I meant child porn, that's harder to argue with. The algorithm is the same though, and just how the Australian government put uncomfortable information leaks onto the banned CP list, so will your government. No, you can't check the list! You'll have to just trust us." Search engines: "Tiananmen Square is located in Beijing China. Here's a cute tourist photo. No further information available." Online Maps: "Tibet (China). Soon: Taiwan (China)." Media distributors: "We'll go into your home, rifle through your albums, and take the ones we've stopped selling. Oh, not physically of course. No-no-no-no, nothing so barbaric! We'll simply remotely instruct your device to delete anything we no longer want you to watch or listen to. Even if you bought it from somewhere else and uploaded it yourself. It matches a hash, you see? It's got to go!" Governments: "Scan a barcode so that we can keep a record of your every movement, for public health reasons. Sure, Google and Apple developed a secure, privacy-preserving method to track exposures. We prefer to use our method instead. Did we forget to mention the data retention period? Don't worry about that. Just assume... indefinite."
  • bcrosby95 5y ago
    Your view of the 1900s is very idyllic.
• BiteCode_dev 5y ago
  The problem is not perceptual hashes. The problem is the back door. Let's not focus on the defect of the train leading you to the concentration camp. The problem is that there is a camp at the end of the rail road.
• legulere 5y ago
  Which photos does Apple scan? Also of emails and messages? Could you swat somebody by sending them benign images that have the same hash?
• btheshoe 5y ago
  I'm not insane in thinking this stuff has to be super vulnerable to adversarial attacks, right? And it's not like adversarial attacks are a solved problem or anything.
  • aix1 5y ago
    Yes, I agree that this is a significant risk.
  • mkl 5y ago
    Wouldn't you need a way to determine if an image you generate has a match in Apple's database? The way it's set up, that's not possible: "Given a user image, the general idea in PSI is to apply the same set of transformations on the image NeuralHash as in the database setup above and do a simple lookup against the blinded known CSAM database. However, the blinding step using the server-side secret is not possible on device because it is unknown to the device. The goal is to run the final step on the server and finish the process on server. This ensures the device doesn’t know the result of the match, but it can encode the result of the on-device match process before uploading to the server." -- https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... (emphasis mine)
    • btheshoe 5y ago
      I was thinking something along the lines of applying small transformations to all images before uploading, or even just images that are known to be problematic. Seems like something that people who traffic cp would be willing to do
• alkonaut 5y ago
  The key here is scale. If the only trigger for action is having (say) a few hundred matching images, or a dozen from the same known set of offending pictures, then I can see how apples “one in a trillion” claim would work. Also, Apple could ignore images from the device camera - since those will never match. This is also in stark contrast to the task faced by photo copyright hunters. They don’t have the luxury of only focusing on those who handle tens of thousands of copyrighted photos. They need to find individual violations because that’s what they are paid to do.
• avnigo 5y ago
  > These cases will be manually reviewed. That is, according to Apple, an Apple employee will then look at your (flagged) pictures. I'm surprised this hasn't gotten enough traction outside of tech news media. Remember the mass celebrity "hacking" of iCloud accounts a few years ago? I wonder how those celebrities would feel knowing that some of their photos may be falsely flagged and shown to other people. And that we expect those humans to act like robots and not sell or leak the photos, etc. Again, I'm surprised we haven't seen a far bigger outcry in the general news media about this yet, but I'm glad to see a lot of articles shining light on how easy it is for false positives and hash collisions to occur, especially at the scale of all iCloud photos.
  • judge2020 5y ago
    They wouldn't be falsely flagged. It doesn't detect naked photos, it detects photos matching real confirmed CSAM based on the NCMEC's database.
    • auggierose 5y ago
      If that would always work, a manual review would not be necessary. Just send the flagged photo and its owner straight to the police.
    • avnigo 5y ago
      The article posted, as well as many others we've seen recently, demonstrate that collisions are possible, and most likely inevitable with the number of photos to be scanned for iCloud, and Apple recognizes this themselves. It doesn't necessarily mean that all flagged photos would be of explicit content, but even if it's not, is Apple telling us that we should have no expectation of privacy for any photos uploaded to iCloud, after running so many marketing campaigns on privacy? The on-device scanning is also under the guise of privacy too, so they wouldn't have to decrypt the photos on their iCloud servers with the keys they hold (and also save some processing power, maybe).
      • spacedcowboy 5y ago
• siscia 5y ago
  What I am missing from all this story, is what triggered Apple to put in place, or even think about, this system. It is clearly a no-trivial project, no other company is doing it, and it will be one of the rare case of a company doing something not for shareholders value but for "goodwill". I am really not understanding the reasoning behind this choice.
  • jeromegv 5y ago
    One theory is that they are getting ready for E2E encryption of iCloud photos. Apple will have zero access to your photos in the cloud. So the only way to get the authorities to accept this new scheme is that there is this backdoor where there is a check client-side for sexual predator photos. Once your photo pass that check locally, it gets encrypted, sent to the cloud, never to be decrypted by apple. Not saying it will happen, but that's a decent theory as of why https://daringfireball.net/2021/08/apple_child_safety_initiatives_slippery_slope https://daringfireball.net/2021/08/apple_child_safety_initia...
  • MontagFTB 5y ago
    Legally, I believe, they are responsible for distribution of CSAM that may wind up in their cloud, regardless of who put it there. Many cloud companies are under considerable legal pressure to find and report it.
  • spacedcowboy 5y ago
    Er, every US company that hosts images in the cloud scans them for CSAM if they have access to the photo, otherwise they’re opening themselves up to a lawsuit. US law requires any ESP (electronic service provider) to alert NCMEC if they become aware of CSAM on their servers. Apple used to comply with this by scanning images on the server in iCloud photos, and now they’re moving that to the device if that image is about to be uploaded to iCloud photos. FWIW, the NYT says Apple reported 265 cases last year to NCMEC, and say Facebook reported 20.3 million. Google [1] are on for 365,319 for July->Dec. I’m still struggling to see what has changed here, apart from people realising what’s been happening.. - it’s the same algorithm that Apple has been using, comparing NCMEC-provided hashes against photos - it’s still only being done on photos that are uploaded to iCloud photos - it’s now done on-device rather than on-server, which removes a roadblock to future e2e encryption on the server. Seems the only real difference is perception. [1] https://transparencyreport.google.com/child-sexual-abuse-material/reporting?hl=en https://transparencyreport.google.com/child-sexual-abuse-mat...
• read_if_gay_ 5y ago
  Big tech has been disintegrating the foundational principles on which our society is built in the name of our society. Every one of their moves is a deeper attack on personal freedom than the last. They need to be dealt with. Stop using their services, buying their products, defending them when they silence people.
• ChrisMarshallNY 5y ago
  That’s a really useful explanation. Thanks!
• Wowfunhappy 5y ago
  > At my company, we use “perceptual hashes” to find copies of an image where each copy has been slightly altered. Kind of off topic, does anyone happen to know of some good software for doing this on a local collection of images? A common sequence of events at my company: 1. We're designing a website for some client. They send us a collection of a zillion photos to pull from. For the page about elephants, we select the perfect elephant photo, which we crop, lightly recolor, compress, and upload. 2. Ten years later, this client sends us a screenshot of the elephant page, and asks if we still have a copy of the original photo. Obviously, absolutely no one at this point remembers the name of the original photo, and we need to either spend hours searching for it or (depending on our current relationship) nicely explain that we can't help. It would be really great if we could do something like a reverse Google image search, but for a local collection. I know it's possible to license e.g. TinEye, but it's not practical for us as a tiny company. What I really want is an open source solution I can set up myself. We used Digicam for a while, and there were a couple of times it was useful. However, for whatever reason it seemed to be extremely crash-prone, and it frequently couldn't find things it really should have been able to find.
  • xioren00 5y ago
    https://pypi.org/project/ImageHash/ https://pypi.org/project/ImageHash/
    • Wowfunhappy 5y ago
      Thank you!
• lliamander 5y ago
  What about genuine duplicate photos? Say there is a stock picture of a landscape, and someone else goes and takes their own picture of the same landscape?
• JacobiX 5y ago
  Given that Apple technology uses NN and triplet embedding loss, the exact same techniques used by neural networks for face recognition, so maybe the same shortcomings would apply here. For example a team of researchers found a 'Master Faces' that can bypass over 40% of Facial ID. Now suppose that you have such an image in your photo library, it would generate so many false positives …
• madmax96 5y ago
  Why not make it so that I can see flagged images in my library? It would give me a lot more confidence that my photos stay private.
• acidioxide 5y ago
  It's really disturbing that, in case of doubt, real person would check photos. That's a red flag.
• kazinator 5y ago
  Perceptual hashing was invented by the Chinese: four-corner code character lookup, that lumps together characters with similar features.
• chucklenorris 5y ago
  So, if there's code on the device that's computing these hashes then it can be extracted. Afterwards it should be possible to add changes to a inocent picture to make it produce a target hash. Getting a hash should pe possible too, just find a known pedo image and run the extracted algorithm. It's only a matter of time until someone makes this
• chucklenorris 5y ago
  This technology is a godsend for the government to catch wistleblowers before they're able to leak information. You wouldn't even hear about those poor souls.