6 ms·

The point in the NeuralHash and PSI system is to preserve user privacy as far as possible. From a technical standpoint, it is not essential - a NeuralHash funct

by toxik 5y ago

The point in the NeuralHash and PSI system is to preserve user privacy as far as possible. From a technical standpoint, it is not essential - a NeuralHash function that returns 0x0000… for everything would still catch CSAM. It's just that it would upload every single image on the user's device. Now, how well this NeuralHash does preserve privacy is a different question, and /not/ one that is being answered by the original post here. In fact, I've not seen anybody look at the hash distribution over natural images, which would be an actual argument against the system.

Retr0id 5y ago
It doesn't really matter whether all images are uploaded, or just 1 in x (for large value of x), due to the Panopticon effect.
- ithkuil 5y ago
  Let's not forget what the alternative is: this is about images that are uploaded on icloud anyway. The alternative is to upload the image in clear (or with ane encryption key that apple controls), and let apple run the CSAM filter on their servers. Apple now has the ability to encrypt the images before sending them to icloud, with a private key you own. Except that some percentage of images that match the CSAM fingerprint with their neural feature extractor will be sent to a CSAM filter on the server side (whose workings we don't have many details about) This whole thing backfired on Apple entirely due to psychological effects, not because they are really doing anything more "panopticon" that they would already able to do now on their icloud storage (after all people are ready sending their photos to apple)
  - simion314 5y ago
    But if Apple really cares about children why they did not done this scans in iCloud like all the others? Did not care as much as Google or Facebook? Seems to me like Apple does not care at all and seems more like a dev with big ego wanted to add neural hashes to his CV but if you can explain how Apple cared for children all this years but only now are doing something I really want to see the explanation
    - deleted 5y ago
      [deleted]
    - theshrike79 5y ago
      Because Apple is the one company that actively tries not to know anything about you. FB and Google will exhaustively analyse every single facet of your online presence and use your pictures to train their ML models for face detection and object detection. Apple, on the other hand, even explicitly splits Map directions to segments so that they can't know where you left from and where you are going to.
      - simion314 5y ago
        Apple was sending unencrypted on the network what application you started, if they would "actively tries not to know anything about you" they could have implemented this better. Anyway how is your assumption make sense , Apple cares about children and about your privacy so scanning your images in iCloud was wrong until 2021 when something changed, what changed? does Apple cares more about children starting from now or they care less about privacy? or are they forced to do it?
  - ben-schaaf 5y ago
    They already have the decryption keys for iCloud. Undoubtedly they've already been running a similar CSAM filter server-side for ages. The only thing doing this stuff client side has done is reduce privacy and erode trust.
    - madeofpalk 5y ago
      > Undoubtedly they've already been running a similar CSAM filter server-side for ages Apparently they have not - They say facebook reported something like 20m images in a year, and Apple reported 250.
    - saynay 5y ago
      I don't see how the privacy is any more eroded than it already was? Eroded trust, sure, but that is mostly because they did a terrible job communicating it.
  - PolCPP 5y ago
    Theres not even a need to do that, upload the encrypted image and its neuralhash.
    - ithkuil 5y ago
      What should they do if the neuralhash matches CSAM? Should they trust that the nerualhash actually matches real CSAM and you will should be reported to the police? That's clearly wrong, since there are going to be false positives, by design. The whole point for this is to be a probabilistic filter, so that they need to run the real CSAM scanner on a subset of files. You can fall into two camps: a) apple should never ever scan my private images I upload on their cloud. b) apple can scan the images once they reach their servers. If you pick (a), then clearly neuralhash shouldn't exist and you can argue against that on the ground that you want utter privacy. But you have to be consistent:every other cloud service that does scan the images server side should receive the same critique. If you pick (b), then you must recognize that this additional machinery doesn't increase their reach to your private data, but quite the opposite, it allows them to implement e2e encryption for 99.9% of your content. You may argue that it's unnecessary and confusing and spooky and be afraid of the slippery slope precendent for other uses.
read_if_gay_ 5y ago
The “actual argument” against the system that this provides is that Apple lied about the likelihood of hash collisions. Therefore, why trust any of their other claims?
- simondotau 5y ago
  Apple's claims are based on the statistical likelihood that there would be 30 collisions with CSAM hashes within one user account. Just because someone has found an image of a nearly featureless diagonal thing which collides with another image of a nearly featureless diagonal thing, that doesn't disprove Apple's claims.
  - BeefWellington 5y ago
    > Apple's claims are based on the statistical likelihood that there would be 30 collisions with CSAM hashes within one user account. Given people can now generate images that collide, it seems like the statistical likelihood has drastically changed since it was originally announced.
- toxik 5y ago
  I find this an unconvincing argument as well, you're saying that because Apple made a false claim, any claim may be valid. This is obviously not the case, what they did was to, albeit likely knowingly, calculate the hash collision probability /if each bit is a coin flip/, which comes out to pow(2, -k) for k bits. It's tiny. Of course, each bit is /not/ an independent coin flip under the NeuralHash function. So again the actual argument becomes: what is that distribution like?
  - read_if_gay_ 5y ago
    Fine, let’s say they didn’t lie, they made misleading claims. Still. Why trust them after that? If a company can make my own smartphone report me to the police, and they want my business, they better prove I can trust them. Apple has plainly done the opposite. The whole ordeal is just utterly 1984.
  - 5y ago
tsimionescu 5y ago
Sure, but a NeuralHash system with a collision chance of 100% would obviously not be abused to say NeuralHash collision implies CSAM - the secondary validation system will definitely be the last bastion. In contrast, a NH system believed to have a collision chance of 1 in a trillion trillion may well be considered infallible, and any detection be directly reported as CSAM, with the 'backend verification' amounting to nothing more than a rubber stamp. Of course, if you implicitly trust Apple not to do the second, than you're right, the NH collision rate doesn't matter too much.

by toxik 5y ago

Retr0id 5y ago

It doesn't really matter whether all images are uploaded, or just 1 in x (for large value of x), due to the Panopticon effect.

ithkuil 5y ago
Let's not forget what the alternative is: this is about images that are uploaded on icloud anyway. The alternative is to upload the image in clear (or with ane encryption key that apple controls), and let apple run the CSAM filter on their servers. Apple now has the ability to encrypt the images before sending them to icloud, with a private key you own. Except that some percentage of images that match the CSAM fingerprint with their neural feature extractor will be sent to a CSAM filter on the server side (whose workings we don't have many details about) This whole thing backfired on Apple entirely due to psychological effects, not because they are really doing anything more "panopticon" that they would already able to do now on their icloud storage (after all people are ready sending their photos to apple)
- simion314 5y ago
  But if Apple really cares about children why they did not done this scans in iCloud like all the others? Did not care as much as Google or Facebook? Seems to me like Apple does not care at all and seems more like a dev with big ego wanted to add neural hashes to his CV but if you can explain how Apple cared for children all this years but only now are doing something I really want to see the explanation
  - deleted 5y ago
    [deleted]
  - theshrike79 5y ago
    Because Apple is the one company that actively tries not to know anything about you. FB and Google will exhaustively analyse every single facet of your online presence and use your pictures to train their ML models for face detection and object detection. Apple, on the other hand, even explicitly splits Map directions to segments so that they can't know where you left from and where you are going to.
    - simion314 5y ago
      Apple was sending unencrypted on the network what application you started, if they would "actively tries not to know anything about you" they could have implemented this better. Anyway how is your assumption make sense , Apple cares about children and about your privacy so scanning your images in iCloud was wrong until 2021 when something changed, what changed? does Apple cares more about children starting from now or they care less about privacy? or are they forced to do it?
- ben-schaaf 5y ago
  They already have the decryption keys for iCloud. Undoubtedly they've already been running a similar CSAM filter server-side for ages. The only thing doing this stuff client side has done is reduce privacy and erode trust.
  - madeofpalk 5y ago
    > Undoubtedly they've already been running a similar CSAM filter server-side for ages Apparently they have not - They say facebook reported something like 20m images in a year, and Apple reported 250.
  - saynay 5y ago
    I don't see how the privacy is any more eroded than it already was? Eroded trust, sure, but that is mostly because they did a terrible job communicating it.
- PolCPP 5y ago
  Theres not even a need to do that, upload the encrypted image and its neuralhash.
  - ithkuil 5y ago
    What should they do if the neuralhash matches CSAM? Should they trust that the nerualhash actually matches real CSAM and you will should be reported to the police? That's clearly wrong, since there are going to be false positives, by design. The whole point for this is to be a probabilistic filter, so that they need to run the real CSAM scanner on a subset of files. You can fall into two camps: a) apple should never ever scan my private images I upload on their cloud. b) apple can scan the images once they reach their servers. If you pick (a), then clearly neuralhash shouldn't exist and you can argue against that on the ground that you want utter privacy. But you have to be consistent:every other cloud service that does scan the images server side should receive the same critique. If you pick (b), then you must recognize that this additional machinery doesn't increase their reach to your private data, but quite the opposite, it allows them to implement e2e encryption for 99.9% of your content. You may argue that it's unnecessary and confusing and spooky and be afraid of the slippery slope precendent for other uses.

read_if_gay_ 5y ago

The “actual argument” against the system that this provides is that Apple lied about the likelihood of hash collisions. Therefore, why trust any of their other claims?

simondotau 5y ago
Apple's claims are based on the statistical likelihood that there would be 30 collisions with CSAM hashes within one user account. Just because someone has found an image of a nearly featureless diagonal thing which collides with another image of a nearly featureless diagonal thing, that doesn't disprove Apple's claims.
- BeefWellington 5y ago
  > Apple's claims are based on the statistical likelihood that there would be 30 collisions with CSAM hashes within one user account. Given people can now generate images that collide, it seems like the statistical likelihood has drastically changed since it was originally announced.
toxik 5y ago
I find this an unconvincing argument as well, you're saying that because Apple made a false claim, any claim may be valid. This is obviously not the case, what they did was to, albeit likely knowingly, calculate the hash collision probability /if each bit is a coin flip/, which comes out to pow(2, -k) for k bits. It's tiny. Of course, each bit is /not/ an independent coin flip under the NeuralHash function. So again the actual argument becomes: what is that distribution like?
- read_if_gay_ 5y ago
  Fine, let’s say they didn’t lie, they made misleading claims. Still. Why trust them after that? If a company can make my own smartphone report me to the police, and they want my business, they better prove I can trust them. Apple has plainly done the opposite. The whole ordeal is just utterly 1984.
- 5y ago

tsimionescu 5y ago

Sure, but a NeuralHash system with a collision chance of 100% would obviously not be abused to say NeuralHash collision implies CSAM - the secondary validation system will definitely be the last bastion. In contrast, a NH system believed to have a collision chance of 1 in a trillion trillion may well be considered infallible, and any detection be directly reported as CSAM, with the 'backend verification' amounting to nothing more than a rubber stamp. Of course, if you implicitly trust Apple not to do the second, than you're right, the NH collision rate doesn't matter too much.