5 ms·

Reddit discussion: https://old.reddit.com/r/MachineLearning/comments/p6hsoh/p_appleneuralhash2onnx_reverseengineered_apple/ https://old.reddit.com/r/MachineLear

Reddit discussion: https://old.reddit.com/r/MachineLearning/comments/p6hsoh/p_appleneuralhash2onnx_reverseengineered_apple/ https://old.reddit.com/r/MachineLearning/comments/p6hsoh/p_a...

gary17the 5y ago
An interesting tidbit: "Believe it or not, [the NeuralHash algorithm for on-device CSAM detection] already exists as early as iOS 14.3, hidden under obfuscated class names."
- c7DJTLrn 5y ago
  So it was quite literally introduced as a trojan horse. "We're so excited to bring you all these new features and bugfixes in iOS 14.3, plus one more thing you'll hear about and object to in future. Too bad."
  - robertoandred 5y ago
    You've never heard of feature flags?
    - gary17the 5y ago
      You mean like...? let scanFile4CSAM: Bool if #available(iOS 16.0, *) { scanFile4CSAM = true } else { scanFile4CSAM = is_iCloudPhotosFile && Device.Settings.is_iCloudPhotosEnabled } Edit: "These efforts will evolve and expand over time."[1] [1] https://www.apple.com/child-safety/ https://www.apple.com/child-safety/
    - hda2 5y ago
      I'm sure apple would like everyone to call their trojan that way. "Feature Flag" lol.
xucheng 5y ago
In addition to generate the adversarial collisions, someone mentioned that it can also be used to train a decoder network to reverse any NeuralHash back to its input image.
- eurasiantiger 5y ago
  This absolutely needs to be done. Also, does Apple deploy different models for different regions/cohorts?
- MauranKilom 5y ago
  That assumes that 96 bits of information are sufficient for (in some sense) uniquely describing the input image. Which, on the one hand, is of course the purpose of the system, but on the other is also clearly mathematically impossible (a 360x360 RGB8 image has 3110400 bits of information). That is, for each 96 bit neural hash value, there exist (on average) 2^3110304 unique input images that hash to that same value. Again, these are of course trivial facts, which do not rule out that image recovery (in a "get back something that looks similar to the original input" sense) is possible, but you should be aware that "similar" to the network need not mean "similar" to a human.
  - xucheng 5y ago
    Just like any autoencoder, it is not about getting back the exact original, which is of course impossible. It is about summarizing the image in 96bits information, which is quite enough to leak the gist of the original image. For example, [1] talks about reversing Microsoft’s PhotoDNA. > but you should be aware that "similar" to the network need not mean "similar" to a human. With techniques like GAN and DLSS, it is quite possible to generate some photo realistic image being enough similar to the original one, or at least leaking some private information. [1]: https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
  - joe_the_user 5y ago
    "...but you should be aware that "similar" to the network need not mean "similar" to a human..." EXCEPT... neural hash also claims to be robust to modifications to images that would result in a similar-to-human-image. If the 96 bits is enough to tag such similar-to-humans results, why couldn't a brute force approach yield such similar-to-humans images? Indeed, a nefarious person intent on producing CSAM could set-up something like a generational-adversarial system that the produced CSAM images using the hashes along with other clues.

Reddit discussion: https://old.reddit.com/r/MachineLearning/comments/p6hsoh/p_appleneuralhash2onnx_reverseengineered_apple/ https://old.reddit.com/r/MachineLearning/comments/p6hsoh/p_a...

gary17the 5y ago

An interesting tidbit: "Believe it or not, [the NeuralHash algorithm for on-device CSAM detection] already exists as early as iOS 14.3, hidden under obfuscated class names."

c7DJTLrn 5y ago
So it was quite literally introduced as a trojan horse. "We're so excited to bring you all these new features and bugfixes in iOS 14.3, plus one more thing you'll hear about and object to in future. Too bad."
- robertoandred 5y ago
  You've never heard of feature flags?
  - gary17the 5y ago
    You mean like...? let scanFile4CSAM: Bool if #available(iOS 16.0, *) { scanFile4CSAM = true } else { scanFile4CSAM = is_iCloudPhotosFile && Device.Settings.is_iCloudPhotosEnabled } Edit: "These efforts will evolve and expand over time."[1] [1] https://www.apple.com/child-safety/ https://www.apple.com/child-safety/
  - hda2 5y ago
    I'm sure apple would like everyone to call their trojan that way. "Feature Flag" lol.

xucheng 5y ago

In addition to generate the adversarial collisions, someone mentioned that it can also be used to train a decoder network to reverse any NeuralHash back to its input image.

eurasiantiger 5y ago
This absolutely needs to be done. Also, does Apple deploy different models for different regions/cohorts?
MauranKilom 5y ago
That assumes that 96 bits of information are sufficient for (in some sense) uniquely describing the input image. Which, on the one hand, is of course the purpose of the system, but on the other is also clearly mathematically impossible (a 360x360 RGB8 image has 3110400 bits of information). That is, for each 96 bit neural hash value, there exist (on average) 2^3110304 unique input images that hash to that same value. Again, these are of course trivial facts, which do not rule out that image recovery (in a "get back something that looks similar to the original input" sense) is possible, but you should be aware that "similar" to the network need not mean "similar" to a human.
- xucheng 5y ago
  Just like any autoencoder, it is not about getting back the exact original, which is of course impossible. It is about summarizing the image in 96bits information, which is quite enough to leak the gist of the original image. For example, [1] talks about reversing Microsoft’s PhotoDNA. > but you should be aware that "similar" to the network need not mean "similar" to a human. With techniques like GAN and DLSS, it is quite possible to generate some photo realistic image being enough similar to the original one, or at least leaking some private information. [1]: https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
- joe_the_user 5y ago
  "...but you should be aware that "similar" to the network need not mean "similar" to a human..." EXCEPT... neural hash also claims to be robust to modifications to images that would result in a similar-to-human-image. If the 96 bits is enough to tag such similar-to-humans results, why couldn't a brute force approach yield such similar-to-humans images? Indeed, a nefarious person intent on producing CSAM could set-up something like a generational-adversarial system that the produced CSAM images using the hashes along with other clues.