4 ms·

"Now you can block people in Drive. To prevent people from sharing unwanted files with you, ..." hahaha. What a coincidence, Google! So you got a hold of the

"Now you can block people in Drive. To prevent people from sharing unwanted files with you, ..." hahaha. What a coincidence, Google! So you got a hold of the neural hashes, and then used an error function and descent to generate images that match a 'hash'? It feels wrong to call them 'hashes' when they're so weak to pre-image attacks. They're not the same idea as cryptographic hashes at all. Also want to underline how spooky it is that some of them do resemble human forms.

411111111111111 5y ago
Some? Literally all are clearly based on pornography. First is veg&butthole, then boobs, next is doggy style etc etc (edit: it seems the order isn't consistent. So I'm likely seeing different images then you.) You can go through them all and see the original pornography if you look at the shapes. To me, it looks more like they started with the real images and tweaked them to make them artsy.
- yesbabyyes 5y ago
  I don't know, this sounds a bit like a Rorschach test to me.
  - deleted 5y ago
    [deleted]
- ASalazarMX 5y ago
  I love this comment. I have no idea how these were generated, but even starting with random noise it's possible to end up with vaguely human shapes if that's what originated the hashes to begin with. These images could be a joke, as I don't think we have a clear technical documentation of how these hashes are generated. Computer vision? Vectors? Face recognition software? It's definitely not a naive hash. Edit: seeing the other comments in this thread referencing Twitter, it looks like it's more naive than expected, as the hash is resistant to resizing, but not to cropping. The implementation can change at Apple's discretion, though.
Applejinx 5y ago
I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin. Not sure exactly how you'd go about doing it, but it seems like there might be a process for 'evening out' areas into solid color that maintains the hash? In which case you're running extensive image processing on illegal images and making variations from those very images. More info on how this is done?
- YokoSix 5y ago
  Genuine question: If those image were really generated from illegal porn, are those images themselves considered illegal? Or in other words: How much do you have to modify illegal images for them to become legal again? Or do they stay illegal no matter how much you transform them?
  - f38zf5vdt 5y ago
    Looking at the script below, it looks like it uses a gradient function for loss so that it learns to approach an image that generates a collision. If the case that the hashes themselves, being a result of a neural network, can be reverse engineered into pornographic images then does that raise a legal quandary? Apple said that the risk of collision is "1 in one trillion" which for a hash function would be terrible. We also don't know what the one trillion images they tested against were. If you upload your regular porn to iCloud, it's likely that pornographic images will raise more false positives than say, pictures of sunsets.
    - d35007 5y ago
      Apple said that the probability of a collision is quite a bit higher than that: > As the system is initially deployed, we do not assume the 3 in 100M image-level false positive rate we mea- sured in our empirical assessment The "1 in 1 trillion" part is the probability that the number of false positives could exceed the threshold needed to trigger a human review: > Apple always chooses the match threshold such that the possibility of any given account being flagged incorrectly is lower than one in one trillion, under a very conservative assumption of the NeuralHash false positive rate in the field. source: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model..., page 10
baybal2 5y ago
Don't forget how it started at Google. Long before they claimed that they scan email for child porn, they made an email scanner to appease China, on a condition they will not target dissidents. I think all remember how it went. Seeing their intimidation work, it only fired up the Chinese government, and led them to only increase their attempts at arm twisting, until Google clumsily pretended to "be tough" while still doing their last attempt behind the scenes negotiations, which, to their big surprise, got them banned overnight.
- panarky 5y ago
  Don't forget how Google admitted their mistake and withdrew from the Chinese market. Even though Google could have made a ton more money by helping China to build the tools of repression. Don't forget that other large US corporations like Microsoft, Apple and Activision do build censorship tools and participate in repressing dissent.
  - baybal2 5y ago
    > Don't forget how Google admitted their mistake and withdrew from the Chinese market. Which I point to the decision they came inadvertently. It was their intention to play games with the regime which backfired on them, not vice versa.
  - woofie11 5y ago
    My recollection was that Google was outgunned by Baidu and similar Chinese tools, who were being actively supported by the CCP, while Google was half-tolerate. At the same time, it was losing security, IP, and reputation. There was a nice PR play around it, but I don't think "Google could have made a ton more money by helping China to build the tools of repression." It's better to build those in-house.
dragonwriter 5y ago
> They're not the same idea as cryptographic hashes at all. There is a reason cryptographic hashes are distinguished; some applications of hashing are only concerned with minimizing non-malicious collisions. (Arguably, this is an application where malicious collisions are an issue, but perceptual hashes don't purport to be cryptographic.)

411111111111111 5y ago

Some? Literally all are clearly based on pornography. First is veg&butthole, then boobs, next is doggy style etc etc (edit: it seems the order isn't consistent. So I'm likely seeing different images then you.) You can go through them all and see the original pornography if you look at the shapes. To me, it looks more like they started with the real images and tweaked them to make them artsy.

yesbabyyes 5y ago
I don't know, this sounds a bit like a Rorschach test to me.
- deleted 5y ago
  [deleted]
ASalazarMX 5y ago
I love this comment. I have no idea how these were generated, but even starting with random noise it's possible to end up with vaguely human shapes if that's what originated the hashes to begin with. These images could be a joke, as I don't think we have a clear technical documentation of how these hashes are generated. Computer vision? Vectors? Face recognition software? It's definitely not a naive hash. Edit: seeing the other comments in this thread referencing Twitter, it looks like it's more naive than expected, as the hash is resistant to resizing, but not to cropping. The implementation can change at Apple's discretion, though.

Applejinx 5y ago

I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin. Not sure exactly how you'd go about doing it, but it seems like there might be a process for 'evening out' areas into solid color that maintains the hash? In which case you're running extensive image processing on illegal images and making variations from those very images. More info on how this is done?

YokoSix 5y ago
Genuine question: If those image were really generated from illegal porn, are those images themselves considered illegal? Or in other words: How much do you have to modify illegal images for them to become legal again? Or do they stay illegal no matter how much you transform them?
- f38zf5vdt 5y ago
  Looking at the script below, it looks like it uses a gradient function for loss so that it learns to approach an image that generates a collision. If the case that the hashes themselves, being a result of a neural network, can be reverse engineered into pornographic images then does that raise a legal quandary? Apple said that the risk of collision is "1 in one trillion" which for a hash function would be terrible. We also don't know what the one trillion images they tested against were. If you upload your regular porn to iCloud, it's likely that pornographic images will raise more false positives than say, pictures of sunsets.
  - d35007 5y ago
    Apple said that the probability of a collision is quite a bit higher than that: > As the system is initially deployed, we do not assume the 3 in 100M image-level false positive rate we mea- sured in our empirical assessment The "1 in 1 trillion" part is the probability that the number of false positives could exceed the threshold needed to trigger a human review: > Apple always chooses the match threshold such that the possibility of any given account being flagged incorrectly is lower than one in one trillion, under a very conservative assumption of the NeuralHash false positive rate in the field. source: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model..., page 10

baybal2 5y ago

Don't forget how it started at Google. Long before they claimed that they scan email for child porn, they made an email scanner to appease China, on a condition they will not target dissidents. I think all remember how it went. Seeing their intimidation work, it only fired up the Chinese government, and led them to only increase their attempts at arm twisting, until Google clumsily pretended to "be tough" while still doing their last attempt behind the scenes negotiations, which, to their big surprise, got them banned overnight.

panarky 5y ago
Don't forget how Google admitted their mistake and withdrew from the Chinese market. Even though Google could have made a ton more money by helping China to build the tools of repression. Don't forget that other large US corporations like Microsoft, Apple and Activision do build censorship tools and participate in repressing dissent.
- baybal2 5y ago
  > Don't forget how Google admitted their mistake and withdrew from the Chinese market. Which I point to the decision they came inadvertently. It was their intention to play games with the regime which backfired on them, not vice versa.
- woofie11 5y ago
  My recollection was that Google was outgunned by Baidu and similar Chinese tools, who were being actively supported by the CCP, while Google was half-tolerate. At the same time, it was losing security, IP, and reputation. There was a nice PR play around it, but I don't think "Google could have made a ton more money by helping China to build the tools of repression." It's better to build those in-house.

dragonwriter 5y ago

> They're not the same idea as cryptographic hashes at all. There is a reason cryptographic hashes are distinguished; some applications of hashing are only concerned with minimizing non-malicious collisions. (Arguably, this is an application where malicious collisions are an issue, but perceptual hashes don't purport to be cryptographic.)