4 ms·

I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin. Not sure exactly how you'd go about doing it, but i

I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin. Not sure exactly how you'd go about doing it, but it seems like there might be a process for 'evening out' areas into solid color that maintains the hash? In which case you're running extensive image processing on illegal images and making variations from those very images. More info on how this is done?

YokoSix 5y ago
Genuine question: If those image were really generated from illegal porn, are those images themselves considered illegal? Or in other words: How much do you have to modify illegal images for them to become legal again? Or do they stay illegal no matter how much you transform them?
- f38zf5vdt 5y ago
  Looking at the script below, it looks like it uses a gradient function for loss so that it learns to approach an image that generates a collision. If the case that the hashes themselves, being a result of a neural network, can be reverse engineered into pornographic images then does that raise a legal quandary? Apple said that the risk of collision is "1 in one trillion" which for a hash function would be terrible. We also don't know what the one trillion images they tested against were. If you upload your regular porn to iCloud, it's likely that pornographic images will raise more false positives than say, pictures of sunsets.
  - d35007 5y ago
    Apple said that the probability of a collision is quite a bit higher than that: > As the system is initially deployed, we do not assume the 3 in 100M image-level false positive rate we mea- sured in our empirical assessment The "1 in 1 trillion" part is the probability that the number of false positives could exceed the threshold needed to trigger a human review: > Apple always chooses the match threshold such that the possibility of any given account being flagged incorrectly is lower than one in one trillion, under a very conservative assumption of the NeuralHash false positive rate in the field. source: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model..., page 10
- Applejinx 5y ago
  Also relevant question: if these images were not at all generated from illegal porn, but they connect to hashes being used to flag illegal porn, is the purpose of this exercise to generate methods to SWAT people over the internet? As in, pursue a mechanism to get these onto somebody's computer in a way that they'll be backed up via iCloud (for instance, if a person's got their email account including trash folder backed up in iCloud, and you send them the pictures which they 'throw away' because it means nothing to them, placing the images in a trash folder in the mail preferences) Is that (a) practical and (b) the intent of this exercise? Seeing as every question I've had here has led to karma burning I figured I'd double down and ask if the person doing this is trying to prepare a weapon for swatting people. There are times I respond to downvoting pressure to 'stop talking!' by getting more interested, which I'm sure is a common reaction among some hackers.
dragonwriter 5y ago
> I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin I would assume these were engineered by getting the perceptual hash valies, using distance from the hash values in the DB as an error function, and starting with an innocuous image and hash value, and iterating to a collision for each.
- zepto 5y ago
  That technique can’t fool Apple’s algorithm. For what it’s worth, the null hypothesis is that they are just fakes and the commenter is at best trying to illustrate a point.
  - dragonwriter 5y ago
    > For what it’s worth, the null hypothesis is that they are just fakes and the commenter is at best trying to illustrate a point. No, that's not “the null hypothesis”. It is a positive claim.
    - zepto 5y ago
      Yes it is the null hypothesis. The poster is making a positive claim without evidence. Indeed the claim is unverifiable. Reasonable priors lead fo a null hypothesis that they are at least simply mistaken. This is without even taking into account other indicators of credibility or authority, or perverse incentives, as priors. This is a rational use of ‘null hypothesis’, but it also matches the scientific use, which would be that the claim is spurious unless experiment shows otherwise. In any case, we know that the poster is in fact wrong in their claim.
      - dragonwriter 5y ago
        > Yes it is the null hypothesis. No, its not. > The poster is making a positive claim without evidence. True. That doesn't make the alternate positive claim you have posited into “the null hypothesis”. A null hypothesis is null. What you are stating may be your prior, but it is not the, or even a valid, null hypothesis. > This is a rational use of ‘null hypothesis’, but it also matches the scientific use, “Null hypothesis” is a very specific scientific term of artz it has no other meaning. And, no, the specific counternarrative presented here does not match the scientific use of “null hypothesis”.
Applejinx 5y ago
Followup, since I can't edit: if my assumption isn't correct, well then, I stand corrected. I said in past tense, 'I assumed', and then asked for more info. That's not forthcoming, just a bunch of very upset assertions that of course I'm wrong and these things can't be reverse engineered from real porn. I'm sure not interested in proving they can. Mind furnishing the info about how it's really done, then? Since according to you (for very obvious reasons) you can never compare these images to the source for the hashes, where did you extract the hashes from? If you can so easily reverse engineer false positives from random data without ever seeing or using genuine porn to produce it, shouldn't you be disseminating this content as widely as you possibly can, rather than warning people about the danger of interacting with these false-positive images? Still puzzled how and why this is being done. Are you trying to render Apple's system useless, or not? Summary: I'm saying "there may be a way to take existing images that are illegal even to possess, and process them to obliterate the image while maintaining the hash. Is that what's being done here?" and the response is "AM NOT!!"

YokoSix 5y ago

Genuine question: If those image were really generated from illegal porn, are those images themselves considered illegal? Or in other words: How much do you have to modify illegal images for them to become legal again? Or do they stay illegal no matter how much you transform them?

f38zf5vdt 5y ago
Looking at the script below, it looks like it uses a gradient function for loss so that it learns to approach an image that generates a collision. If the case that the hashes themselves, being a result of a neural network, can be reverse engineered into pornographic images then does that raise a legal quandary? Apple said that the risk of collision is "1 in one trillion" which for a hash function would be terrible. We also don't know what the one trillion images they tested against were. If you upload your regular porn to iCloud, it's likely that pornographic images will raise more false positives than say, pictures of sunsets.
- d35007 5y ago
  Apple said that the probability of a collision is quite a bit higher than that: > As the system is initially deployed, we do not assume the 3 in 100M image-level false positive rate we mea- sured in our empirical assessment The "1 in 1 trillion" part is the probability that the number of false positives could exceed the threshold needed to trigger a human review: > Apple always chooses the match threshold such that the possibility of any given account being flagged incorrectly is lower than one in one trillion, under a very conservative assumption of the NeuralHash false positive rate in the field. source: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model..., page 10
Applejinx 5y ago
Also relevant question: if these images were not at all generated from illegal porn, but they connect to hashes being used to flag illegal porn, is the purpose of this exercise to generate methods to SWAT people over the internet? As in, pursue a mechanism to get these onto somebody's computer in a way that they'll be backed up via iCloud (for instance, if a person's got their email account including trash folder backed up in iCloud, and you send them the pictures which they 'throw away' because it means nothing to them, placing the images in a trash folder in the mail preferences) Is that (a) practical and (b) the intent of this exercise? Seeing as every question I've had here has led to karma burning I figured I'd double down and ask if the person doing this is trying to prepare a weapon for swatting people. There are times I respond to downvoting pressure to 'stop talking!' by getting more interested, which I'm sure is a common reaction among some hackers.

dragonwriter 5y ago

> I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin I would assume these were engineered by getting the perceptual hash valies, using distance from the hash values in the DB as an error function, and starting with an innocuous image and hash value, and iterating to a collision for each.

zepto 5y ago
That technique can’t fool Apple’s algorithm. For what it’s worth, the null hypothesis is that they are just fakes and the commenter is at best trying to illustrate a point.
- dragonwriter 5y ago
  > For what it’s worth, the null hypothesis is that they are just fakes and the commenter is at best trying to illustrate a point. No, that's not “the null hypothesis”. It is a positive claim.
  - zepto 5y ago
    Yes it is the null hypothesis. The poster is making a positive claim without evidence. Indeed the claim is unverifiable. Reasonable priors lead fo a null hypothesis that they are at least simply mistaken. This is without even taking into account other indicators of credibility or authority, or perverse incentives, as priors. This is a rational use of ‘null hypothesis’, but it also matches the scientific use, which would be that the claim is spurious unless experiment shows otherwise. In any case, we know that the poster is in fact wrong in their claim.
    - dragonwriter 5y ago
      > Yes it is the null hypothesis. No, its not. > The poster is making a positive claim without evidence. True. That doesn't make the alternate positive claim you have posited into “the null hypothesis”. A null hypothesis is null. What you are stating may be your prior, but it is not the, or even a valid, null hypothesis. > This is a rational use of ‘null hypothesis’, but it also matches the scientific use, “Null hypothesis” is a very specific scientific term of artz it has no other meaning. And, no, the specific counternarrative presented here does not match the scientific use of “null hypothesis”.

Applejinx 5y ago

Followup, since I can't edit: if my assumption isn't correct, well then, I stand corrected. I said in past tense, 'I assumed', and then asked for more info. That's not forthcoming, just a bunch of very upset assertions that of course I'm wrong and these things can't be reverse engineered from real porn. I'm sure not interested in proving they can. Mind furnishing the info about how it's really done, then? Since according to you (for very obvious reasons) you can never compare these images to the source for the hashes, where did you extract the hashes from? If you can so easily reverse engineer false positives from random data without ever seeing or using genuine porn to produce it, shouldn't you be disseminating this content as widely as you possibly can, rather than warning people about the danger of interacting with these false-positive images? Still puzzled how and why this is being done. Are you trying to render Apple's system useless, or not? Summary: I'm saying "there may be a way to take existing images that are illegal even to possess, and process them to obliterate the image while maintaining the hash. Is that what's being done here?" and the response is "AM NOT!!"