30 ms·

As the images never leave the device (and you can't build an image from the hash) then surely they are dependent on the victim certifying that the image is a pi

As the images never leave the device (and you can't build an image from the hash) then surely they are dependent on the victim certifying that the image is a picture of them. They have just made a tool to ban 20 arbitrary images at a time from social media that is gated by some questions that anyone can lie to. I wonder if you could interact with the API directly if you were feeling evil Real revenge porn uploaders could just change a pixel / reencode / tint the image like you do for flesh tone detection

IAmEveryone 5y ago
A perceptual hash is immune against the sort of manipulation you mention. One method I’ve implemented myself is recording changes in brightness along a path in the image. Inverting colors would work, but also make the image somewhat worthless. Flipping would also work, but is usually protected against by adding the flipped hash as well.
- quacksilver 5y ago
  True. I am not sure exactly what hash is being used (though others seem to indicate md5) I have heard from colleagues in the past that consumers of illegal pornography used to (or possibly still do) commonly apply a negative filter or hue shift to images to avoid hash matches and flesh tone detection. This helps when combined with other stuff if someone scans their disk or they get raided and have too much stuff for law enforcement to look through properly. They then set up their screen or display settings to counteract the filter when viewing it. Thankfully I have no experience with illegal pornography and definitely don't want to gain any, so hopefully I will never be able to confirm that firsthand.
  - dcallies 5y ago
    It’s somewhere in the FAQ at https://stopncii.org/faq/ https://stopncii.org/faq/, but It’s PDQ for photos (https://github.com/facebook/ThreatExchange/tree/main/pdq https://github.com/facebook/ThreatExchange/tree/main/pdq) and MD5 for videos. PDQ is resistant to some modifications (it focuses on the ones that come from regular usage, such as changing the format from gif to jpg, or a filter changing colors or brightness), but it’s not as resistant to modifications as you could get by training dedicated classifiers or other approaches that you might do with the original media or by storing more context, which StopNCII chose not to do.
BeefWellington 5y ago
I predict people taking famous people's profile pics and classifying them as NCII. Or perhaps suppressing a news story by submitting the generated thumbnail images for various platforms on desktop and mobile. Seems ripe for abuse to me, though the goal is laudable.
camillomiller 5y ago
Indeed. Reaction so far: Apple tries to fix a real problem botching its corporate comms badly — the Internet has a meltdown over it. Facebook does the same times 10 and with way more options to trick the system —- same old.
- mschuster91 5y ago
  The key difference is where the processing is happening. With Facebook, you can delete your account - with Apple's stuff, you'd have to get rid of potentially tens of thousands of dollars worth of equipment to escape the spying.

IAmEveryone 5y ago

A perceptual hash is immune against the sort of manipulation you mention. One method I’ve implemented myself is recording changes in brightness along a path in the image. Inverting colors would work, but also make the image somewhat worthless. Flipping would also work, but is usually protected against by adding the flipped hash as well.

quacksilver 5y ago
True. I am not sure exactly what hash is being used (though others seem to indicate md5) I have heard from colleagues in the past that consumers of illegal pornography used to (or possibly still do) commonly apply a negative filter or hue shift to images to avoid hash matches and flesh tone detection. This helps when combined with other stuff if someone scans their disk or they get raided and have too much stuff for law enforcement to look through properly. They then set up their screen or display settings to counteract the filter when viewing it. Thankfully I have no experience with illegal pornography and definitely don't want to gain any, so hopefully I will never be able to confirm that firsthand.
- dcallies 5y ago
  It’s somewhere in the FAQ at https://stopncii.org/faq/ https://stopncii.org/faq/, but It’s PDQ for photos (https://github.com/facebook/ThreatExchange/tree/main/pdq https://github.com/facebook/ThreatExchange/tree/main/pdq) and MD5 for videos. PDQ is resistant to some modifications (it focuses on the ones that come from regular usage, such as changing the format from gif to jpg, or a filter changing colors or brightness), but it’s not as resistant to modifications as you could get by training dedicated classifiers or other approaches that you might do with the original media or by storing more context, which StopNCII chose not to do.

BeefWellington 5y ago

I predict people taking famous people's profile pics and classifying them as NCII. Or perhaps suppressing a news story by submitting the generated thumbnail images for various platforms on desktop and mobile. Seems ripe for abuse to me, though the goal is laudable.

camillomiller 5y ago

Indeed. Reaction so far: Apple tries to fix a real problem botching its corporate comms badly — the Internet has a meltdown over it. Facebook does the same times 10 and with way more options to trick the system —- same old.

mschuster91 5y ago
The key difference is where the processing is happening. With Facebook, you can delete your account - with Apple's stuff, you'd have to get rid of potentially tens of thousands of dollars worth of equipment to escape the spying.