8 ms·

https://towardsdatascience.com/black-box-attacks-on-perceptual-image-hashes-with-gans-cc1be11f277?gi=cfb3a66600e5 https://towardsdatascience.com/black-box-attac

by imroot 5y ago

https://towardsdatascience.com/black-box-attacks-on-perceptual-image-hashes-with-gans-cc1be11f277?gi=cfb3a66600e5 https://towardsdatascience.com/black-box-attacks-on-perceptu... I mean, who's to say that my hash is actually what they think it is... ...and if it's not uploaded to iCloud for someone to manually review what the image is, is that going to be enough to get a search warrant for my phone? "AUSA SOMEONE: But, Your Honor, The Defendant's phone has an image that matched a hash of a CSAM picture!" "DEFENDANT: I'm not going to give you my phone's password." "THE COURT: Bonk: Stay in jail for contempt." I understand why they're doing it. I don't agree with it. At all, and I say that as a survivor of childhood sexual abuse.

Crontab 5y ago
I have always been concerned that this system could be weaponized as a way gain access to someone's account. For example: - Add the hash of a non-pornographic image to the database - Using a burner email address, email the non-pornographic image to the target's Gmail address. The target wouldn't think anything of it. - The innocent image would trigger a CP alert, giving law enforcement the pretense it needs to access the account
- smsm42 5y ago
  This is how the police gets warrantless searches "for drugs" - having identified the subject, call for a dog and have it "alert" on the subject (or the vehicle belonging to the subject), after which there's a probable cause for a search and everything that is found can be used against the subject. The fact that there was no drugs is irrelevant. Now let's translate it to digital world and eliminate the non-scalable components - and you get AI "alerting" on the content of your account and it's a probable cause for searching you. But the best part is of course the Apple doesn't even need a probable cause to search you - they aren't the police, so they are free to search anything at their pleasure and then report to the police. And this report, by itself, would be the probable cause, of course.
zug_zug 5y ago
EDIT: It appears they are using perceptual hashes, which are likely much more prone to collisions. The following math does NOT apply to the tech Apple is using. >> I mean, who's to say that my hash is actually what they think it is... Let's do the math. assume they are using a UUID for the hash. "For example, the number of random version-4 UUIDs which need to be generated in order to have a 50% probability of at least one collision is 2.71 quintillion" [1] "The odds of This number is equivalent to generating 1 billion UUIDs per second for about 85 years." And that's for just 1 collision between two random photos. Considering that the pool of bad hashes is not going to be 2.7 quintillion but a fraction of that, this is more than adequate. And of course they could use a longer checksum if that's the concern. (Not to take a stance on the parent issue, just to explain that hash's are probably a higher standard of proof than DNA matches) 1. https://en.wikipedia.org/wiki/Universally_unique_identifier#Collisions https://en.wikipedia.org/wiki/Universally_unique_identifier#...
- mod50ack 5y ago
  They're doing perceptible hashes, not file hashes. This was covered earlier in a different article posted here.
- sebzim4500 5y ago
  What if an attacker has intentionally produced an image with the same hash as a well known blacklisted image? Then they could send you the new image and fuck up your life.
  - Causality1 5y ago
    Hell, what if you were browsing a site like Facebook or Imgur or Reddit, saw an abuse image, reported it, but it still lives in your phone's browser cache?
    - voxic11 5y ago
      This is exactly why I use firefox and switch it to in-memory caching only.
    - 5y ago
JKCalhoun 5y ago
I understand the danger of the false positive but I suspect someone with actual child pornography on their device would trigger a dozen or more matches. Clever software would ignore a single match.
colinmhayes 5y ago
Fifth amendment holds that the state can't compel someone to give up their password.
- jtbayly 5y ago
  But that hasn't stopped judges from demanding it and keeping people in jail for contempt of court when they refuse.
- iso1631 5y ago
  But you can be jailed for 18 months for not giving it up
  - colinmhayes 5y ago
    Not if your lawyer is any good.
    - curryst 5y ago
      https://arstechnica.com/tech-policy/2020/02/man-who-refused-to-decrypt-hard-drives-is-free-after-four-years-in-jail/ https://arstechnica.com/tech-policy/2020/02/man-who-refused-... Nope, federal law states they can hold you for 18 months. So long as they can claim to already know what's on the device (which they will, because Apple pinged them about it), they can hold you for 18 months.
      - colinmhayes 5y ago
        Alright, after further investigation it looks like the Supreme Court has so far declined to rule on this issue so it depends on the state. It sounds like most states supreme courts have ruled that you can't be forced to unlock a device, but some, like pennsylvania say that if the state can prove you know the password they can force you to unlock the device. Definitely a pretty thorny situation.
- clusterfish 5y ago
  Except 100 miles from the border apparently https://www.aclu.org/other/constitution-100-mile-border-zone https://www.aclu.org/other/constitution-100-mile-border-zone

by imroot 5y ago

Crontab 5y ago

I have always been concerned that this system could be weaponized as a way gain access to someone's account. For example: - Add the hash of a non-pornographic image to the database - Using a burner email address, email the non-pornographic image to the target's Gmail address. The target wouldn't think anything of it. - The innocent image would trigger a CP alert, giving law enforcement the pretense it needs to access the account

smsm42 5y ago
This is how the police gets warrantless searches "for drugs" - having identified the subject, call for a dog and have it "alert" on the subject (or the vehicle belonging to the subject), after which there's a probable cause for a search and everything that is found can be used against the subject. The fact that there was no drugs is irrelevant. Now let's translate it to digital world and eliminate the non-scalable components - and you get AI "alerting" on the content of your account and it's a probable cause for searching you. But the best part is of course the Apple doesn't even need a probable cause to search you - they aren't the police, so they are free to search anything at their pleasure and then report to the police. And this report, by itself, would be the probable cause, of course.

zug_zug 5y ago

EDIT: It appears they are using perceptual hashes, which are likely much more prone to collisions. The following math does NOT apply to the tech Apple is using. >> I mean, who's to say that my hash is actually what they think it is... Let's do the math. assume they are using a UUID for the hash. "For example, the number of random version-4 UUIDs which need to be generated in order to have a 50% probability of at least one collision is 2.71 quintillion" [1] "The odds of This number is equivalent to generating 1 billion UUIDs per second for about 85 years." And that's for just 1 collision between two random photos. Considering that the pool of bad hashes is not going to be 2.7 quintillion but a fraction of that, this is more than adequate. And of course they could use a longer checksum if that's the concern. (Not to take a stance on the parent issue, just to explain that hash's are probably a higher standard of proof than DNA matches) 1. https://en.wikipedia.org/wiki/Universally_unique_identifier#Collisions https://en.wikipedia.org/wiki/Universally_unique_identifier#...

mod50ack 5y ago
They're doing perceptible hashes, not file hashes. This was covered earlier in a different article posted here.
sebzim4500 5y ago
What if an attacker has intentionally produced an image with the same hash as a well known blacklisted image? Then they could send you the new image and fuck up your life.
- Causality1 5y ago
  Hell, what if you were browsing a site like Facebook or Imgur or Reddit, saw an abuse image, reported it, but it still lives in your phone's browser cache?
  - voxic11 5y ago
    This is exactly why I use firefox and switch it to in-memory caching only.
  - 5y ago

JKCalhoun 5y ago

I understand the danger of the false positive but I suspect someone with actual child pornography on their device would trigger a dozen or more matches. Clever software would ignore a single match.

colinmhayes 5y ago

Fifth amendment holds that the state can't compel someone to give up their password.

jtbayly 5y ago
But that hasn't stopped judges from demanding it and keeping people in jail for contempt of court when they refuse.
iso1631 5y ago
But you can be jailed for 18 months for not giving it up
- colinmhayes 5y ago
  Not if your lawyer is any good.
  - curryst 5y ago
    https://arstechnica.com/tech-policy/2020/02/man-who-refused-to-decrypt-hard-drives-is-free-after-four-years-in-jail/ https://arstechnica.com/tech-policy/2020/02/man-who-refused-... Nope, federal law states they can hold you for 18 months. So long as they can claim to already know what's on the device (which they will, because Apple pinged them about it), they can hold you for 18 months.
    - colinmhayes 5y ago
      Alright, after further investigation it looks like the Supreme Court has so far declined to rule on this issue so it depends on the state. It sounds like most states supreme courts have ruled that you can't be forced to unlock a device, but some, like pennsylvania say that if the state can prove you know the password they can force you to unlock the device. Definitely a pretty thorny situation.
clusterfish 5y ago
Except 100 miles from the border apparently https://www.aclu.org/other/constitution-100-mile-border-zone https://www.aclu.org/other/constitution-100-mile-border-zone