5 ms·

PhotoDNA

Bayes7 2y ago
Is there any good article/paper that describes how it actually works or is implemented not just in high-level and hand-waving terms?
- unsnap_biceps 2y ago
  https://hackerfactor.com/blog/index.php?/archives/931-PhotoDNA-and-Limitations.html https://hackerfactor.com/blog/index.php?/archives/931-PhotoD... is linked from https://anishathalye.com/inverting-photodna/ https://anishathalye.com/inverting-photodna/ and is likely a good starting point
  - Bayes7 2y ago
    thanks!
  - Springtime 2y ago
    Yeah Hacker Factor's multi-post critiques are where I first saw it analyzed. For reference they run the popular fotoforensics.com image analysis site. They also have scathing critique (eg [1]) about the Adobe-led C2PA digital provenance signing, having themselves been part of various groups that seek solutions to the provenance problem. [1] https://www.hackerfactor.com/blog/index.php?/archives/1013-C2PAs-Worst-Case-Scenario.html https://www.hackerfactor.com/blog/index.php?/archives/1013-C...
- majke 2y ago
  The secrecy of the inner tech is intentional.
- bawolff 2y ago
  There tends to be more information under the search term "perceptual hashing"
noduerme 2y ago
What's novel about this? iirc, Apple withdrew its plan to hash photos client-side a couple years ago after an outcry. Dropbox has been hashing every file forever to save storage space. Store your shit with a cloud provider, expect it to get scanned, right? Also, there are a million cute methods to make two different photos produce the same hash; that was actually what the outcry about Apple's version was about. The more the hash algorithm tried to produce the same hash for different variants of a photo, the more likely it was that someone could get their hands on a flagged hash and theoretically send you an innocuous looking photo that registered as CSAM. Pretty sure that's why Apple pulled it.
- kalleboo 2y ago
  > theoretically send you an innocuous looking photo that registered as CSAM. Pretty sure that's why Apple pulled it. Apple also had human reviewers in the mix. The only reason they pulled it was PR/optics.
  - zaptrem 2y ago
    Wouldn't that require exfiltrating the original photo? I remember them swearing that wasn't part of the deal.
    - jchw 2y ago
      Going from memory here but IIRC the deal was that on device they'd produce a hash using a known pHash, and if that was positive, they'd send the photo to check it against a second pHash that wasn't publicly-disclosed (to try to mitigate the problem of intentional collisions) and then if both of them were positive matches, they would have human reviewers in the loop.
      - nullc 2y ago
        Not "if then send" -- if that were the case you could detect that your images were matching and then use the system as an oracle to e.g. detect that the database might contain disfavored political content in addition to whatever they claim it contains. The system was proposed as part of the upload process to your private encrypted cloud storage. So they already have the files. The system was designed so that if there were a sufficient number of hits against an encrypted database then a party possessing a particular private key would learn the private key for the encrypted files, as a passive effect of there being matches. Like if I mutter random letters of my password while entering it, you'd eventually passively learn my password just by hanging around while I logged in each morning. This way any matching is completely undetectable on the users end, at least until the private key possessing parties or the people they share the information with choose to take some action against the user.
zaptrem 2y ago
https://anishathalye.com/inverting-photodna/ https://anishathalye.com/inverting-photodna/ You could almost certainly produce nearly photo realistic PhotoDNA inversions with a finetuned diffusion model now. Is it possible to create a perceptual hashing algorithm where this isn't possible?
- tashbarg 2y ago
  You can certainly not produce inversions. The data that is left in the hash is not enough to produce anything vaguely photorealistic. However, you can fill the gaps and generate photorealistic photos that fit to the extremely reduced information you get from the hash. You are generating believable (as defined by the training data) photos that fit the hash. That’s a huge difference. Statements like yours are extremely dangerous. Without proper understanding of what GenAI can and can not do, people start relying on things that are not there. Imagine your photorealistic inversion AI putting a mole or a wrinkle in the face of somebody without any foundation in the actual hash. Just because it fits better to the trained data. Explain that to the judge, when the person with just the right facial features sits in front of them.
  - jasonvorhe 2y ago
    This modern narrative of people posting their opinions or assumptions somewhere being "dangerous" because someone could just believe it is much more dangerous because it can be applied to any opinion anywhere that was ever published. No judge will ever rule on something based on a comment they read in the Internet.
    - tashbarg 2y ago
      Judges usually rely on experts in forensic science who, of course, are infallible and absolutely not influenced by what they read online during their day. https://innocenceproject.org/misapplication-of-forensic-science/ https://innocenceproject.org/misapplication-of-forensic-scie... It is dangerous to push the narrative that GenAI can "put information back" where it was once removed. Especially dangerous, because most GenAI is built to put something there that is extremely believable. And while an innocent comment on HN might not play the biggest role, the linked project claims exactly what it can - by definition - not do ("a PhotoDNA hash can be used to produce thumbnail-quality reproductions of the original image") and it looks scientific, too.
ogurechny 2y ago
Secretive, unaccountable, uncontrollably expanding, driven by shady “independent” NGO which is in fact completely in bed with certain branches of government. Russian DPI censorship system is really like that... Oh, I'm sorry, we're discussing something that is a decade older, and belongs to a “free world”, which is a completely different thing. These things are simply selling their services to the highest bidder. It's a business model on a power connections market. They are made to be offered to, and controlled by, entities that enjoy having such tools. Sometimes they are also offered to smaller fish, like media corporations, to hurt competitors (pirates and foreign services). Also, social media corporations can proudly state that they themselves “censor nothing”, because it's outsourced. There's a great portrayal of people who run such services: https://www.newyorker.com/magazine/2019/11/04/a-cybersecurity-firms-sharp-rise-and-stunning-collapse https://www.newyorker.com/magazine/2019/11/04/a-cybersecurit...
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]

Is there any good article/paper that describes how it actually works or is implemented not just in high-level and hand-waving terms?

unsnap_biceps 2y ago
https://hackerfactor.com/blog/index.php?/archives/931-PhotoDNA-and-Limitations.html https://hackerfactor.com/blog/index.php?/archives/931-PhotoD... is linked from https://anishathalye.com/inverting-photodna/ https://anishathalye.com/inverting-photodna/ and is likely a good starting point
- Bayes7 2y ago
  thanks!
- Springtime 2y ago
  Yeah Hacker Factor's multi-post critiques are where I first saw it analyzed. For reference they run the popular fotoforensics.com image analysis site. They also have scathing critique (eg [1]) about the Adobe-led C2PA digital provenance signing, having themselves been part of various groups that seek solutions to the provenance problem. [1] https://www.hackerfactor.com/blog/index.php?/archives/1013-C2PAs-Worst-Case-Scenario.html https://www.hackerfactor.com/blog/index.php?/archives/1013-C...
majke 2y ago
The secrecy of the inner tech is intentional.
bawolff 2y ago
There tends to be more information under the search term "perceptual hashing"

noduerme 2y ago

What's novel about this? iirc, Apple withdrew its plan to hash photos client-side a couple years ago after an outcry. Dropbox has been hashing every file forever to save storage space. Store your shit with a cloud provider, expect it to get scanned, right? Also, there are a million cute methods to make two different photos produce the same hash; that was actually what the outcry about Apple's version was about. The more the hash algorithm tried to produce the same hash for different variants of a photo, the more likely it was that someone could get their hands on a flagged hash and theoretically send you an innocuous looking photo that registered as CSAM. Pretty sure that's why Apple pulled it.

kalleboo 2y ago
> theoretically send you an innocuous looking photo that registered as CSAM. Pretty sure that's why Apple pulled it. Apple also had human reviewers in the mix. The only reason they pulled it was PR/optics.
- zaptrem 2y ago
  Wouldn't that require exfiltrating the original photo? I remember them swearing that wasn't part of the deal.
  - jchw 2y ago
    Going from memory here but IIRC the deal was that on device they'd produce a hash using a known pHash, and if that was positive, they'd send the photo to check it against a second pHash that wasn't publicly-disclosed (to try to mitigate the problem of intentional collisions) and then if both of them were positive matches, they would have human reviewers in the loop.
    - nullc 2y ago
      Not "if then send" -- if that were the case you could detect that your images were matching and then use the system as an oracle to e.g. detect that the database might contain disfavored political content in addition to whatever they claim it contains. The system was proposed as part of the upload process to your private encrypted cloud storage. So they already have the files. The system was designed so that if there were a sufficient number of hits against an encrypted database then a party possessing a particular private key would learn the private key for the encrypted files, as a passive effect of there being matches. Like if I mutter random letters of my password while entering it, you'd eventually passively learn my password just by hanging around while I logged in each morning. This way any matching is completely undetectable on the users end, at least until the private key possessing parties or the people they share the information with choose to take some action against the user.

zaptrem 2y ago

https://anishathalye.com/inverting-photodna/ https://anishathalye.com/inverting-photodna/ You could almost certainly produce nearly photo realistic PhotoDNA inversions with a finetuned diffusion model now. Is it possible to create a perceptual hashing algorithm where this isn't possible?

tashbarg 2y ago
You can certainly not produce inversions. The data that is left in the hash is not enough to produce anything vaguely photorealistic. However, you can fill the gaps and generate photorealistic photos that fit to the extremely reduced information you get from the hash. You are generating believable (as defined by the training data) photos that fit the hash. That’s a huge difference. Statements like yours are extremely dangerous. Without proper understanding of what GenAI can and can not do, people start relying on things that are not there. Imagine your photorealistic inversion AI putting a mole or a wrinkle in the face of somebody without any foundation in the actual hash. Just because it fits better to the trained data. Explain that to the judge, when the person with just the right facial features sits in front of them.
- jasonvorhe 2y ago
  This modern narrative of people posting their opinions or assumptions somewhere being "dangerous" because someone could just believe it is much more dangerous because it can be applied to any opinion anywhere that was ever published. No judge will ever rule on something based on a comment they read in the Internet.
  - tashbarg 2y ago
    Judges usually rely on experts in forensic science who, of course, are infallible and absolutely not influenced by what they read online during their day. https://innocenceproject.org/misapplication-of-forensic-science/ https://innocenceproject.org/misapplication-of-forensic-scie... It is dangerous to push the narrative that GenAI can "put information back" where it was once removed. Especially dangerous, because most GenAI is built to put something there that is extremely believable. And while an innocent comment on HN might not play the biggest role, the linked project claims exactly what it can - by definition - not do ("a PhotoDNA hash can be used to produce thumbnail-quality reproductions of the original image") and it looks scientific, too.

ogurechny 2y ago

Secretive, unaccountable, uncontrollably expanding, driven by shady “independent” NGO which is in fact completely in bed with certain branches of government. Russian DPI censorship system is really like that... Oh, I'm sorry, we're discussing something that is a decade older, and belongs to a “free world”, which is a completely different thing. These things are simply selling their services to the highest bidder. It's a business model on a power connections market. They are made to be offered to, and controlled by, entities that enjoy having such tools. Sometimes they are also offered to smaller fish, like media corporations, to hurt competitors (pirates and foreign services). Also, social media corporations can proudly state that they themselves “censor nothing”, because it's outsourced. There's a great portrayal of people who run such services: https://www.newyorker.com/magazine/2019/11/04/a-cybersecurity-firms-sharp-rise-and-stunning-collapse https://www.newyorker.com/magazine/2019/11/04/a-cybersecurit...

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]