4 ms·

I've never used it, but apparently you can hash a image in such a way that small changes will result in the same hash. I think that is what Apple was doing with

I've never used it, but apparently you can hash a image in such a way that small changes will result in the same hash. I think that is what Apple was doing with it's child abuse filter announced earlier this year.

mannykannot 5y ago
In a sense, image recognition is like that - multiple different images hash to "cat", for example. For the purpose here, robust hashing seems to be the relevant term: https://www.netclean.com/netclean-report-2019/robust-hashing/ https://www.netclean.com/netclean-report-2019/robust-hashing...
shakna 5y ago
You're probably thinking of a perceptual hash. [0] It's a relatively new field, academically speaking, so only a few players dominate, and there is a huge amount of research going into it. There are other techniques for automatic flagging of similar files, like PhotoDNA's [1] enormous database - which I believe PornHub are already using. [0] https://en.wikipedia.org/wiki/Perceptual_hashing https://en.wikipedia.org/wiki/Perceptual_hashing [1] https://www.microsoft.com/en-us/photodna https://www.microsoft.com/en-us/photodna
- toomuchtodo 5y ago
  It seems like a natural evolution of PhotoDNA to have this content (non consensual adult content) flagged programmatically, with a flow for folks to attest to content to be fingerprinted and scanned for.
tomatotomato37 5y ago
I don't know if perpetual hashes will work with full videos though, and linking the hash to a singular frame seems very unreliable, especially considering how randomly lossy video compression is
- shakna 5y ago
  Most video-based perceptual hashes work by taking a series of thumbnails at set intervals, say every 5% of the video, assembling a collage, and then making the perceptual hash against the collage. As you note, it isn't particularly reliable, but it does actually tend to work fairly well against the loss of video compression. However, it doesn't tend to work well when the video is recut, with time added or removed. There is some research trying to combine approaches for finding important keyframes (ML-powered thumbnail searching) with perceptual hashing [0], which is looking promising, but it's still fairly unreliable. [0] https://doi.org/10.1007/s10207-013-0211-z https://doi.org/10.1007/s10207-013-0211-z
acchow 5y ago
> I've never used it, but apparently you can hash a image in such a way that small changes will result in the same hash This is equivalent to image recognition. It works ok, but has lots of false positives and false negatives.

mannykannot 5y ago

In a sense, image recognition is like that - multiple different images hash to "cat", for example. For the purpose here, robust hashing seems to be the relevant term: https://www.netclean.com/netclean-report-2019/robust-hashing/ https://www.netclean.com/netclean-report-2019/robust-hashing...

shakna 5y ago

You're probably thinking of a perceptual hash. [0] It's a relatively new field, academically speaking, so only a few players dominate, and there is a huge amount of research going into it. There are other techniques for automatic flagging of similar files, like PhotoDNA's [1] enormous database - which I believe PornHub are already using. [0] https://en.wikipedia.org/wiki/Perceptual_hashing https://en.wikipedia.org/wiki/Perceptual_hashing [1] https://www.microsoft.com/en-us/photodna https://www.microsoft.com/en-us/photodna

toomuchtodo 5y ago
It seems like a natural evolution of PhotoDNA to have this content (non consensual adult content) flagged programmatically, with a flow for folks to attest to content to be fingerprinted and scanned for.

tomatotomato37 5y ago

I don't know if perpetual hashes will work with full videos though, and linking the hash to a singular frame seems very unreliable, especially considering how randomly lossy video compression is

shakna 5y ago
Most video-based perceptual hashes work by taking a series of thumbnails at set intervals, say every 5% of the video, assembling a collage, and then making the perceptual hash against the collage. As you note, it isn't particularly reliable, but it does actually tend to work fairly well against the loss of video compression. However, it doesn't tend to work well when the video is recut, with time added or removed. There is some research trying to combine approaches for finding important keyframes (ML-powered thumbnail searching) with perceptual hashing [0], which is looking promising, but it's still fairly unreliable. [0] https://doi.org/10.1007/s10207-013-0211-z https://doi.org/10.1007/s10207-013-0211-z

acchow 5y ago

> I've never used it, but apparently you can hash a image in such a way that small changes will result in the same hash This is equivalent to image recognition. It works ok, but has lots of false positives and false negatives.