3 ms·

It’s not about looking ‘kind of similar’. The point is to match images that have been resized or had contrast enhanced etc. That’s all.

by zepto 5y ago

heavyset_go 5y ago
That's literally how perceptual hashes work. Two images that look similar to each other will have the same, or similar, hashes[1][2]. [1] https://news.ycombinator.com/item?id=28091750 https://news.ycombinator.com/item?id=28091750 [2] https://news.ycombinator.com/item?id=28110159 https://news.ycombinator.com/item?id=28110159
- zepto 5y ago
  Yes, it will match two images that look the same even after minor transformations. That isn’t the same as saying it will match things that look kind of the same.
  - heavyset_go 5y ago
    That's a distinction with no functional difference. Two images that were modified from a source image will sometimes kind of look like one another, and so will two images that coincidentally look like one another. The first two will have similar or the same hashes, as will the latter two. That's how you get false positives. And it's incorrect. The way perceptual hashing works is that an image is shrunk down to a 8x8 or 26x26 etc image and then transformations are applied to them to exaggerate features. If two images look kind of the same when shrunken down, they will have the same or similar hashes. If two images kind of look the same when shrunken down, then their parent images will also kind of look the same. Please read the OPs of the two links I posted. They're both from people who work in this field. The latter link is from someone[2] who invented many perceptual hashing methods himself that are used widely across the industry. Both articles touch on this subject, and the first[1] one includes two photo examples. I have built products using these methods, and what is said by these two experts matches my experiences. [1] https://rentafounder.com/the-problem-with-perceptual-hashes/ https://rentafounder.com/the-problem-with-perceptual-hashes/ [2] https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
    - zepto 5y ago
      It’s not a distinction without a difference. ‘Kind of like’ is going to be read by most people as ‘easily fooled’. That is a reflection of the false positive rate. It all depends on the quality and tuning of the algorithm and how it is deployed. If you are going to imply false positives are common, then you need to back it up. Nobody is saying false positives are impossible. Apple is saying false positives are on the order of one in a trillion per user account per year. That doesn’t sound like something that matches images that are only ‘kind of similar’. Yes - cryptographic hashes have much lower false positives rates even than that, but that is a distinction without a difference since both make the risk negligible. > The way perceptual hashing works is that an image is shrunk down to a 8x8 or 26x26 image and then Which is it for Apple’s hashes? There is no point in reading old articles about perceptual hashes if the conclusions don’t apply to Apple’s neuralhash algorithm. If they don’t then reading about other hashes is just a distraction. What can you tell us about the likelyhood of Apple’s hashes to create false positives?
      - heavyset_go 5y ago
        > It’s not a distinction without a difference. ‘Kind of like’ is going to be read by most people as ‘easily fooled’. I'm not really concerned with what you're afraid most people will think. Two images that kind of look like one another will have the same or similar hashes. There are literal examples of this in the links I posted above. And it's literally the point of perceptual hashing, to find images that look similar to a source image by comparing hash similarity. > If you are going to imply false positives are common, then you need to back it up. I just did with two links I posted above. Twice. > Apple is saying false positives are on the order of one in a trillion per user account per year. Sounds like a claim that wasn't replicated or independently verified. Of course Apple is going to say their system is nearly perfect, that's what all companies do. The onus is on Apple to prove that their marketing claims reflect reality. > There is no point in reading old articles about perceptual hashes if the conclusions don’t apply to Apple’s neuralhash algorithm. If they don’t then reading about other hashes is just a distraction. The onus is on Apple to demonstrate that their methods are remarkably different from the rest of the science and industry. This is like saying the normal principles of computing don't apply to new Apple products because they might have invented a new brand computing paradigm that isn't anything like any classical or quantum computer mentioned in scientific literature at all. Yeah, maybe they did, but it's unlikely and the onus is on Apple to prove it.
shuckles 5y ago
In particular, false positives with perceptual hashes are not because they are similar in semantic content but because they are similar in whatever features the neural network determined stay stable across transformations. Your fall colors landscape photo is just as liable to be a neural hash match as your college sweetheart’s nudes.
- zepto 5y ago
  Right - which in this case are protected against by the visual derivative.

It’s not about looking ‘kind of similar’. The point is to match images that have been resized or had contrast enhanced etc. That’s all.

by zepto 5y ago

It’s not about looking ‘kind of similar’. The point is to match images that have been resized or had contrast enhanced etc. That’s all.

heavyset_go 5y ago

That's literally how perceptual hashes work. Two images that look similar to each other will have the same, or similar, hashes[1][2]. [1] https://news.ycombinator.com/item?id=28091750 https://news.ycombinator.com/item?id=28091750 [2] https://news.ycombinator.com/item?id=28110159 https://news.ycombinator.com/item?id=28110159

zepto 5y ago
Yes, it will match two images that look the same even after minor transformations. That isn’t the same as saying it will match things that look kind of the same.
- heavyset_go 5y ago
  That's a distinction with no functional difference. Two images that were modified from a source image will sometimes kind of look like one another, and so will two images that coincidentally look like one another. The first two will have similar or the same hashes, as will the latter two. That's how you get false positives. And it's incorrect. The way perceptual hashing works is that an image is shrunk down to a 8x8 or 26x26 etc image and then transformations are applied to them to exaggerate features. If two images look kind of the same when shrunken down, they will have the same or similar hashes. If two images kind of look the same when shrunken down, then their parent images will also kind of look the same. Please read the OPs of the two links I posted. They're both from people who work in this field. The latter link is from someone[2] who invented many perceptual hashing methods himself that are used widely across the industry. Both articles touch on this subject, and the first[1] one includes two photo examples. I have built products using these methods, and what is said by these two experts matches my experiences. [1] https://rentafounder.com/the-problem-with-perceptual-hashes/ https://rentafounder.com/the-problem-with-perceptual-hashes/ [2] https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
  - zepto 5y ago
    It’s not a distinction without a difference. ‘Kind of like’ is going to be read by most people as ‘easily fooled’. That is a reflection of the false positive rate. It all depends on the quality and tuning of the algorithm and how it is deployed. If you are going to imply false positives are common, then you need to back it up. Nobody is saying false positives are impossible. Apple is saying false positives are on the order of one in a trillion per user account per year. That doesn’t sound like something that matches images that are only ‘kind of similar’. Yes - cryptographic hashes have much lower false positives rates even than that, but that is a distinction without a difference since both make the risk negligible. > The way perceptual hashing works is that an image is shrunk down to a 8x8 or 26x26 image and then Which is it for Apple’s hashes? There is no point in reading old articles about perceptual hashes if the conclusions don’t apply to Apple’s neuralhash algorithm. If they don’t then reading about other hashes is just a distraction. What can you tell us about the likelyhood of Apple’s hashes to create false positives?
    - heavyset_go 5y ago
      > It’s not a distinction without a difference. ‘Kind of like’ is going to be read by most people as ‘easily fooled’. I'm not really concerned with what you're afraid most people will think. Two images that kind of look like one another will have the same or similar hashes. There are literal examples of this in the links I posted above. And it's literally the point of perceptual hashing, to find images that look similar to a source image by comparing hash similarity. > If you are going to imply false positives are common, then you need to back it up. I just did with two links I posted above. Twice. > Apple is saying false positives are on the order of one in a trillion per user account per year. Sounds like a claim that wasn't replicated or independently verified. Of course Apple is going to say their system is nearly perfect, that's what all companies do. The onus is on Apple to prove that their marketing claims reflect reality. > There is no point in reading old articles about perceptual hashes if the conclusions don’t apply to Apple’s neuralhash algorithm. If they don’t then reading about other hashes is just a distraction. The onus is on Apple to demonstrate that their methods are remarkably different from the rest of the science and industry. This is like saying the normal principles of computing don't apply to new Apple products because they might have invented a new brand computing paradigm that isn't anything like any classical or quantum computer mentioned in scientific literature at all. Yeah, maybe they did, but it's unlikely and the onus is on Apple to prove it.

shuckles 5y ago

In particular, false positives with perceptual hashes are not because they are similar in semantic content but because they are similar in whatever features the neural network determined stay stable across transformations. Your fall colors landscape photo is just as liable to be a neural hash match as your college sweetheart’s nudes.

zepto 5y ago
Right - which in this case are protected against by the visual derivative.