4 ms·

Apple’s documents said they require multiple hits before anything happens, as the article notes. They can (and have) adjusted that number to any desired balance

Apple’s documents said they require multiple hits before anything happens, as the article notes. They can (and have) adjusted that number to any desired balance of false positive to negatives. How can they say it’s 1 in a trillion? You test the algorithm on a bunch of random negatives, see how many positives you get, and do one division and one multiplication. This isn’t rocket science. So, while there are many arguments against this program, this isn’t it. It’s also somewhat strange to believe the idea of collisions in hashes of far smaller size than the images they are run on somehow escaped Apple and/or really anyone mildly competent.

bt1a 5y ago
That would not be a good way to arrive at an accurate estimate. Would you not need dozens of trillions of photos to begin with in order to get an accurate estimate when the occurrence rate is so small?
- KarlKemp 5y ago
  What? No... Or, more accurately: if you need "dozens of trillions" that implies a false positive rate so low, it's practically of no concern. You'd want to look up the poisson distribution for this. But, to get at this intuitively: say you have a bunch of eggs, some of which may be spoiled. How many would you have to crack open, to get a meaningful idea of how many are still fine, and how many are not? The absolute number depends on the fraction that are off. But independent of that, you'd usually start trusting your sample when you've seen 5 to 10 spoiled ones. So Apple runs the hash algorithm on random photos. They find 20 false positives in the first ten million. Given that error rate, how many positives would it require for the average photo collection of 10,000 to be certain at at 1:a trillion level that it's not just coincidence? Throw it into, for example, https://keisan.casio.com/exec/system/1180573179 https://keisan.casio.com/exec/system/1180573179 with lambda = 0.2 (you're expecting one false positive for every 50,000 at the error rate we assumed, or 0.2 for 10,000), and n = 10 (we've found 10 positives in this photo library) to see the chances of that, 2.35x10^-14, or 2.35 / 100 trillion.
fogof 5y ago
I was unhappy to find this comment so far down and even unhappier to see it downvoted. I'm not a fan of the decrease in privacy Apple is creating with this move but I think this forum has gotten its feelings for Apple caught up with its response to a completely valid criticism of an anti-Apple article. To explain things even further, let's say that the perceptual algorithm makes a false positive 1% of the time. That is, 1 in every 100 completely normal pictures are incorrectly matched with some picture in the child pornography database. There's no reason to think (at least none springs to mind, happy to hear suggestions) that a false positive in one image will make it any more likely to see a false positive in another image. Thus, if you have a phone with 1000 pictures on it, and it takes 40 trigger a match, there's less than a 1 in a trillion probability that this would happen if the pictures are all normal.
- IfOnlyYouKnew 5y ago
  At this point, the COVID vaccines seem to barely have majority support on HN, and “cancel culture” would win any survey on our times’ top problems, beating “women inventing stories of rape’ and “the black guy mentioning something borderline political at work, just because he’s paid 5/8th as much as others”. An inability to follow even the most elementary argument from statistics isn’t really surprising. Although I can’t quite say if it’s actual inability, or follows from the fact that it supports the wrong outcome.

bt1a 5y ago

That would not be a good way to arrive at an accurate estimate. Would you not need dozens of trillions of photos to begin with in order to get an accurate estimate when the occurrence rate is so small?

KarlKemp 5y ago
What? No... Or, more accurately: if you need "dozens of trillions" that implies a false positive rate so low, it's practically of no concern. You'd want to look up the poisson distribution for this. But, to get at this intuitively: say you have a bunch of eggs, some of which may be spoiled. How many would you have to crack open, to get a meaningful idea of how many are still fine, and how many are not? The absolute number depends on the fraction that are off. But independent of that, you'd usually start trusting your sample when you've seen 5 to 10 spoiled ones. So Apple runs the hash algorithm on random photos. They find 20 false positives in the first ten million. Given that error rate, how many positives would it require for the average photo collection of 10,000 to be certain at at 1:a trillion level that it's not just coincidence? Throw it into, for example, https://keisan.casio.com/exec/system/1180573179 https://keisan.casio.com/exec/system/1180573179 with lambda = 0.2 (you're expecting one false positive for every 50,000 at the error rate we assumed, or 0.2 for 10,000), and n = 10 (we've found 10 positives in this photo library) to see the chances of that, 2.35x10^-14, or 2.35 / 100 trillion.

fogof 5y ago

I was unhappy to find this comment so far down and even unhappier to see it downvoted. I'm not a fan of the decrease in privacy Apple is creating with this move but I think this forum has gotten its feelings for Apple caught up with its response to a completely valid criticism of an anti-Apple article. To explain things even further, let's say that the perceptual algorithm makes a false positive 1% of the time. That is, 1 in every 100 completely normal pictures are incorrectly matched with some picture in the child pornography database. There's no reason to think (at least none springs to mind, happy to hear suggestions) that a false positive in one image will make it any more likely to see a false positive in another image. Thus, if you have a phone with 1000 pictures on it, and it takes 40 trigger a match, there's less than a 1 in a trillion probability that this would happen if the pictures are all normal.

IfOnlyYouKnew 5y ago
At this point, the COVID vaccines seem to barely have majority support on HN, and “cancel culture” would win any survey on our times’ top problems, beating “women inventing stories of rape’ and “the black guy mentioning something borderline political at work, just because he’s paid 5/8th as much as others”. An inability to follow even the most elementary argument from statistics isn’t really surprising. Although I can’t quite say if it’s actual inability, or follows from the fact that it supports the wrong outcome.