5 ms·

I suspect that's why they have some threshold that moves the false positive rate to one in one trillion. The iMessage bit is different - it's only on device, o

I suspect that's why they have some threshold that moves the false positive rate to one in one trillion. The iMessage bit is different - it's only on device, only on child accounts, and only alerts parents. It's more akin to a parental control feature than anything else.

bnj 5y ago
This is a great point, thanks for adding that detail.
samatman 5y ago
It was pretty dumb of Apple to announce both of these things at the same time. They have nothing to do with each other, and I've seen a dozen people on HN confuse them. If the message is getting muddled here, it will be hopelessly conflated in less technical circles. I'm concerned and upset about the CSAM filter for all the reasons that keep hitting the front page, but don't care about the opt-in parental controls at all, and if I had kids, I might want them. But if I thought the CSAM filter worked like the nudie-detector filter, I'd be wigging out.
- deleted 5y ago
  [deleted]
lamontcg 5y ago
It doesn't matter if the cryptographic algorithm is one in a trillion. If there's a concurrency bug in the application which applies the algorithm the wrong account could be flagged against a legitimate image in the database. If the human involved overly trusts the system they may not validate against the actual file in the users account, or may assume the user deleted it somehow and figure its "safter" to let law enforcement figure it out. Bugs in the application of the code, combined with human complacency and mistakes can certainly lead to errors, even if the cryptographic algorithm itself was perfect. We really need to bring back comp.risks
skinkestek 5y ago
> I suspect that's why they have some threshold that moves the false positive rate to one in one trillion. So now instead of sending just one nice innocent very high resolution images of "Tokyo City" or something with something horrific hidden somewhere you have to send a few such images. That is reassuring. I can never believe anyone except me will think about that. (If the system is too dumb to detect this it is worthless, and if it is smart enough this opens the floodgates for anyone wanting to make trouble for just about anyone.)
- nomel 5y ago
  This would require multiple hash collisions.
  - skinkestek 5y ago
    No need for hash collisions, that's what makes it so bad: just edit multiple known verybad images in multiple nice high res photos like I tried to explain above.
    - nomel 5y ago
      It’s a visual hash, not something like an md5. And now you’re talking of a directed attack to intentionally make hash collisions?

bnj 5y ago

This is a great point, thanks for adding that detail.

samatman 5y ago

It was pretty dumb of Apple to announce both of these things at the same time. They have nothing to do with each other, and I've seen a dozen people on HN confuse them. If the message is getting muddled here, it will be hopelessly conflated in less technical circles. I'm concerned and upset about the CSAM filter for all the reasons that keep hitting the front page, but don't care about the opt-in parental controls at all, and if I had kids, I might want them. But if I thought the CSAM filter worked like the nudie-detector filter, I'd be wigging out.

deleted 5y ago
[deleted]

lamontcg 5y ago

It doesn't matter if the cryptographic algorithm is one in a trillion. If there's a concurrency bug in the application which applies the algorithm the wrong account could be flagged against a legitimate image in the database. If the human involved overly trusts the system they may not validate against the actual file in the users account, or may assume the user deleted it somehow and figure its "safter" to let law enforcement figure it out. Bugs in the application of the code, combined with human complacency and mistakes can certainly lead to errors, even if the cryptographic algorithm itself was perfect. We really need to bring back comp.risks

skinkestek 5y ago

> I suspect that's why they have some threshold that moves the false positive rate to one in one trillion. So now instead of sending just one nice innocent very high resolution images of "Tokyo City" or something with something horrific hidden somewhere you have to send a few such images. That is reassuring. I can never believe anyone except me will think about that. (If the system is too dumb to detect this it is worthless, and if it is smart enough this opens the floodgates for anyone wanting to make trouble for just about anyone.)

nomel 5y ago
This would require multiple hash collisions.
- skinkestek 5y ago
  No need for hash collisions, that's what makes it so bad: just edit multiple known verybad images in multiple nice high res photos like I tried to explain above.
  - nomel 5y ago
    It’s a visual hash, not something like an md5. And now you’re talking of a directed attack to intentionally make hash collisions?