3 ms·

> "The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it." This ca

> "The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it." This can be abused to spam Apple's manual review process, grinding it down to a halt. You've cost Apple time and money by making them review each such fake report.

incrudible 5y ago
> You've cost Apple time and money by making them review each such fake report. Ok, but… how do I profit? If I wanted to waste Apple employee time, I could surely find a way to do it, but why would I? The functioning of society relies on the fact that people generally have better things to do than waste each others time.
- playguardin 5y ago
  You have justified the review of an "innocent" image that may be of interest. To justify investigations and whatever else. It's pretty easy to understand... Big tech bad mmmkay!?
- stemlord 5y ago
  It's a form of protest, thus not seen as a waste of time by those doing it.
- syshum 5y ago
  >>Ok, but… how do I profit? To some people the ethical problems with this system are worth the effort to make their implementation impossible. if you are asking for monetary profit, that does not have to be a goal for someone to want to engage in disruption of this service.
- incompatible 5y ago
  Perhaps somebody could make unflaggable CSAM images by giving them the same hashes as very widely available images.
  - BizarroLand 5y ago
    Or you could identify the factors that cause a hash to be computed and then start generating random images that compute to the same hash, creating 10s of thousands of images of digital noise that all look alike to the computer.
- perryizgr8 5y ago
  > but why would I? For the lulz.
ec109685 5y ago
It can’t be. There’s a different private hash function that also has to match that particular csam image’s hash value before a human sees it. An adversarial attack can’t produce that one since the expected value isn’t known.
- GistNoesis 5y ago
  This second "secret" hash function, because it is applied to raw offensive content that Apple can't have, has to be shared at least with people maintaining the CSAM database. You can't rely that it won't ever leak, and when it does, it will be almost undetectable and have huge consequences. As soon as the first on-device CSAM flag has been raised, it becomes a legal and political problem. Even without a second matching hash, it already put Apple in an untenable position. They already are in a mud fight with the pigs. They can't say : we got 100M hits this month on our first CSAM filter but we only reported 10 cases, because to avoid false positives our second filter throw everything to dev/null, and we didn't even manually reviewed them because your privacy matter to us. It has become a political problem where for good measure they will have to report cases to make the numbers look "good". Attackers of the system can also plant false negatives aka real CSAM that has been modified enough to pass the first hash but fail this second hash. So that, in the audit, independent security researchers who review Apple system, will be able to say that Apple automated system, sided with the bad guys, by rejecting true CSAM and not reporting it. Also remember, that Apple can also do something else than what they say they do for PR reasons : maybe some secret law will force them to reveal to the authorities as soon as the first flag has been raised, and force them not tell about it. And because it's in the name of fighting the "bad guys", that's something most people expect them to do. From the user perspective, there is nothing we can audit, it's all security by obscurity disguised with pseudo-crypto-PR, it's just a big "Trust us" blanked signed paper that will soon be used to dragnet surveil anyone for any content.

incrudible 5y ago

> You've cost Apple time and money by making them review each such fake report. Ok, but… how do I profit? If I wanted to waste Apple employee time, I could surely find a way to do it, but why would I? The functioning of society relies on the fact that people generally have better things to do than waste each others time.

playguardin 5y ago
You have justified the review of an "innocent" image that may be of interest. To justify investigations and whatever else. It's pretty easy to understand... Big tech bad mmmkay!?
stemlord 5y ago
It's a form of protest, thus not seen as a waste of time by those doing it.
syshum 5y ago
>>Ok, but… how do I profit? To some people the ethical problems with this system are worth the effort to make their implementation impossible. if you are asking for monetary profit, that does not have to be a goal for someone to want to engage in disruption of this service.
incompatible 5y ago
Perhaps somebody could make unflaggable CSAM images by giving them the same hashes as very widely available images.
- BizarroLand 5y ago
  Or you could identify the factors that cause a hash to be computed and then start generating random images that compute to the same hash, creating 10s of thousands of images of digital noise that all look alike to the computer.
perryizgr8 5y ago
> but why would I? For the lulz.

ec109685 5y ago

It can’t be. There’s a different private hash function that also has to match that particular csam image’s hash value before a human sees it. An adversarial attack can’t produce that one since the expected value isn’t known.

GistNoesis 5y ago
This second "secret" hash function, because it is applied to raw offensive content that Apple can't have, has to be shared at least with people maintaining the CSAM database. You can't rely that it won't ever leak, and when it does, it will be almost undetectable and have huge consequences. As soon as the first on-device CSAM flag has been raised, it becomes a legal and political problem. Even without a second matching hash, it already put Apple in an untenable position. They already are in a mud fight with the pigs. They can't say : we got 100M hits this month on our first CSAM filter but we only reported 10 cases, because to avoid false positives our second filter throw everything to dev/null, and we didn't even manually reviewed them because your privacy matter to us. It has become a political problem where for good measure they will have to report cases to make the numbers look "good". Attackers of the system can also plant false negatives aka real CSAM that has been modified enough to pass the first hash but fail this second hash. So that, in the audit, independent security researchers who review Apple system, will be able to say that Apple automated system, sided with the bad guys, by rejecting true CSAM and not reporting it. Also remember, that Apple can also do something else than what they say they do for PR reasons : maybe some secret law will force them to reveal to the authorities as soon as the first flag has been raised, and force them not tell about it. And because it's in the name of fighting the "bad guys", that's something most people expect them to do. From the user perspective, there is nothing we can audit, it's all security by obscurity disguised with pseudo-crypto-PR, it's just a big "Trust us" blanked signed paper that will soon be used to dragnet surveil anyone for any content.