5 ms·

Apple’s system design does not easily extend to checking vague subject matter. Every step of the process is tied to hashes of specific photographs. And do you

Apple’s system design does not easily extend to checking vague subject matter. Every step of the process is tied to hashes of specific photographs. And do you seriously think they didn’t check the legality of what they built? Really?

mrathys 5y ago
That is not correct. They are algorithms that detect photos that resemble specific photographs.
- robertoandred 5y ago
  “Resemble”, in the same way that a cropped photo resembles the uncropped original.
  - simion314 5y ago
    And in the same way a human resembles a butterfly. didn't you learn from the case where algorithms matched white noise with some copyright material? There are many iPhones and many images in each iPhones, there will be false matches. Add on top of that that the database of hashes is secret, that it can be updated in secret, that the algorithm is secret and the "threshold" is also secret and you have a lot of suspicion from people.
    - robertoandred 5y ago
      No, not in the way a human resembles a butterfly. But for the sake of argument, fine: some pictures of you look exactly like pictures of a butterfly, enough to trigger the flagging threshold. These false positives would easily be caught during the review and nothing would happen.
      - simion314 5y ago
        I do not trust the review people, they are not even Apple employees, they are cheap contractors that hire cheap labor,this people are treated like crap so I can see them making mistakes , is not like you never heard of Apple review guy decided X and when other guy checked he decided !X .
TimTheTinker 5y ago
> specific photographs That's the catch. Nothing in the system design prevents them from adding hashes of other types of photographs to that database. > And do you seriously think they didn’t check the legality of what they built? IANAL, but the law clearly states that transmitting CSAM to any party other than NCMEC is a felony. Apple != NCMEC. More info: https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
- robertoandred 5y ago
  And the manual review would catch other types of photographs. And the law clearly states that that doesn’t apply if an immediate effort is made to involve law enforcement. Besides, Apple is not transmitting it to Apple, the user is.
- yub4578 5y ago
  “IANAL” is used to avoid liability for offering legal advice in settings where it’s not permitted. It’s not used to hand wave away that you don’t know what you’re talking about. The very same law you’re citing describes, in detail, the good faith diligence process that requires the service provider to verify the suspected material before transmission to NCMEC. But no, some random blog and you, you two have a handle on legal analysis that the most litigiously sensitive entity on Earth must have missed while designing one of the most litigiously sensitive systems ever fielded by humans. How’d they miss felony criminal liability, right? It’s just too easy to overlook while designing a system whose sole purpose is to gather legally actionable evidence against other people. As someone who’s built these systems for over a decade, it’s remarkable how one Apple press release can make everyone so hopelessly uninformed and confident that they know the score. Nobody used the acronym CSAM until a week ago except for people like me, those of us haunted by (actual) nightmares of this shit while HN distantly pontificates on the apparent sacrilege of MD5ing your photos of a family vacation to Barbados to see if you happen to be sharing images of children being raped. Nobody commenting on this has ever seen child pornography. I’d take that to the bank. Did you know the organized outfits design well-polished sites like PornHub, complete with React and a design palette? 35 thumbnails of different seven year olds right on the front page, filterable by sexual situation. Filterable by how many adults are involved. With a comment section, even!, and God help you if you even begin to imagine what is said there. You’re right, though, let’s think about your privacy and the criminal liability for Apple for taking action on something that clearly doesn’t matter to anyone except those stuck with dealing with it. Get real. Sometimes the lack of perspective among otherwise smart people really worries me, and this conversation about Apple’s motives for the last week or so has worried me the most yet.

mrathys 5y ago

That is not correct. They are algorithms that detect photos that resemble specific photographs.

robertoandred 5y ago
“Resemble”, in the same way that a cropped photo resembles the uncropped original.
- simion314 5y ago
  And in the same way a human resembles a butterfly. didn't you learn from the case where algorithms matched white noise with some copyright material? There are many iPhones and many images in each iPhones, there will be false matches. Add on top of that that the database of hashes is secret, that it can be updated in secret, that the algorithm is secret and the "threshold" is also secret and you have a lot of suspicion from people.
  - robertoandred 5y ago
    No, not in the way a human resembles a butterfly. But for the sake of argument, fine: some pictures of you look exactly like pictures of a butterfly, enough to trigger the flagging threshold. These false positives would easily be caught during the review and nothing would happen.
    - simion314 5y ago
      I do not trust the review people, they are not even Apple employees, they are cheap contractors that hire cheap labor,this people are treated like crap so I can see them making mistakes , is not like you never heard of Apple review guy decided X and when other guy checked he decided !X .

TimTheTinker 5y ago

> specific photographs That's the catch. Nothing in the system design prevents them from adding hashes of other types of photographs to that database. > And do you seriously think they didn’t check the legality of what they built? IANAL, but the law clearly states that transmitting CSAM to any party other than NCMEC is a felony. Apple != NCMEC. More info: https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...

robertoandred 5y ago
And the manual review would catch other types of photographs. And the law clearly states that that doesn’t apply if an immediate effort is made to involve law enforcement. Besides, Apple is not transmitting it to Apple, the user is.
yub4578 5y ago
“IANAL” is used to avoid liability for offering legal advice in settings where it’s not permitted. It’s not used to hand wave away that you don’t know what you’re talking about. The very same law you’re citing describes, in detail, the good faith diligence process that requires the service provider to verify the suspected material before transmission to NCMEC. But no, some random blog and you, you two have a handle on legal analysis that the most litigiously sensitive entity on Earth must have missed while designing one of the most litigiously sensitive systems ever fielded by humans. How’d they miss felony criminal liability, right? It’s just too easy to overlook while designing a system whose sole purpose is to gather legally actionable evidence against other people. As someone who’s built these systems for over a decade, it’s remarkable how one Apple press release can make everyone so hopelessly uninformed and confident that they know the score. Nobody used the acronym CSAM until a week ago except for people like me, those of us haunted by (actual) nightmares of this shit while HN distantly pontificates on the apparent sacrilege of MD5ing your photos of a family vacation to Barbados to see if you happen to be sharing images of children being raped. Nobody commenting on this has ever seen child pornography. I’d take that to the bank. Did you know the organized outfits design well-polished sites like PornHub, complete with React and a design palette? 35 thumbnails of different seven year olds right on the front page, filterable by sexual situation. Filterable by how many adults are involved. With a comment section, even!, and God help you if you even begin to imagine what is said there. You’re right, though, let’s think about your privacy and the criminal liability for Apple for taking action on something that clearly doesn’t matter to anyone except those stuck with dealing with it. Get real. Sometimes the lack of perspective among otherwise smart people really worries me, and this conversation about Apple’s motives for the last week or so has worried me the most yet.