3 ms·

One does not need to go to court to have their life ruined by accusations. Ironically, there's quite a few examples of this over the years for alleged CSAM. On

One does not need to go to court to have their life ruined by accusations. Ironically, there's quite a few examples of this over the years for alleged CSAM. One example that sticks out in my mind is a pair of grandparents who photographed their grandchildren playing in the back yard. A photo tech flagged their photo, they were arrested, and it took their lawyer going through the hoops to get a review of the photo for the charges to be dropped.

shapefrog 5y ago
Sure, there are always cases - but was their photo of a grey blob that matched a hash but is clearly a grey blob or of a naked child? If the photo was a grey blob and they had to go through a judicial review for someone to look at the photo and confirm 'yes that is a grey blob' then color me wrong.
- falcolas 5y ago
  I'd view a "grey blob" to be the MVP of hash collisions. I doubt that this will end with grey blobs - I see it ending with common images (memes would be great for this) being invisibly altered to collide with a CSAM hash.
  - shapefrog 5y ago
    If you need a judicial review to confim that a slightly altered Bernie in Coat and Gloves meme is not the same image as the picture of a child being raped that they have on file then we have way bigger problems.
    - falcolas 5y ago
      Here's the thing with CSAM - it's illegal to view and transmit. So nobody, until the police have confiscated your devices, will actually be able to verify that it is a "child being raped." They'll view visual hashes, look at descriptions, and so forth, but nobody from Apple will actually be looking at them, because then they are guilty of viewing and transmitting CSAM. I noted in another comment, even the prosecutors and defense lawyers in the case typically only get a description of the content, they don't see it themselves.
      - shapefrog 5y ago
        This is just not true. Human review is conducted. Apple will conduct human review, facebook conduct human review, NCMEC will conduct human review, law enforcement will conduct human review, lawyers and judges will conduct human review. Over the years there have been countless articles etc about how fucked up being a reviewer of content flagged at all the tech companies is. https://www.vice.com/en/article/a35xk5/facebook-moderators-are-suing-for-trauma-ptsd https://www.vice.com/en/article/a35xk5/facebook-moderators-a...
      - hannasanarion 5y ago
        Where did you get this idea, scooby doo? It is not illegal to be an unwilling recipient of illegal material. If a package shows up at your door with a bomb, you're not gonna be thrown in jail for having a bomb.
        hnfong 5y ago
        In theory, sure. At the very least, you'd be one of the primary suspects, and if you somehow got a bad lawyer, all bets are off. https://hongkongfp.com/2021/08/12/judges-criticise-hong-kongs-justice-dept-over-prosecution-of-possibly-innocent-man/ https://hongkongfp.com/2021/08/12/judges-criticise-hong-kong...
        hannasanarion 5y ago
        Okay, and when a cursory look at the bomb actually reveals it to be a fisher-price toy, what then? What is the scenario where a grey blob gets on your phone that sets off CSAM alerts, an investigator looks at it and sees only a grey blob, and then still decides to alert the authorities even though it's just a grey blob, and the authorities still decide to arrest you even though it's just a grey blob, and the DA still decides to prosecute you even though it's just a grey blob, and a jury still decides to convict you, even though it's still just a grey blob? You're the one who's off in theory-land imagining that every person in the entire justice system is just as stupid as this algorithm is.
        int_19h 5y ago

shapefrog 5y ago

Sure, there are always cases - but was their photo of a grey blob that matched a hash but is clearly a grey blob or of a naked child? If the photo was a grey blob and they had to go through a judicial review for someone to look at the photo and confirm 'yes that is a grey blob' then color me wrong.

falcolas 5y ago
I'd view a "grey blob" to be the MVP of hash collisions. I doubt that this will end with grey blobs - I see it ending with common images (memes would be great for this) being invisibly altered to collide with a CSAM hash.
- shapefrog 5y ago
  If you need a judicial review to confim that a slightly altered Bernie in Coat and Gloves meme is not the same image as the picture of a child being raped that they have on file then we have way bigger problems.
  - falcolas 5y ago
    Here's the thing with CSAM - it's illegal to view and transmit. So nobody, until the police have confiscated your devices, will actually be able to verify that it is a "child being raped." They'll view visual hashes, look at descriptions, and so forth, but nobody from Apple will actually be looking at them, because then they are guilty of viewing and transmitting CSAM. I noted in another comment, even the prosecutors and defense lawyers in the case typically only get a description of the content, they don't see it themselves.
    - shapefrog 5y ago
      This is just not true. Human review is conducted. Apple will conduct human review, facebook conduct human review, NCMEC will conduct human review, law enforcement will conduct human review, lawyers and judges will conduct human review. Over the years there have been countless articles etc about how fucked up being a reviewer of content flagged at all the tech companies is. https://www.vice.com/en/article/a35xk5/facebook-moderators-are-suing-for-trauma-ptsd https://www.vice.com/en/article/a35xk5/facebook-moderators-a...
    - hannasanarion 5y ago
      Where did you get this idea, scooby doo? It is not illegal to be an unwilling recipient of illegal material. If a package shows up at your door with a bomb, you're not gonna be thrown in jail for having a bomb.
      - hnfong 5y ago
        In theory, sure. At the very least, you'd be one of the primary suspects, and if you somehow got a bad lawyer, all bets are off. https://hongkongfp.com/2021/08/12/judges-criticise-hong-kongs-justice-dept-over-prosecution-of-possibly-innocent-man/ https://hongkongfp.com/2021/08/12/judges-criticise-hong-kong...
        hannasanarion 5y ago
        Okay, and when a cursory look at the bomb actually reveals it to be a fisher-price toy, what then? What is the scenario where a grey blob gets on your phone that sets off CSAM alerts, an investigator looks at it and sees only a grey blob, and then still decides to alert the authorities even though it's just a grey blob, and the authorities still decide to arrest you even though it's just a grey blob, and the DA still decides to prosecute you even though it's just a grey blob, and a jury still decides to convict you, even though it's still just a grey blob? You're the one who's off in theory-land imagining that every person in the entire justice system is just as stupid as this algorithm is.
      - int_19h 5y ago