4 ms·

It's an incredibly bad thing. It's also an incredibly poor excuse to justify backdooring phones. Cops need to investigate the same way they always have, look f

It's an incredibly bad thing. It's also an incredibly poor excuse to justify backdooring phones. Cops need to investigate the same way they always have, look for clues, go undercover, infiltrate, find where this stuff is actually being made, etc. Scanning everyone's phones would make their jobs significantly easier, no doubt, but it simply isn't worth the cost to us as a society and there is simply no good counter-argument to that.

deleted 3y ago
[deleted]
theshrike79 3y ago
Let's take a step back here and bring in some facts. "Apple" wasn't scanning your phone, neither was there a "backdoor". If you would've had iCloud upload enabled (you'd be uploading all your photos to Apple's server, a place where they could scan ALL of your media anyway), the phone would've downloaded a set of hashes of KNOWN and HUMAN VERIFIED photos and videos of sexual abuse material. [1] After THREE matches of known and checked CSAM, a check done 100% on-device with zero data moving anywhere, a "reduced-quality copy" would've been sent to a human for verification. If it was someone sending you hashbombs of intentional false matches or an innocuous pic that matched because some mathematical anomaly, the actual human would notice this instantly and no action would've been taken. ...but I still think I was the only HNer who actually read Apple's spec and just didn't go with Twitter hot-takes, so I'm fighting windmills over here. Yes, there is always the risk that an authoritarian government could force Apple to insert checks for stuff other than CSAM to the downloaded database. But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale. (see point above about local checks not being enabled unless iCloud sync is enabled) [1] It wasn't an SHA-1 hash where changing a single bit in the source would make the hash invalid, the people doing that were actually competent.
- PrimeMcFly 3y ago
  > "Apple" wasn't scanning your phone, neither was there a "backdoor". Yes, you're right. But I've seen calls for phones to scan all content before being uploaded or encrypted, and it often feels, at least in some countries, that could still plausible happen. I suppose that's what I had in mind when I wrote my comment. > But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale. There's a difference with them actively doing it and announcing they are doing it, vs the possibility they are doing it silently without consent.
  - theshrike79 3y ago
    ALL cloud providers are actively scanning all of your content right now, unless you specifically encrypt it yourself. It's a cost of doing business pretty much, you need to give the authorites access to customers data or they can go "but think of the children, there might be child abuse material in there!" and that's really damn hard to argue against. Thus: checking stuff on-device and keeping the cloud locked so that not even the hoster can access it nor can they create a backdoor.
    - PrimeMcFly 3y ago
      > you need to give the authorites access to customers data At least some cloud providers require a warrant before doing so.
      - theshrike79 3y ago
        Of course they need a warrant. Officer Johnson from Randomtown Alabama can't just call up Google and tell they want access to everything in GDrive :D But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to.
        PrimeMcFly 3y ago
        > Of course they need a warrant. Right, but your post kind of made it seem like they didn't. And with the CSASM stuff, they didn't need anything to match hashes. So what happens when that gets expanded to other types of content that they still don't need a warrant for? Like torrent files? > But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to. Agreed, but most don't use encryption, and should still have privacy rights for their data.
        theshrike79 3y ago
        If they would have expanded it beyond CSAM, in that instance I would have joined the rest of the internet on the picket lines protesting against the system. The point of the system would've been that EVERYONE'S content would've been encrypted in the cloud without them needing to do anything.
- Moldoteck 3y ago
  i thought they can't scan the media in icloud, since media is encrypted, no? also: If it was someone sending you hashbombs of intentional false matches or an innocuous pic that matched because some mathematical anomaly, the actual human would notice this instantly and no action would've been taken. - if someone is doing this, imagine the scale- thousands of pics that should be human-evaluated, scaled to thousands of people, it'll be just plain ignored, meaning system loses it's purpose. also, you say that it'll be enabled only if icloud bck is enabled, but it's not guaranteed, this assumption can later change... and it doesn't make sense, for me your 2 statements contradict themselves: - if apple can scan your photos in icloud AND for this feature to be enabled, you must enable icloud, why they should send hashes to you? they can scan the photos anyway in icloud, since all your photos are backed up there. Unless... they can't scan photos in icloud since these are encrypted, meaning scanning can be done only locally before photos are sent, meaning icloud enabling is not mandatory and it could work without it. Either way the csam scanning is imo pointless, on one hand bc of privacy reasons(and we've seen that if state is able to use a backdoor, it'll use it when needed) and on the other hand, because of generative algorithms: photos can be manipulated to trigger csam even if human eye can see another thing (aka hashbomb) OR a sick/ill intentioned person can generate a legit csam like photo just by using target ppl's face(or description of their face), in this case I don't even know if they are breaking the law or not, since the image is totally generated but is looking totally illegal
  - theshrike79 3y ago
    You do know that we currently have "thousands of people" watching for and tagging the most heinous shit people upload to social media, right? There are multiple sources for this how we use outsourced people from Africa and Asia to weed through all of the filth people upload on FB alone. "Looking illegal" isn't enough to trigger a CSAM check in this case. It's perfectly normal to take pictures of your own kids without clothes in most of Europe for example. Nothing illegal. That's why the checks would've been explicitly done on known and confirmed CSAM images. It wasn't some kind of check_if_penis() -algorigthm, or one of the shitty ones that trigger if there's too much (white) skin colour in an image.
    - 3y ago
- taikahessu 3y ago
  You might've read the spec but you're missing the point and your approach is naive. For me it's about crossing the line. If you want to be snooping around my phone or my house, you need a warrant and go to official channels provides by my gov officials. And you really think it's as simple as picking apples from oranges? I mean come on. Yes, it's easy to implement hash check to see if u have some known child porn in your cloud. But was that the use case for the advocates? No. Their use case was to try finding abuse and that would need a more thorough scanning. And once we're there, we have to make hard decisions on what is porn or abuse. If u think it's easy, then you need to it through harder. Think of some picture from sauna where there are naked family, might be harmful? But normal here in Finland. What about a stick figure cartoon what depicts a some shady sexual positions with a smaller child-like figure in it? Or what about grooming, asking for naked pics? How is this system going to prevent that? I mean, I get why ppl would want something like this. But it isn't the right solution imho.
  - theshrike79 3y ago
    First of all, your analogy of "sending officials to your home without a warrant" is again you completely misunderstanding the feature. The local scanning would've been enabled only if you would've uploaded the photos to iCloud anyway. To complete your crappy analogy: You had already agreed to send the information of everything contained in your apartment to a company, and now you're getting into a hissy-fit when they are sending a robot visit your house with a list of known child pornography photos and checking if you have any? There weren't any "hard decisions" it wasn't a "this has a naked person in it" -scanner. It wasn't an "abuse checked" it was specifically created to find known CP. It checked your photos against a NeuralHash of KNOWN child pornography. So the only chance your sauna photos would've been flagged if they were included in a set of CP photos distributed widely enough so that they're added to the CSAM database. And nobody claimed the system would magically cure the world of grooming, where did you get that idea from? Although the current filters in iMessage might help if the abuser is stupid enough to use that for grooming.
    - taikahessu 3y ago
      Sorry for my previous posts' hostile nature. Thank you for clearing up the actual proposed implementation details. It sounds like I came to pretty much the same conclusion as you did. I might have been quite unclear, but I was actually referring to (in my mind) the inevitable stage 2 of the implementation, if you give the little finger here ... theres no stopping. When I read about this, I was feeling sad, frustrated and a bit angry too. I like my iphone, i switched from android because they aged too quickly. Given the nature of Apple ecosystem, I took this as crossing the line. And i would have to strongly look for alternatives, since this scanning is not something i want to accept and i would vote with my money elsewhere. Yes, i know they scan all photos already. But to scan MY photos and hold me accountable, that is something i am not okay with. Yes, its only icloud, for now, but i dont see the point of using some 3rd party cloud, i used to, but in apple device it would be suboptimal for my daily use etc etc. My point was of the original case of protecting children. That is what the advocate groups want. That is what I want. But I just draw the line there what i feel is my personal space, that you don't scan my photos or files, and that is not the answer. Since there are these few behemoth FAANG companies ruling the ecosystems and impacting our lives, the battle of privacy needs to be fought right there. So, for me that would be the line as I see the progression of that path is much worse. That is where I raise my hand and say, I'm out, what next? No smartphone? Maybe. :) Peace.
rob74 3y ago
If CSAM was still done the way it "always has been", then "cops" relying on the methods they always had would be a valid answer. But since tech has enabled the distribution of CSAM at unprecedented scales, I think the requests by law enforcement to also make their job a bit easier have some merit...
- PrimeMcFly 3y ago
  I don't. They can find those materials the same way abusers work their way into communities to have access to them in the first place. The increased scale only means they need more people working on it.
  - rob74 3y ago
    More people. Paid from taxes. Ok, let's increase taxes! Oh, wait...
    - PrimeMcFly 3y ago
      Well yes, let's absolutely increase taxes for the ultra-wealthy why don't pay nearly enough. I don't see the problem. That aside, we could also, at least in the US, stop taking such a ridiculous stance against drugs and instead prioritize finding makers of CSASM materials. The people are there, they are just not being utilized effectively.
- danaris 3y ago
  The technology has changed distribution, yes. It hasn't particularly changed production, which is where the actual abuse happens. There are still actual people abusing and filming actual children, and those can be found by the police by the same old-fashioned methods they've always had available. (Plus many new ones that don't violate everyone's civil liberties or destroy the security of every networked device.)

deleted 3y ago

[deleted]

theshrike79 3y ago

Let's take a step back here and bring in some facts. "Apple" wasn't scanning your phone, neither was there a "backdoor". If you would've had iCloud upload enabled (you'd be uploading all your photos to Apple's server, a place where they could scan ALL of your media anyway), the phone would've downloaded a set of hashes of KNOWN and HUMAN VERIFIED photos and videos of sexual abuse material. [1] After THREE matches of known and checked CSAM, a check done 100% on-device with zero data moving anywhere, a "reduced-quality copy" would've been sent to a human for verification. If it was someone sending you hashbombs of intentional false matches or an innocuous pic that matched because some mathematical anomaly, the actual human would notice this instantly and no action would've been taken. ...but I still think I was the only HNer who actually read Apple's spec and just didn't go with Twitter hot-takes, so I'm fighting windmills over here. Yes, there is always the risk that an authoritarian government could force Apple to insert checks for stuff other than CSAM to the downloaded database. But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale. (see point above about local checks not being enabled unless iCloud sync is enabled) [1] It wasn't an SHA-1 hash where changing a single bit in the source would make the hash invalid, the people doing that were actually competent.

PrimeMcFly 3y ago
> "Apple" wasn't scanning your phone, neither was there a "backdoor". Yes, you're right. But I've seen calls for phones to scan all content before being uploaded or encrypted, and it often feels, at least in some countries, that could still plausible happen. I suppose that's what I had in mind when I wrote my comment. > But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale. There's a difference with them actively doing it and announcing they are doing it, vs the possibility they are doing it silently without consent.
- theshrike79 3y ago
  ALL cloud providers are actively scanning all of your content right now, unless you specifically encrypt it yourself. It's a cost of doing business pretty much, you need to give the authorites access to customers data or they can go "but think of the children, there might be child abuse material in there!" and that's really damn hard to argue against. Thus: checking stuff on-device and keeping the cloud locked so that not even the hoster can access it nor can they create a backdoor.
  - PrimeMcFly 3y ago
    > you need to give the authorites access to customers data At least some cloud providers require a warrant before doing so.
    - theshrike79 3y ago
      Of course they need a warrant. Officer Johnson from Randomtown Alabama can't just call up Google and tell they want access to everything in GDrive :D But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to.
      - PrimeMcFly 3y ago
        > Of course they need a warrant. Right, but your post kind of made it seem like they didn't. And with the CSASM stuff, they didn't need anything to match hashes. So what happens when that gets expanded to other types of content that they still don't need a warrant for? Like torrent files? > But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to. Agreed, but most don't use encryption, and should still have privacy rights for their data.
        theshrike79 3y ago
        If they would have expanded it beyond CSAM, in that instance I would have joined the rest of the internet on the picket lines protesting against the system. The point of the system would've been that EVERYONE'S content would've been encrypted in the cloud without them needing to do anything.
Moldoteck 3y ago
i thought they can't scan the media in icloud, since media is encrypted, no? also: If it was someone sending you hashbombs of intentional false matches or an innocuous pic that matched because some mathematical anomaly, the actual human would notice this instantly and no action would've been taken. - if someone is doing this, imagine the scale- thousands of pics that should be human-evaluated, scaled to thousands of people, it'll be just plain ignored, meaning system loses it's purpose. also, you say that it'll be enabled only if icloud bck is enabled, but it's not guaranteed, this assumption can later change... and it doesn't make sense, for me your 2 statements contradict themselves: - if apple can scan your photos in icloud AND for this feature to be enabled, you must enable icloud, why they should send hashes to you? they can scan the photos anyway in icloud, since all your photos are backed up there. Unless... they can't scan photos in icloud since these are encrypted, meaning scanning can be done only locally before photos are sent, meaning icloud enabling is not mandatory and it could work without it. Either way the csam scanning is imo pointless, on one hand bc of privacy reasons(and we've seen that if state is able to use a backdoor, it'll use it when needed) and on the other hand, because of generative algorithms: photos can be manipulated to trigger csam even if human eye can see another thing (aka hashbomb) OR a sick/ill intentioned person can generate a legit csam like photo just by using target ppl's face(or description of their face), in this case I don't even know if they are breaking the law or not, since the image is totally generated but is looking totally illegal
- theshrike79 3y ago
  You do know that we currently have "thousands of people" watching for and tagging the most heinous shit people upload to social media, right? There are multiple sources for this how we use outsourced people from Africa and Asia to weed through all of the filth people upload on FB alone. "Looking illegal" isn't enough to trigger a CSAM check in this case. It's perfectly normal to take pictures of your own kids without clothes in most of Europe for example. Nothing illegal. That's why the checks would've been explicitly done on known and confirmed CSAM images. It wasn't some kind of check_if_penis() -algorigthm, or one of the shitty ones that trigger if there's too much (white) skin colour in an image.
  - 3y ago
taikahessu 3y ago
You might've read the spec but you're missing the point and your approach is naive. For me it's about crossing the line. If you want to be snooping around my phone or my house, you need a warrant and go to official channels provides by my gov officials. And you really think it's as simple as picking apples from oranges? I mean come on. Yes, it's easy to implement hash check to see if u have some known child porn in your cloud. But was that the use case for the advocates? No. Their use case was to try finding abuse and that would need a more thorough scanning. And once we're there, we have to make hard decisions on what is porn or abuse. If u think it's easy, then you need to it through harder. Think of some picture from sauna where there are naked family, might be harmful? But normal here in Finland. What about a stick figure cartoon what depicts a some shady sexual positions with a smaller child-like figure in it? Or what about grooming, asking for naked pics? How is this system going to prevent that? I mean, I get why ppl would want something like this. But it isn't the right solution imho.
- theshrike79 3y ago
  First of all, your analogy of "sending officials to your home without a warrant" is again you completely misunderstanding the feature. The local scanning would've been enabled only if you would've uploaded the photos to iCloud anyway. To complete your crappy analogy: You had already agreed to send the information of everything contained in your apartment to a company, and now you're getting into a hissy-fit when they are sending a robot visit your house with a list of known child pornography photos and checking if you have any? There weren't any "hard decisions" it wasn't a "this has a naked person in it" -scanner. It wasn't an "abuse checked" it was specifically created to find known CP. It checked your photos against a NeuralHash of KNOWN child pornography. So the only chance your sauna photos would've been flagged if they were included in a set of CP photos distributed widely enough so that they're added to the CSAM database. And nobody claimed the system would magically cure the world of grooming, where did you get that idea from? Although the current filters in iMessage might help if the abuser is stupid enough to use that for grooming.
  - taikahessu 3y ago
    Sorry for my previous posts' hostile nature. Thank you for clearing up the actual proposed implementation details. It sounds like I came to pretty much the same conclusion as you did. I might have been quite unclear, but I was actually referring to (in my mind) the inevitable stage 2 of the implementation, if you give the little finger here ... theres no stopping. When I read about this, I was feeling sad, frustrated and a bit angry too. I like my iphone, i switched from android because they aged too quickly. Given the nature of Apple ecosystem, I took this as crossing the line. And i would have to strongly look for alternatives, since this scanning is not something i want to accept and i would vote with my money elsewhere. Yes, i know they scan all photos already. But to scan MY photos and hold me accountable, that is something i am not okay with. Yes, its only icloud, for now, but i dont see the point of using some 3rd party cloud, i used to, but in apple device it would be suboptimal for my daily use etc etc. My point was of the original case of protecting children. That is what the advocate groups want. That is what I want. But I just draw the line there what i feel is my personal space, that you don't scan my photos or files, and that is not the answer. Since there are these few behemoth FAANG companies ruling the ecosystems and impacting our lives, the battle of privacy needs to be fought right there. So, for me that would be the line as I see the progression of that path is much worse. That is where I raise my hand and say, I'm out, what next? No smartphone? Maybe. :) Peace.

rob74 3y ago

If CSAM was still done the way it "always has been", then "cops" relying on the methods they always had would be a valid answer. But since tech has enabled the distribution of CSAM at unprecedented scales, I think the requests by law enforcement to also make their job a bit easier have some merit...

PrimeMcFly 3y ago
I don't. They can find those materials the same way abusers work their way into communities to have access to them in the first place. The increased scale only means they need more people working on it.
- rob74 3y ago
  More people. Paid from taxes. Ok, let's increase taxes! Oh, wait...
  - PrimeMcFly 3y ago
    Well yes, let's absolutely increase taxes for the ultra-wealthy why don't pay nearly enough. I don't see the problem. That aside, we could also, at least in the US, stop taking such a ridiculous stance against drugs and instead prioritize finding makers of CSASM materials. The people are there, they are just not being utilized effectively.
danaris 3y ago
The technology has changed distribution, yes. It hasn't particularly changed production, which is where the actual abuse happens. There are still actual people abusing and filming actual children, and those can be found by the police by the same old-fashioned methods they've always had available. (Plus many new ones that don't violate everyone's civil liberties or destroy the security of every networked device.)