14 ms·

> “Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit," Neuenschwander wrote. "It would als

> “Scanning every user’s privately stored iCloud data would create new threat vectors for data thieves to find and exploit," Neuenschwander wrote. "It would also inject the potential for a slippery slope of unintended consequences. Scanning for one type of content, for instance, opens the door for bulk surveillance and could create a desire to search other encrypted messaging systems across content types.” Both of these arguments are absolutely, unambiguously, correct. The other side of the coin is that criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to. This is, I argue, a bad thing. Is is bad for the individuals who are re-victimised on every share. It is also bad for the fabric of society at large, in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it. Does the tech industry have any alternate solutions that could functionally mitigate this abuse? Does the industry feel that it has any responsibility at all to do so? Or do we all just shout "yay, individual freedom wins again!" and forget about the actual problem that this (misguided) initiative was originally aimed at?

PrimeMcFly 3y ago
It's an incredibly bad thing. It's also an incredibly poor excuse to justify backdooring phones. Cops need to investigate the same way they always have, look for clues, go undercover, infiltrate, find where this stuff is actually being made, etc. Scanning everyone's phones would make their jobs significantly easier, no doubt, but it simply isn't worth the cost to us as a society and there is simply no good counter-argument to that.
- deleted 3y ago
  [deleted]
- theshrike79 3y ago
  Let's take a step back here and bring in some facts. "Apple" wasn't scanning your phone, neither was there a "backdoor". If you would've had iCloud upload enabled (you'd be uploading all your photos to Apple's server, a place where they could scan ALL of your media anyway), the phone would've downloaded a set of hashes of KNOWN and HUMAN VERIFIED photos and videos of sexual abuse material. [1] After THREE matches of known and checked CSAM, a check done 100% on-device with zero data moving anywhere, a "reduced-quality copy" would've been sent to a human for verification. If it was someone sending you hashbombs of intentional false matches or an innocuous pic that matched because some mathematical anomaly, the actual human would notice this instantly and no action would've been taken. ...but I still think I was the only HNer who actually read Apple's spec and just didn't go with Twitter hot-takes, so I'm fighting windmills over here. Yes, there is always the risk that an authoritarian government could force Apple to insert checks for stuff other than CSAM to the downloaded database. But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale. (see point above about local checks not being enabled unless iCloud sync is enabled) [1] It wasn't an SHA-1 hash where changing a single bit in the source would make the hash invalid, the people doing that were actually competent.
  - PrimeMcFly 3y ago
    > "Apple" wasn't scanning your phone, neither was there a "backdoor". Yes, you're right. But I've seen calls for phones to scan all content before being uploaded or encrypted, and it often feels, at least in some countries, that could still plausible happen. I suppose that's what I had in mind when I wrote my comment. > But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale. There's a difference with them actively doing it and announcing they are doing it, vs the possibility they are doing it silently without consent.
    - theshrike79 3y ago
      ALL cloud providers are actively scanning all of your content right now, unless you specifically encrypt it yourself. It's a cost of doing business pretty much, you need to give the authorites access to customers data or they can go "but think of the children, there might be child abuse material in there!" and that's really damn hard to argue against. Thus: checking stuff on-device and keeping the cloud locked so that not even the hoster can access it nor can they create a backdoor.
      - PrimeMcFly 3y ago
        > you need to give the authorites access to customers data At least some cloud providers require a warrant before doing so.
        theshrike79 3y ago
        Of course they need a warrant. Officer Johnson from Randomtown Alabama can't just call up Google and tell they want access to everything in GDrive :D But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to.
        PrimeMcFly 3y ago
        > Of course they need a warrant. Right, but your post kind of made it seem like they didn't. And with the CSASM stuff, they didn't need anything to match hashes. So what happens when that gets expanded to other types of content that they still don't need a warrant for? Like torrent files? > But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to. Agreed, but most don't use encryption, and should still have privacy rights for their data.
- rob74 3y ago
  If CSAM was still done the way it "always has been", then "cops" relying on the methods they always had would be a valid answer. But since tech has enabled the distribution of CSAM at unprecedented scales, I think the requests by law enforcement to also make their job a bit easier have some merit...
  - PrimeMcFly 3y ago
    I don't. They can find those materials the same way abusers work their way into communities to have access to them in the first place. The increased scale only means they need more people working on it.
    - rob74 3y ago
      More people. Paid from taxes. Ok, let's increase taxes! Oh, wait...
      - PrimeMcFly 3y ago
        Well yes, let's absolutely increase taxes for the ultra-wealthy why don't pay nearly enough. I don't see the problem. That aside, we could also, at least in the US, stop taking such a ridiculous stance against drugs and instead prioritize finding makers of CSASM materials. The people are there, they are just not being utilized effectively.
  - danaris 3y ago
    The technology has changed distribution, yes. It hasn't particularly changed production, which is where the actual abuse happens. There are still actual people abusing and filming actual children, and those can be found by the police by the same old-fashioned methods they've always had available. (Plus many new ones that don't violate everyone's civil liberties or destroy the security of every networked device.)
draw_down 3y ago
[dead]
- danpalmer 3y ago
  [flagged]
  - draw_down 3y ago
    [dead]
danpalmer 3y ago
I agree that those statements are correct, however my reading of the proposed Apple implementation was that it struck a good balance between maximising the ability to discover CSAM, minimising the threat vectors, minimising false positives, and minimising the possibility that a malicious government could force Apple to implement bulk surveillance. I'm all for privacy, but those who put it above all else are already likely not using Apple devices because of the lack of control. I feel like for Apple's target market the implementation was reasonable. I think Apple backed down on it because of the vocal minority of privacy zealots (for want of a better term) decided it wasn't the right set of trade-offs for them. Given Apple's aim to be a leader in privacy they had to appease this group. I think that community provides a lot of value and oversight, and I broadly agree with their views, but in this case it feels like we lost a big win on the fight against CSAM in order to gain minor, theoretical benefits for user privacy.
- HelloNurse 3y ago
  But "the ability to discover CSAM" is by itself an excuse for mass surveillance, not a bona fide goal. It is certainly possible, instead, to investigate, then find likely pedophiles, and then get a search warrant.
  - danpalmer 3y ago
    Discovering users sharing CSAM is a goal isn't it? That's why governments around the world require cloud storage providers to scan for it – because waiting until the police receive a report of someone is not really feasible. A proactive approach is necessary and mandated in many countries.
    - Moldoteck 3y ago
      imo diminishing ppl's privacy is a goal. Apple's csam could be tricked in different ways, esp with generative algorithms, like an malicious person will send you an album with 100+ normal looking photos(to the eye) but altered to trigger csam, now govt needs to check 100+ photos per person per send and dismiss the false positives. Since this can be replicated, imagine gov't will need to scan 100k similar usecases just for 1k ppl? that's insane, they'll either not check them, so system became obsolete(bc in this case ill intentioned ppl can just send an album of 5k photos, all triggering csam and only a bunch will be real csam. multiplied by nr of these ill ppl, you understand system is easy to game, or they spend thousands of hours checking all this photos and checking each person. Another vector of attack is generation of legit looking csam, bc, generating algorithms are too good now, but in this case(afaik) it's not a crime, since image is fully generated(either by only using ppl's face as starting point or using the description of their face tweaked enough to look realistic). So what we get is: - a system that can be gamed in different ways - a system that's not proved to be effective before releasing - a system that may potentially drive those ppl to other platforms with e2ee that don't have the csam scan(i assume since they know what e2ee is, they can find a platform without csam), so again obsolete AND: - a system that can't be verified by users (like is the csam list legit, can it trigger other things, is the implementation safe?) - a system that can be altered by govt by altering the csam list to target specific ppl (idk snowden or some journalist that found something sketchy) - a system that can be altered by apple/other company by altering csam list for ad targeting purposes Idk, maybe i'm overreacting, but I've seen what a repressive gov can do, and with such an instrument it's frightening what surveillance vectors can be opened
version_five 3y ago
Mass surveillance is never an appropriate solution, let's start with that. I don't belive tech has an over weighted responsibility to solve society's problems, and in fact it's generally better if we don't try and pretend more tech is the answer. Advocating for more money and more prioritization for this area of law enforcement is still the way to go if it's a priority area. Policing seems to be drifting towards "mall cop" work, giving easy fines, enabled by lazy electronic surveillance casting a wide net. Let's put resources towards actual detective work.
- danpalmer 3y ago
  Mass surveillance is bad, but I think there are versions of it that are far less bad than others. Apple's proposed solution would have theoretically only reported cases that were much more than likely to be already known instances of CSAM (i.e. not pictures of your kids), and if nothing else is reported, can we say that they were really surveilled? In some very strict sense, yes, but in terms of outcomes, no.
  - Muehevoll 3y ago
    [flagged]
  - Moldoteck 3y ago
    ok, and in theory, with new generative algorithms, do you think it's still ok? Suppose apple implements this, suppose someone finds a way to generate meme images that can trigger apple's algorithm(but human can't see anything wrong), suppose that someone wants to harm you and sends you a bunch of memes and you save them. What will happen? Or what does happen if somebody is using generative algorithm to create csam like images by using people's face as base but the rest of the image is generated, should this also trigger csam? Also, you can not guarantee that apple/google will use only known instances of csam, what if, govt orders them/google to scan for other type of content under the hood, like documents or god knows what else bc govt want's to screw that person (for the sake of example let's suppose the targeted person is some journalist that discovered shady stuff and govt wants to put em in prison), bc you know, you don't have access to either algorithms and csam scan list that they are using, system could be abused and usually could means 'sometime' it will
awesomeMilou 3y ago
I'd very much like a source on your claim that "[...] criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to." > Does the tech industry have any alternate solutions that could functionally mitigate this abuse? Does the industry feel that it has any responsibility at all to do so? Or do we all just shout "yay, individual freedom wins again!" and forget about the actual problem that this (misguided) initiative was originally aimed at? The issue at large here is not the tech industry, but law enforcement agencies and the correctional system. Law enforcement has proven time and time again themselves that the most effective way to apprehend large criminal networks in this area is by undercover investigation. So no, I don't think it is the tech industries role to play the extended arm of some ill conceived surveillance state. Because Apple is right: This is a slippery slope and anyone that doesn't think malicious political actors will use this as a foot in the door to argue for more invasive surveillance measures, using this exact pre-filtering technology are just naive idiots, in my oppinion.
acumenical 3y ago
[flagged]
- mvonballmo 3y ago
  I honestly hadn't considered that when I asked someone to use Signal or Threema instead of Facebook Messenger that they would think I was a pedophile or drug addict. Food for thought.
  - acumenical 3y ago
    For what it's worth, I don't think using Signal or Threema is enough to make you an E2EE enthusiast, and wanting to speak without your speech later used against you is maybe the purest reason for E2EE. I meant more so the type of people who are into Tor or I2P.
bigDinosaur 3y ago
The extreme hysteria created by anything related to children often seems to be carte blanche to destroy privacy and implement backdoors in applications. Most child abuse comes from family members (which must be solved at the source), and the ultra extreme cases simply make awful law (doing away with E2EE or instituting mass surveillance to catch an incredibly small minority is absurd). Much like other 'tough on crime' measures (of which destroying E2EE is one) the real problems need to be solved not at the point of consumption (drugs, guns, gangs, cartels) but at the root causes. Getting rid of E2EE just opens the avenue for the abuse of us by the government but in no way guarantees we'll meaningfully make children safer. And no, we are not 'condoning' it when we declare E2EE an overall good thing. Real life is about tradeoffs not absolutes, and the tradeoff here is protection from the government for potentially billions vs. maybe arresting a few thousand more real criminals. This is a standard utilitarian tradeoff that 'condones' nothing.
- practice9 3y ago
  > which must be solved at the source Governments are not good at this type of thing. It requires careful analysis, planning and actual decisions. But slap on a regulation and require private companies to do the hard work for you - now we are talking!
  - autoexec 3y ago
    Don't worry, we fill our homes and pockets with enough cameras and microphones from private companies that the government can require the monitoring of everyone in every family 24/7 to make sure we're finally safe!
- gnfargbl 3y ago
  > Most child abuse comes from family members (which must be solved at the source) Yes. Since becoming an abuser is a process and not a moment, part of the solution must be making access to CSAM much harder. > And no, we are not 'condoning' it when we declare E2EE an overall good thing. Agreed. I'm sorry if I worded things in a way that caused you to see an implication which was not intended. To be clear: E2EE is a good thing. Championing E2EE is not equivalent to condoning CSAM. What I did say is that in failing to try and provide any meaningful solutions to this unintended consequence of E2EE, the industry is effectively condoning the problem. > This is a standard utilitarian tradeoff If that's the best we can do, I'm very disappointed. That position says that to achieve privacy, I must tolerate CSAM. I want both privacy and for us not to tolerate CSAM. I don't know what the solution is, but that is what I wish the industry were aiming for. At the moment, the industry seems to be aiming for nothing but a shrug of the shoulders.
glogla 3y ago
[flagged]
- red_admiral 3y ago
  This morality may not be so unusual outside the tech "filter bubble". And wherever someone, like the OP, appears to be serious, my own personal morality says the absolute least they deserve is an equally serious answer.
  - gnfargbl 3y ago
    I'm confused by what you mean by "morality" here. The only moral position that I am communicating is that child sexual abuse is a real thing that really happens, and it is bad for both the individual and for society. That's it. There's no subtext. There is explicitly no refutation of the arguments against client-side CSAM scanning which, I will say again, are unambiguously correct. Is being against child sexual abuse really an unusual opinion in the tech industry? Have we really all gone so far along the Heinlein/Rand road that any mention of a real negative outcome gets immediately dismissed with the empathy-free thought-terminating-cliche "think of the children?"
    - glogla 3y ago
      > Is being against child sexual abuse really an unusual opinion in the tech industry? Nobody said that. You are being manipulative an trying to make it look that people who disagree with you are somehow pro-child abuse. In saying "Does the tech industry have any alternate solutions that could functionally mitigate this abuse?" you are trying to pain a picture in which child abuse is somehow "tech industry" fault. You are also trying to paint a complete Panopticon in which every interpersonal communication is subjected to surveillance by the state as somehow the default, that end to end encrypted electronic communication is changing - while the truth is that personal communication was private for hundreds of years, because it was impossible for the state to listen in on everything.
      - gnfargbl 3y ago
0x000042 3y ago
> Both of these arguments are absolutely, unambiguously, correct. Indeed, they are correct. And they were also brought up when Apple announced that they would introduce this Orwellian system. Now they act like they just realized this.
mrtksn 3y ago
>The other side of the coin is that criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to. I think that companies might need to enable some kind of mechanism for offline investigation of the devices though. The CSAM is a real problem, there are real predators out there, the only risks isn't CSAM and law enforcement does really need to have a way to investigate devices. Previously, my proposal was the ability to force the device scan the user content for fingerprints of the suspected content but only with physical access. Physical access enforces the law enforcement to actually have a real and official investigation with strong enough reasons to spend resources and risk repercussions when done improperly. However, the project of scanning all the user content for policing the users was one thing that irked and later relieved when Apple abandoned the project. Apple's explanation is good and I agree with them but IMHO the more important aspects are: 1) Being able to trust your devices being on your side. That is, your device shouldn't be policing you and shouldn't be snitching on you. At this time you might think that the authorities who would have controlled yor device are on your side but don't forget that those authorities can change. Today the devices may be catching CSAM, some day the authorities can start demanding catching people opposing vaccines and an election or a revolution later they can start catching people who want to have an abortion or having premarital sexual relations or other non-kosher affairs. 2) Being free of the notion that you are always watched. If your device can choose to reveal your private thoughts or business, be it by mistake or by design, you can no longer have thoughts that are unaligned with the official ones. This is like the idea of a god who is always watching you but instead of a creator and angels you get C level businessmen and employees who go through your stuff when the devices triggers decryption of your data(by false positives or by true positives). Anyway, policing everyone all the time must be an idea that is rejected by the free world, if the free world doesn't intent to be as free as Democratic People's Republic of Korea is democratic.
liveoneggs 3y ago
How many times are you okay with having your own children taken from you while the thought police make sure your latest family beach vacation wasn't actually trafficking? How many times will actual abusers be allowed to go free while your own family is victimized by the authorities and what ratio do you find acceptable?
seanieb 3y ago
We were not even at a point where that question needs to be asked. Federal and state police, some of the best funded, equipped and trained police in the world are so inundated with cases that they are forced to limit their investigations to just toddlers and babies. What use is it to add more and more cases to a mountain of uninvestigated crimes? Whats needed is more police clearing the existing caseload.
- BLKNSLVR 3y ago
  Same goes for mandatory reporting that teachers do in Australia. The resources available to investigate are only able to cope with those who are in "immediate danger of losing their lives". That's an unbelievably depressingly low bar for society.
sebstefan 3y ago
>Both of these arguments are absolutely, unambiguously, correct. Oh, please. As if we couldn't just compare the hashes of the pictures people are storing against a CSAM database of hashes that gets regularly updated When this was proposed people would respond "But they could just mirror the pictures or cut a pixel off!" Who cares? You got that picture from some place in the dark web, and eventually someone will stumble upon it and add it to the database. Unless the person individually edits the pictures as they store them, that makes it so that you're never sure if your hashes will posthumously start matching against the DB. People who wank off to CSAM have a user behavior similar to any other porn user, they don't store 1 picture, they store dozens, and just adding that step makes them likely to trip up, or straight up just use another service altogether "What if there's a collision?" I don't know, go one step further with hashing a specific part of the file and see if it still matches? This whole thing felt like an overblown fearmongering campaign from "freedom ain't free" individualists. I've never seen anything wrong with content hosters using a simple hash against you like this.
- ksaj 3y ago
  The hashes can not have collisions anymore, because modern forensics hash with both md5 and sha512, and both hashes must be together for use in any legal case. The odds of both of them having a collision is big enough to flat out say it's not going to happen. But even if there was an md5 hash collision back when md5 was the only one hash use, it still doesn't matter because upon viewing the image that matched, if it's not csam, it doesn't matter. Having said that, the chance of dozens of images matching hashes known to be associated to csam is also so unlikely as to be unthinkable. Where there is smoke, there is fire. And further, a hash alone is meaningless, since in court there must be a presentation of evidence. If the image that set off the csam alarm by hash collision is say, an automobile, there is no case to be had. So all this talk about hash issues is absolutely moot. Source: I have worked as an expert witness and presented for cases involving csam (back when we called it Child Pornography, because the CSAM moniker hadn't come about yet), so the requirements are well known to me. Having said all that, I am an EFF member, and I prefer cryptography to work, and spying on users to be illegal.
  - 3y ago
traceroute66 3y ago
> criminals are using E2EE communication systems to share sexual abuse material Blah blah blah, the same old argument given by the "think of the children" people. There are many ways to counter that old chestnut, but really, we only need to remember the most basic fundamental facts: 1) Encryption is mathematics 2) Criminals are criminals Can you ban mathematics ? No. Can you stop criminals being criminals ? No. So, let's imagine you are able to successfully backdoor E2EE globally, on all devices and all platforms. Sure, the "think of the children" people will rejoice and start singing "Hallelujah". And the governments will rub their hands with glee with all the new data they have access to. But the criminals ? Do you honestly think they'll think "oh no, game over" ? No of course not. They'll pay some cryptographer in need of some money to develop a new E2EE tool and carry on. Business as usual.
- ben_w 3y ago
  > But the criminals ? Do you honestly think they'll think "oh no, game over" ? > No of course not. They'll pay some cryptographer in need of some money to develop a new E2EE tool and carry on. Business as usual. I used to think this, I changed my mind: just as it's difficult to do security correctly even when it's a legal requirement, only the most competent criminal organisations will do this correctly. Unfortunately, the other issue: > And the governments will rub their hands with glee with all the new data they have access to. Is 100% still the case, and almost impossible to get anyone to care about.
  - Moldoteck 3y ago
    they can just use/switch to unpatched devices with some opensource e2ee without big effort. Result? Criminals will continue doing criminal stuff, rest of the planet will be under surveillance system that can be altered at govt/company's will without your knowledge to either target specific ppl(from govt) or target groups of ppl(for 'relevant' ads)
    - ben_w 3y ago
      > they can just use/switch to unpatched devices with some opensource e2ee without big effort. Assuming no new zero-days, and that they were even doing best practices in the first place. How many legit companies do best practices?
numpad0 3y ago
I don't like how these sentiments are written as if (C)SAM sharing is only type of crimes to ever be committed, which is devastating and PTSD inducing and life crippling, but say, not a murder. It could be one, or it could be major financial crimes, terrorism conspiracy, et cetera. Yet, the only justification around for mass surveillance for camera pictures, the important societal matter to rest literally on "the other side of the coin", is "some minority of people could be sharing naked photos of young members of society for highly unethical acts of viewing". wtf.
isykt 3y ago
How are individuals re-victimized with every share? That makes no sense. Your LinkedIn profile photo could be on a billboard in rural China, what would it be to you?
- Simulacra 3y ago
  There are several cases of victims being harassed and haunted by photos and video of them as a child being sexually abused. This is one of the reasons for Masha's law. https://www.nbcphiladelphia.com/news/local/child-porn-victim-masha-law/1968838/ https://www.nbcphiladelphia.com/news/local/child-porn-victim...
  - akira2501 3y ago
    This makes no mention of direct harassment. How easy is it to connect random pictures of children with actual living adults? She's suing people she's never had any direct contact with. The government itself notifies her each time someone is arrested and in possession of an image of her, why, it does not say, but none of this sounds like a necessary healthy resolution to the underlying problem.
ben_w 3y ago
> Is is bad for the individuals who are re-victimised on every share. Absolutely. > It is also bad for the fabric of society at large, in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it. Much less clear. There's always been the argument: does it provide an outlet with no _new_ (emphasis on "new" so people don't skim that word) victims, or does it encourage people to act out their desires for real? I don't have any answer to this question; but the answer matters. I do have a guess, which is "both at the same time in different people", because humans don't have one-size-fits-all responses. Even beyond photographs, this was already a question with drawings; now we also have AI, creating new problems with both deepfakes of real people and ex-nihilo (victimless?) images. > Does the tech industry have any alternate solutions that could functionally mitigate this abuse? Yes. We can build it into the display devices, or use a variation of Van Eck phreaking to the same effect. We can modify WiFi to act as wall-penetrating radar with the capacity to infer pose, heart rate, and breathing of multiple people nearby even if they're next door, so that if they act out their desires beyond the screen, they'll be caught immediately. We can put CCTV cameras everywhere, watch remotely what's on the screens, and also through a combination of eye tracking and (infrared or just noticing a change in geometry) who is aroused while looking at a forbidden subject and require such people to be on suppressants. Note however that I have not said which acts or images: this is because the options are symmetrical under replacement for every other act and image, including (depending on the option) non-sexual ones. There are places in the world where being gay has the death penalty. And if I remember my internet meme history right, whichever state Mr Hands was in, accidentally decriminalised his sex when the Federal courts decided states couldn't outlaw being gay and because that state had only one word in law for everything they deemed "unnatural".
themgt 3y ago
It is also bad for the fabric of society at large, in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it. Does the tech industry have any alternate solutions that could functionally mitigate this abuse? I'd suggest there's a lot the not-tech industry could do to stop condoning abhorrent behavior that stops short of installing scanners on billions of computing devices. It's become a bit of a trope at this point, but it's bizarre to see a guy who is/was spokesman for a "minor attracted persons" (i.e. pedos) advocacy group getting published negatively reviewing the controversial new sex trafficking movie ... in Bloomberg: https://www.bloomberg.com/opinion/articles/2023-07-15/qanon-and-sound-of-freedom-both-rely-on-tired-hollywood-tropes#xj4y7vzkg https://www.bloomberg.com/opinion/articles/2023-07-15/qanon-... For some background: https://www.opindia.com/2021/08/meet-noah-berlatsky-prostasia-foundation-wonder-woman-rutgers-university-press-author-normalising-pedophilia/ https://www.opindia.com/2021/08/meet-noah-berlatsky-prostasi... His 501(c)(3) non-profit also advocates in favor of pedophilic dolls and has a "No Children Harmed certification seal" program for pedophilic dolls/etc: https://prostasia.org/blog/dolls-prevent-child-sexual-abuse/ https://prostasia.org/blog/dolls-prevent-child-sexual-abuse/ https://prostasia.org/no-children-harmed/ https://prostasia.org/no-children-harmed/ I'm not sure you can criminalize stuff like this, but it sets off my alarm bells when pedophile advocates are being published in mainstream news at the same time there's a moral panic around the need to scan everyone's hard drives. Is society actually trying to solve this problem, or is this more like renewing the Patriot Act to record every American's phone calls at the same time we're allied with al Qaeda offshoots in Syria? Interesting how terrorism has been the primary other argument for banning/backdooring all encryption. ---- As an aside I couldn't find ~anything about this group "Heat Initiative" Apple is responding to? Other than a TEDx talk by the founder a couple years ago which again seems very focused on "encrypted platforms" as the primary problem that needs solving: https://www.ted.com/talks/sarah_gardner_searching_for_a_child_in_a_private_world https://www.ted.com/talks/sarah_gardner_searching_for_a_chil...
- vacuity 3y ago
  Can't solve social problems with technology, as they say. And as mentioned elsewhere, most child abuse is perpetrated by family members and other close connections.
deleted 3y ago
[deleted]
tomjen3 3y ago
The real issue is that FBI/NSA/CIA has abused our trust in the so completely that we have to make E2E communication. From assasinating people like Fred Hampton to national security letters, the government has completely lost the trust of tech. That is a bigger problem and it will take a long time to fix. So long that I suspect that anybody reading this is long dead, but its like the saying with planting trees.
lamontcg 3y ago
Are you willing to go to jail after someone hacks into your phone and uploads CSAM to it which triggers this detection mechanism?
FredPret 3y ago
No, no, no. There is no parallel to be drawn between better encryption and worse outcomes for kids. Should we also outlaw high-performance cars because these sometimes serve as effective getaway vehicles for criminals? CSAM producers and consumers should be found and punished via old-fashioned methods. How was this done in the past? Did we just never catch any human traffickers / rapists? No, we had detectives who went around detecting and presumably kicking down doors. To outlaw large sections of mathematics because of this is absurd. And from the amount of power it would give big governments / big businesses, the fabric of society doesn't stand a chance.
- hx8 3y ago
  I think it's a pretty hardline opinion to state law enforcement should be confined to "old-fashioned" methods. Tech is changing the world. Let's not let it be a de-facto lawless world. Yea, LE/IC clearly have gone too far in many modern tactics. Yea, it's possible to build a surveillance/police state much more efficiently than ever before. Yea, we should be vigilant against authoritarianism.
- techsupporter 3y ago
  > Should we also outlaw high-performance cars because these sometimes serve as effective getaway vehicles for criminals? What if we change the last bit after the "because" to "these sometimes are used at unsafe speeds and, intentionally or not, kill people who are not in cars?" Because, at least for me, the answer is an unambiguous yes. I agree that privacy and security should be available to everyone. But we also shouldn't count on being able to find people who are doing vile things--to children or adults--because the person messed up their opsec. I think Apple is correct here but as an industry we have to be putting our brains to thinking about this. "To outlaw large sections of mathematics" is hyperbole because we use mathematics to do a lot of things, some useful and some not.
  - ddq 3y ago
    I think it just takes some outside-the-box thinking. Compromising E2EE is a much easier solution, but there has to be a harder, yet better one. We can't just sidestep important rights for convenience's sake; the right to privacy is being eroded in so many ways that it must be actively safeguarded. If it needs better detective work, so be it.
tshaddox 3y ago
> Both of these arguments are absolutely, unambiguously, correct. I don't really buy any "slippery slope" arguments for this stuff. Apple already can push any conceivable software it wants to all of its phones, so the slope is already as slippery as it can possibly be. It just doesn't make sense to say "Apple shouldn't implement this minimal version of photo-scanning now even though I don't think it's bad, because that's a slippery slope for them to implement some future version of scanning that I do think is bad." They already have the capability to push any software to their phones at any time! They could just skip directly to the version you think is bad!
- mrits 3y ago
  Your comment confused me. Isn't Apple still scanning iPhones for CSAM just not the iCloud? I don't see any additional threat vectors by doing it locally.
  - Anechoic 3y ago
    Other way around-they’re scanning iCloud but not photos stored on iPhones, except in iMessage.
phpisthebest 3y ago
>>and at rates which they were not previously able to. your source for proof of that? >> in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it. No. this narrative of "silence is violence" and "no action is support" etc is 100% wrong. You started out great, but i can not get behind this type of thinking... >>Does the tech industry have any alternate solutions that could functionally mitigate this abuse? Why is it a "tech industry" problem >>Or do we all just shout "yay, individual freedom wins again!" For me the answer is simple... Yes individual freedom is more important that everything. I will never support curbing individual freedom on the alter of any kind proclaimed government solution to a social safety problem. Largely because I know enough about history to understand that not only will they no solve that social safety problem, many in government are probably participating in the problem and have the power to exempt themselves, while abusing the very tools and powers we give them to fight X, for completely unrelated purposes. Very quickly any tool we would give them to fight CSAM would be used for Drug Enforcement, Terrorism, etc. It would not be long before the AI based phashes detect some old lady's Tomato plants as weed and we have an entire DEA paramilitary unit raiding her home...
kafrofrite 3y ago
My two cents reg. this. Creating backdoors that allow encryption schemes to be subverted is _fundamentally_ going to cause harm on the internet, and eventually fail the weakest users/those that need privacy/security the most. A mechanism that can subvert cryptographic protocols can be used by any party, including oppressive regimes, private entities etc. that have the resources/will/knowledge to use the backdoor etc. Backdoors harm both the trust on the web (which can have an impact on economic transactions among many others) and the people that need security/privacy the most. In the meantime, criminals will wise up and move their operations elsewhere where no backdoors exist. We basically end up with a broken internet, we are putting people in harm's way and the criminals we are targeting are probably updating their OPSEC/MO not to rely on E2EE.
Manuel_D 3y ago
> The other side of the coin is that criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to. ...regardless of whether Apple rolls out E2EE right? End to end encryption is available through a whole host of open-source tools, and should Apple deploy CSAM scanning the crooks will just migrate to a different chat tool.
thinking_ape 3y ago
I’m sorry that there are victims, and it’s a horrible crime but no matter what I don’t believe my privacy and my freedoms should be sacrificed for this. A small percentage of people are involved in this crime and subjecting every single person to illegal searches and possibly getting wrongly identified or nation states using this to imprison its enemies is wrong. Sometimes there is no solution that satisfies everyone and sometimes the only good solution is the least shittiest solution but still a shitty solution. In this case, I believe that my and everyone else’s freedoms and privacy are worth it and we should instead spend much money time and effort trying to catch these criminals instead of scanning everyone’s phones which won’t ultimately work.
deleted 3y ago
[deleted]
jahewson 3y ago
The destruction of freedom for a good cause is still the destruction of freedom.
rockemsockem 3y ago
So these people who are coordinating and transferring CSAM are presumably bringing others into the fold to more effectively distribute things. Otherwise digital technology would not make distribution and coordination easier. So law enforcement just needs to infiltrate these groups exactly the same way that they did in the past. The only difference is they don't even need to meet these creeps face to face until they arrest them. They can do it safely behind a screen and they can also scale their own efforts far more effectively.
nullc 3y ago
> The other side of the coin is that criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to. You mean at rates greater than in the 1970s when it was legal in the US and there were a bunch of commercial publications creating and distributing it (prior to the "Protection of Children Against Sexual Exploitation Act" in 1977 (which targeted the production) and "Child Protection and Obscenity Enforcement Act" in 1988 (which targeted the distribution))? I find that doubtful. > Does the industry feel that it has any responsibility at all to do so? Why should it? Should every home builder and landlord install a camera in every bedroom to stream back to the government to be sure that no children are being molested? Why not? It's technologically possible today. The fact that garbage people do garbage things to vulnerable people doesn't mean that the entire world must be commandeered to stop it. E2E encryption has replaced a lot of in person meeting that also would have been secure against dragnet surveillance. We shouldn't lose our privacy just because it became technically possible to take it away, no different than the cameras in bedrooms mentioned above.

PrimeMcFly 3y ago

It's an incredibly bad thing. It's also an incredibly poor excuse to justify backdooring phones. Cops need to investigate the same way they always have, look for clues, go undercover, infiltrate, find where this stuff is actually being made, etc. Scanning everyone's phones would make their jobs significantly easier, no doubt, but it simply isn't worth the cost to us as a society and there is simply no good counter-argument to that.

deleted 3y ago
[deleted]
theshrike79 3y ago
Let's take a step back here and bring in some facts. "Apple" wasn't scanning your phone, neither was there a "backdoor". If you would've had iCloud upload enabled (you'd be uploading all your photos to Apple's server, a place where they could scan ALL of your media anyway), the phone would've downloaded a set of hashes of KNOWN and HUMAN VERIFIED photos and videos of sexual abuse material. [1] After THREE matches of known and checked CSAM, a check done 100% on-device with zero data moving anywhere, a "reduced-quality copy" would've been sent to a human for verification. If it was someone sending you hashbombs of intentional false matches or an innocuous pic that matched because some mathematical anomaly, the actual human would notice this instantly and no action would've been taken. ...but I still think I was the only HNer who actually read Apple's spec and just didn't go with Twitter hot-takes, so I'm fighting windmills over here. Yes, there is always the risk that an authoritarian government could force Apple to insert checks for stuff other than CSAM to the downloaded database. But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale. (see point above about local checks not being enabled unless iCloud sync is enabled) [1] It wasn't an SHA-1 hash where changing a single bit in the source would make the hash invalid, the people doing that were actually competent.
- PrimeMcFly 3y ago
  > "Apple" wasn't scanning your phone, neither was there a "backdoor". Yes, you're right. But I've seen calls for phones to scan all content before being uploaded or encrypted, and it often feels, at least in some countries, that could still plausible happen. I suppose that's what I had in mind when I wrote my comment. > But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale. There's a difference with them actively doing it and announcing they are doing it, vs the possibility they are doing it silently without consent.
  - theshrike79 3y ago
    ALL cloud providers are actively scanning all of your content right now, unless you specifically encrypt it yourself. It's a cost of doing business pretty much, you need to give the authorites access to customers data or they can go "but think of the children, there might be child abuse material in there!" and that's really damn hard to argue against. Thus: checking stuff on-device and keeping the cloud locked so that not even the hoster can access it nor can they create a backdoor.
    - PrimeMcFly 3y ago
      > you need to give the authorites access to customers data At least some cloud providers require a warrant before doing so.
      - theshrike79 3y ago
        Of course they need a warrant. Officer Johnson from Randomtown Alabama can't just call up Google and tell they want access to everything in GDrive :D But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to.
        PrimeMcFly 3y ago
        > Of course they need a warrant. Right, but your post kind of made it seem like they didn't. And with the CSASM stuff, they didn't need anything to match hashes. So what happens when that gets expanded to other types of content that they still don't need a warrant for? Like torrent files? > But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to. Agreed, but most don't use encryption, and should still have privacy rights for their data.
rob74 3y ago
If CSAM was still done the way it "always has been", then "cops" relying on the methods they always had would be a valid answer. But since tech has enabled the distribution of CSAM at unprecedented scales, I think the requests by law enforcement to also make their job a bit easier have some merit...
- PrimeMcFly 3y ago
  I don't. They can find those materials the same way abusers work their way into communities to have access to them in the first place. The increased scale only means they need more people working on it.
  - rob74 3y ago
    More people. Paid from taxes. Ok, let's increase taxes! Oh, wait...
    - PrimeMcFly 3y ago
      Well yes, let's absolutely increase taxes for the ultra-wealthy why don't pay nearly enough. I don't see the problem. That aside, we could also, at least in the US, stop taking such a ridiculous stance against drugs and instead prioritize finding makers of CSASM materials. The people are there, they are just not being utilized effectively.
- danaris 3y ago
  The technology has changed distribution, yes. It hasn't particularly changed production, which is where the actual abuse happens. There are still actual people abusing and filming actual children, and those can be found by the police by the same old-fashioned methods they've always had available. (Plus many new ones that don't violate everyone's civil liberties or destroy the security of every networked device.)

draw_down 3y ago

[dead]

danpalmer 3y ago
[flagged]
- draw_down 3y ago
  [dead]

danpalmer 3y ago

I agree that those statements are correct, however my reading of the proposed Apple implementation was that it struck a good balance between maximising the ability to discover CSAM, minimising the threat vectors, minimising false positives, and minimising the possibility that a malicious government could force Apple to implement bulk surveillance. I'm all for privacy, but those who put it above all else are already likely not using Apple devices because of the lack of control. I feel like for Apple's target market the implementation was reasonable. I think Apple backed down on it because of the vocal minority of privacy zealots (for want of a better term) decided it wasn't the right set of trade-offs for them. Given Apple's aim to be a leader in privacy they had to appease this group. I think that community provides a lot of value and oversight, and I broadly agree with their views, but in this case it feels like we lost a big win on the fight against CSAM in order to gain minor, theoretical benefits for user privacy.

HelloNurse 3y ago
But "the ability to discover CSAM" is by itself an excuse for mass surveillance, not a bona fide goal. It is certainly possible, instead, to investigate, then find likely pedophiles, and then get a search warrant.
- danpalmer 3y ago
  Discovering users sharing CSAM is a goal isn't it? That's why governments around the world require cloud storage providers to scan for it – because waiting until the police receive a report of someone is not really feasible. A proactive approach is necessary and mandated in many countries.
  - Moldoteck 3y ago
    imo diminishing ppl's privacy is a goal. Apple's csam could be tricked in different ways, esp with generative algorithms, like an malicious person will send you an album with 100+ normal looking photos(to the eye) but altered to trigger csam, now govt needs to check 100+ photos per person per send and dismiss the false positives. Since this can be replicated, imagine gov't will need to scan 100k similar usecases just for 1k ppl? that's insane, they'll either not check them, so system became obsolete(bc in this case ill intentioned ppl can just send an album of 5k photos, all triggering csam and only a bunch will be real csam. multiplied by nr of these ill ppl, you understand system is easy to game, or they spend thousands of hours checking all this photos and checking each person. Another vector of attack is generation of legit looking csam, bc, generating algorithms are too good now, but in this case(afaik) it's not a crime, since image is fully generated(either by only using ppl's face as starting point or using the description of their face tweaked enough to look realistic). So what we get is: - a system that can be gamed in different ways - a system that's not proved to be effective before releasing - a system that may potentially drive those ppl to other platforms with e2ee that don't have the csam scan(i assume since they know what e2ee is, they can find a platform without csam), so again obsolete AND: - a system that can't be verified by users (like is the csam list legit, can it trigger other things, is the implementation safe?) - a system that can be altered by govt by altering the csam list to target specific ppl (idk snowden or some journalist that found something sketchy) - a system that can be altered by apple/other company by altering csam list for ad targeting purposes Idk, maybe i'm overreacting, but I've seen what a repressive gov can do, and with such an instrument it's frightening what surveillance vectors can be opened

version_five 3y ago

Mass surveillance is never an appropriate solution, let's start with that. I don't belive tech has an over weighted responsibility to solve society's problems, and in fact it's generally better if we don't try and pretend more tech is the answer. Advocating for more money and more prioritization for this area of law enforcement is still the way to go if it's a priority area. Policing seems to be drifting towards "mall cop" work, giving easy fines, enabled by lazy electronic surveillance casting a wide net. Let's put resources towards actual detective work.

danpalmer 3y ago
Mass surveillance is bad, but I think there are versions of it that are far less bad than others. Apple's proposed solution would have theoretically only reported cases that were much more than likely to be already known instances of CSAM (i.e. not pictures of your kids), and if nothing else is reported, can we say that they were really surveilled? In some very strict sense, yes, but in terms of outcomes, no.
- Muehevoll 3y ago
  [flagged]
- Moldoteck 3y ago
  ok, and in theory, with new generative algorithms, do you think it's still ok? Suppose apple implements this, suppose someone finds a way to generate meme images that can trigger apple's algorithm(but human can't see anything wrong), suppose that someone wants to harm you and sends you a bunch of memes and you save them. What will happen? Or what does happen if somebody is using generative algorithm to create csam like images by using people's face as base but the rest of the image is generated, should this also trigger csam? Also, you can not guarantee that apple/google will use only known instances of csam, what if, govt orders them/google to scan for other type of content under the hood, like documents or god knows what else bc govt want's to screw that person (for the sake of example let's suppose the targeted person is some journalist that discovered shady stuff and govt wants to put em in prison), bc you know, you don't have access to either algorithms and csam scan list that they are using, system could be abused and usually could means 'sometime' it will

awesomeMilou 3y ago

I'd very much like a source on your claim that "[...] criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to." > Does the tech industry have any alternate solutions that could functionally mitigate this abuse? Does the industry feel that it has any responsibility at all to do so? Or do we all just shout "yay, individual freedom wins again!" and forget about the actual problem that this (misguided) initiative was originally aimed at? The issue at large here is not the tech industry, but law enforcement agencies and the correctional system. Law enforcement has proven time and time again themselves that the most effective way to apprehend large criminal networks in this area is by undercover investigation. So no, I don't think it is the tech industries role to play the extended arm of some ill conceived surveillance state. Because Apple is right: This is a slippery slope and anyone that doesn't think malicious political actors will use this as a foot in the door to argue for more invasive surveillance measures, using this exact pre-filtering technology are just naive idiots, in my oppinion.

acumenical 3y ago

[flagged]

mvonballmo 3y ago
I honestly hadn't considered that when I asked someone to use Signal or Threema instead of Facebook Messenger that they would think I was a pedophile or drug addict. Food for thought.
- acumenical 3y ago
  For what it's worth, I don't think using Signal or Threema is enough to make you an E2EE enthusiast, and wanting to speak without your speech later used against you is maybe the purest reason for E2EE. I meant more so the type of people who are into Tor or I2P.

bigDinosaur 3y ago

The extreme hysteria created by anything related to children often seems to be carte blanche to destroy privacy and implement backdoors in applications. Most child abuse comes from family members (which must be solved at the source), and the ultra extreme cases simply make awful law (doing away with E2EE or instituting mass surveillance to catch an incredibly small minority is absurd). Much like other 'tough on crime' measures (of which destroying E2EE is one) the real problems need to be solved not at the point of consumption (drugs, guns, gangs, cartels) but at the root causes. Getting rid of E2EE just opens the avenue for the abuse of us by the government but in no way guarantees we'll meaningfully make children safer. And no, we are not 'condoning' it when we declare E2EE an overall good thing. Real life is about tradeoffs not absolutes, and the tradeoff here is protection from the government for potentially billions vs. maybe arresting a few thousand more real criminals. This is a standard utilitarian tradeoff that 'condones' nothing.

practice9 3y ago
> which must be solved at the source Governments are not good at this type of thing. It requires careful analysis, planning and actual decisions. But slap on a regulation and require private companies to do the hard work for you - now we are talking!
- autoexec 3y ago
  Don't worry, we fill our homes and pockets with enough cameras and microphones from private companies that the government can require the monitoring of everyone in every family 24/7 to make sure we're finally safe!
gnfargbl 3y ago
> Most child abuse comes from family members (which must be solved at the source) Yes. Since becoming an abuser is a process and not a moment, part of the solution must be making access to CSAM much harder. > And no, we are not 'condoning' it when we declare E2EE an overall good thing. Agreed. I'm sorry if I worded things in a way that caused you to see an implication which was not intended. To be clear: E2EE is a good thing. Championing E2EE is not equivalent to condoning CSAM. What I did say is that in failing to try and provide any meaningful solutions to this unintended consequence of E2EE, the industry is effectively condoning the problem. > This is a standard utilitarian tradeoff If that's the best we can do, I'm very disappointed. That position says that to achieve privacy, I must tolerate CSAM. I want both privacy and for us not to tolerate CSAM. I don't know what the solution is, but that is what I wish the industry were aiming for. At the moment, the industry seems to be aiming for nothing but a shrug of the shoulders.

glogla 3y ago

[flagged]

red_admiral 3y ago
This morality may not be so unusual outside the tech "filter bubble". And wherever someone, like the OP, appears to be serious, my own personal morality says the absolute least they deserve is an equally serious answer.
- gnfargbl 3y ago
  I'm confused by what you mean by "morality" here. The only moral position that I am communicating is that child sexual abuse is a real thing that really happens, and it is bad for both the individual and for society. That's it. There's no subtext. There is explicitly no refutation of the arguments against client-side CSAM scanning which, I will say again, are unambiguously correct. Is being against child sexual abuse really an unusual opinion in the tech industry? Have we really all gone so far along the Heinlein/Rand road that any mention of a real negative outcome gets immediately dismissed with the empathy-free thought-terminating-cliche "think of the children?"
  - glogla 3y ago
    > Is being against child sexual abuse really an unusual opinion in the tech industry? Nobody said that. You are being manipulative an trying to make it look that people who disagree with you are somehow pro-child abuse. In saying "Does the tech industry have any alternate solutions that could functionally mitigate this abuse?" you are trying to pain a picture in which child abuse is somehow "tech industry" fault. You are also trying to paint a complete Panopticon in which every interpersonal communication is subjected to surveillance by the state as somehow the default, that end to end encrypted electronic communication is changing - while the truth is that personal communication was private for hundreds of years, because it was impossible for the state to listen in on everything.
    - gnfargbl 3y ago

0x000042 3y ago

> Both of these arguments are absolutely, unambiguously, correct. Indeed, they are correct. And they were also brought up when Apple announced that they would introduce this Orwellian system. Now they act like they just realized this.

mrtksn 3y ago

>The other side of the coin is that criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to. I think that companies might need to enable some kind of mechanism for offline investigation of the devices though. The CSAM is a real problem, there are real predators out there, the only risks isn't CSAM and law enforcement does really need to have a way to investigate devices. Previously, my proposal was the ability to force the device scan the user content for fingerprints of the suspected content but only with physical access. Physical access enforces the law enforcement to actually have a real and official investigation with strong enough reasons to spend resources and risk repercussions when done improperly. However, the project of scanning all the user content for policing the users was one thing that irked and later relieved when Apple abandoned the project. Apple's explanation is good and I agree with them but IMHO the more important aspects are: 1) Being able to trust your devices being on your side. That is, your device shouldn't be policing you and shouldn't be snitching on you. At this time you might think that the authorities who would have controlled yor device are on your side but don't forget that those authorities can change. Today the devices may be catching CSAM, some day the authorities can start demanding catching people opposing vaccines and an election or a revolution later they can start catching people who want to have an abortion or having premarital sexual relations or other non-kosher affairs. 2) Being free of the notion that you are always watched. If your device can choose to reveal your private thoughts or business, be it by mistake or by design, you can no longer have thoughts that are unaligned with the official ones. This is like the idea of a god who is always watching you but instead of a creator and angels you get C level businessmen and employees who go through your stuff when the devices triggers decryption of your data(by false positives or by true positives). Anyway, policing everyone all the time must be an idea that is rejected by the free world, if the free world doesn't intent to be as free as Democratic People's Republic of Korea is democratic.

liveoneggs 3y ago

How many times are you okay with having your own children taken from you while the thought police make sure your latest family beach vacation wasn't actually trafficking? How many times will actual abusers be allowed to go free while your own family is victimized by the authorities and what ratio do you find acceptable?

seanieb 3y ago

We were not even at a point where that question needs to be asked. Federal and state police, some of the best funded, equipped and trained police in the world are so inundated with cases that they are forced to limit their investigations to just toddlers and babies. What use is it to add more and more cases to a mountain of uninvestigated crimes? Whats needed is more police clearing the existing caseload.

BLKNSLVR 3y ago
Same goes for mandatory reporting that teachers do in Australia. The resources available to investigate are only able to cope with those who are in "immediate danger of losing their lives". That's an unbelievably depressingly low bar for society.

sebstefan 3y ago

>Both of these arguments are absolutely, unambiguously, correct. Oh, please. As if we couldn't just compare the hashes of the pictures people are storing against a CSAM database of hashes that gets regularly updated When this was proposed people would respond "But they could just mirror the pictures or cut a pixel off!" Who cares? You got that picture from some place in the dark web, and eventually someone will stumble upon it and add it to the database. Unless the person individually edits the pictures as they store them, that makes it so that you're never sure if your hashes will posthumously start matching against the DB. People who wank off to CSAM have a user behavior similar to any other porn user, they don't store 1 picture, they store dozens, and just adding that step makes them likely to trip up, or straight up just use another service altogether "What if there's a collision?" I don't know, go one step further with hashing a specific part of the file and see if it still matches? This whole thing felt like an overblown fearmongering campaign from "freedom ain't free" individualists. I've never seen anything wrong with content hosters using a simple hash against you like this.

ksaj 3y ago
The hashes can not have collisions anymore, because modern forensics hash with both md5 and sha512, and both hashes must be together for use in any legal case. The odds of both of them having a collision is big enough to flat out say it's not going to happen. But even if there was an md5 hash collision back when md5 was the only one hash use, it still doesn't matter because upon viewing the image that matched, if it's not csam, it doesn't matter. Having said that, the chance of dozens of images matching hashes known to be associated to csam is also so unlikely as to be unthinkable. Where there is smoke, there is fire. And further, a hash alone is meaningless, since in court there must be a presentation of evidence. If the image that set off the csam alarm by hash collision is say, an automobile, there is no case to be had. So all this talk about hash issues is absolutely moot. Source: I have worked as an expert witness and presented for cases involving csam (back when we called it Child Pornography, because the CSAM moniker hadn't come about yet), so the requirements are well known to me. Having said all that, I am an EFF member, and I prefer cryptography to work, and spying on users to be illegal.
- 3y ago

traceroute66 3y ago

> criminals are using E2EE communication systems to share sexual abuse material Blah blah blah, the same old argument given by the "think of the children" people. There are many ways to counter that old chestnut, but really, we only need to remember the most basic fundamental facts: 1) Encryption is mathematics 2) Criminals are criminals Can you ban mathematics ? No. Can you stop criminals being criminals ? No. So, let's imagine you are able to successfully backdoor E2EE globally, on all devices and all platforms. Sure, the "think of the children" people will rejoice and start singing "Hallelujah". And the governments will rub their hands with glee with all the new data they have access to. But the criminals ? Do you honestly think they'll think "oh no, game over" ? No of course not. They'll pay some cryptographer in need of some money to develop a new E2EE tool and carry on. Business as usual.

ben_w 3y ago
> But the criminals ? Do you honestly think they'll think "oh no, game over" ? > No of course not. They'll pay some cryptographer in need of some money to develop a new E2EE tool and carry on. Business as usual. I used to think this, I changed my mind: just as it's difficult to do security correctly even when it's a legal requirement, only the most competent criminal organisations will do this correctly. Unfortunately, the other issue: > And the governments will rub their hands with glee with all the new data they have access to. Is 100% still the case, and almost impossible to get anyone to care about.
- Moldoteck 3y ago
  they can just use/switch to unpatched devices with some opensource e2ee without big effort. Result? Criminals will continue doing criminal stuff, rest of the planet will be under surveillance system that can be altered at govt/company's will without your knowledge to either target specific ppl(from govt) or target groups of ppl(for 'relevant' ads)
  - ben_w 3y ago
    > they can just use/switch to unpatched devices with some opensource e2ee without big effort. Assuming no new zero-days, and that they were even doing best practices in the first place. How many legit companies do best practices?

numpad0 3y ago

I don't like how these sentiments are written as if (C)SAM sharing is only type of crimes to ever be committed, which is devastating and PTSD inducing and life crippling, but say, not a murder. It could be one, or it could be major financial crimes, terrorism conspiracy, et cetera. Yet, the only justification around for mass surveillance for camera pictures, the important societal matter to rest literally on "the other side of the coin", is "some minority of people could be sharing naked photos of young members of society for highly unethical acts of viewing". wtf.

isykt 3y ago

How are individuals re-victimized with every share? That makes no sense. Your LinkedIn profile photo could be on a billboard in rural China, what would it be to you?

Simulacra 3y ago
There are several cases of victims being harassed and haunted by photos and video of them as a child being sexually abused. This is one of the reasons for Masha's law. https://www.nbcphiladelphia.com/news/local/child-porn-victim-masha-law/1968838/ https://www.nbcphiladelphia.com/news/local/child-porn-victim...
- akira2501 3y ago
  This makes no mention of direct harassment. How easy is it to connect random pictures of children with actual living adults? She's suing people she's never had any direct contact with. The government itself notifies her each time someone is arrested and in possession of an image of her, why, it does not say, but none of this sounds like a necessary healthy resolution to the underlying problem.

ben_w 3y ago

> Is is bad for the individuals who are re-victimised on every share. Absolutely. > It is also bad for the fabric of society at large, in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it. Much less clear. There's always been the argument: does it provide an outlet with no _new_ (emphasis on "new" so people don't skim that word) victims, or does it encourage people to act out their desires for real? I don't have any answer to this question; but the answer matters. I do have a guess, which is "both at the same time in different people", because humans don't have one-size-fits-all responses. Even beyond photographs, this was already a question with drawings; now we also have AI, creating new problems with both deepfakes of real people and ex-nihilo (victimless?) images. > Does the tech industry have any alternate solutions that could functionally mitigate this abuse? Yes. We can build it into the display devices, or use a variation of Van Eck phreaking to the same effect. We can modify WiFi to act as wall-penetrating radar with the capacity to infer pose, heart rate, and breathing of multiple people nearby even if they're next door, so that if they act out their desires beyond the screen, they'll be caught immediately. We can put CCTV cameras everywhere, watch remotely what's on the screens, and also through a combination of eye tracking and (infrared or just noticing a change in geometry) who is aroused while looking at a forbidden subject and require such people to be on suppressants. Note however that I have not said which acts or images: this is because the options are symmetrical under replacement for every other act and image, including (depending on the option) non-sexual ones. There are places in the world where being gay has the death penalty. And if I remember my internet meme history right, whichever state Mr Hands was in, accidentally decriminalised his sex when the Federal courts decided states couldn't outlaw being gay and because that state had only one word in law for everything they deemed "unnatural".

themgt 3y ago

It is also bad for the fabric of society at large, in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it. Does the tech industry have any alternate solutions that could functionally mitigate this abuse? I'd suggest there's a lot the not-tech industry could do to stop condoning abhorrent behavior that stops short of installing scanners on billions of computing devices. It's become a bit of a trope at this point, but it's bizarre to see a guy who is/was spokesman for a "minor attracted persons" (i.e. pedos) advocacy group getting published negatively reviewing the controversial new sex trafficking movie ... in Bloomberg: https://www.bloomberg.com/opinion/articles/2023-07-15/qanon-and-sound-of-freedom-both-rely-on-tired-hollywood-tropes#xj4y7vzkg https://www.bloomberg.com/opinion/articles/2023-07-15/qanon-... For some background: https://www.opindia.com/2021/08/meet-noah-berlatsky-prostasia-foundation-wonder-woman-rutgers-university-press-author-normalising-pedophilia/ https://www.opindia.com/2021/08/meet-noah-berlatsky-prostasi... His 501(c)(3) non-profit also advocates in favor of pedophilic dolls and has a "No Children Harmed certification seal" program for pedophilic dolls/etc: https://prostasia.org/blog/dolls-prevent-child-sexual-abuse/ https://prostasia.org/blog/dolls-prevent-child-sexual-abuse/ https://prostasia.org/no-children-harmed/ https://prostasia.org/no-children-harmed/ I'm not sure you can criminalize stuff like this, but it sets off my alarm bells when pedophile advocates are being published in mainstream news at the same time there's a moral panic around the need to scan everyone's hard drives. Is society actually trying to solve this problem, or is this more like renewing the Patriot Act to record every American's phone calls at the same time we're allied with al Qaeda offshoots in Syria? Interesting how terrorism has been the primary other argument for banning/backdooring all encryption. ---- As an aside I couldn't find ~anything about this group "Heat Initiative" Apple is responding to? Other than a TEDx talk by the founder a couple years ago which again seems very focused on "encrypted platforms" as the primary problem that needs solving: https://www.ted.com/talks/sarah_gardner_searching_for_a_child_in_a_private_world https://www.ted.com/talks/sarah_gardner_searching_for_a_chil...

vacuity 3y ago
Can't solve social problems with technology, as they say. And as mentioned elsewhere, most child abuse is perpetrated by family members and other close connections.

deleted 3y ago

[deleted]

tomjen3 3y ago

The real issue is that FBI/NSA/CIA has abused our trust in the so completely that we have to make E2E communication. From assasinating people like Fred Hampton to national security letters, the government has completely lost the trust of tech. That is a bigger problem and it will take a long time to fix. So long that I suspect that anybody reading this is long dead, but its like the saying with planting trees.

lamontcg 3y ago

Are you willing to go to jail after someone hacks into your phone and uploads CSAM to it which triggers this detection mechanism?

FredPret 3y ago

No, no, no. There is no parallel to be drawn between better encryption and worse outcomes for kids. Should we also outlaw high-performance cars because these sometimes serve as effective getaway vehicles for criminals? CSAM producers and consumers should be found and punished via old-fashioned methods. How was this done in the past? Did we just never catch any human traffickers / rapists? No, we had detectives who went around detecting and presumably kicking down doors. To outlaw large sections of mathematics because of this is absurd. And from the amount of power it would give big governments / big businesses, the fabric of society doesn't stand a chance.

hx8 3y ago
I think it's a pretty hardline opinion to state law enforcement should be confined to "old-fashioned" methods. Tech is changing the world. Let's not let it be a de-facto lawless world. Yea, LE/IC clearly have gone too far in many modern tactics. Yea, it's possible to build a surveillance/police state much more efficiently than ever before. Yea, we should be vigilant against authoritarianism.
techsupporter 3y ago
> Should we also outlaw high-performance cars because these sometimes serve as effective getaway vehicles for criminals? What if we change the last bit after the "because" to "these sometimes are used at unsafe speeds and, intentionally or not, kill people who are not in cars?" Because, at least for me, the answer is an unambiguous yes. I agree that privacy and security should be available to everyone. But we also shouldn't count on being able to find people who are doing vile things--to children or adults--because the person messed up their opsec. I think Apple is correct here but as an industry we have to be putting our brains to thinking about this. "To outlaw large sections of mathematics" is hyperbole because we use mathematics to do a lot of things, some useful and some not.
- ddq 3y ago
  I think it just takes some outside-the-box thinking. Compromising E2EE is a much easier solution, but there has to be a harder, yet better one. We can't just sidestep important rights for convenience's sake; the right to privacy is being eroded in so many ways that it must be actively safeguarded. If it needs better detective work, so be it.

tshaddox 3y ago

> Both of these arguments are absolutely, unambiguously, correct. I don't really buy any "slippery slope" arguments for this stuff. Apple already can push any conceivable software it wants to all of its phones, so the slope is already as slippery as it can possibly be. It just doesn't make sense to say "Apple shouldn't implement this minimal version of photo-scanning now even though I don't think it's bad, because that's a slippery slope for them to implement some future version of scanning that I do think is bad." They already have the capability to push any software to their phones at any time! They could just skip directly to the version you think is bad!

mrits 3y ago
Your comment confused me. Isn't Apple still scanning iPhones for CSAM just not the iCloud? I don't see any additional threat vectors by doing it locally.
- Anechoic 3y ago
  Other way around-they’re scanning iCloud but not photos stored on iPhones, except in iMessage.

phpisthebest 3y ago

>>and at rates which they were not previously able to. your source for proof of that? >> in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it. No. this narrative of "silence is violence" and "no action is support" etc is 100% wrong. You started out great, but i can not get behind this type of thinking... >>Does the tech industry have any alternate solutions that could functionally mitigate this abuse? Why is it a "tech industry" problem >>Or do we all just shout "yay, individual freedom wins again!" For me the answer is simple... Yes individual freedom is more important that everything. I will never support curbing individual freedom on the alter of any kind proclaimed government solution to a social safety problem. Largely because I know enough about history to understand that not only will they no solve that social safety problem, many in government are probably participating in the problem and have the power to exempt themselves, while abusing the very tools and powers we give them to fight X, for completely unrelated purposes. Very quickly any tool we would give them to fight CSAM would be used for Drug Enforcement, Terrorism, etc. It would not be long before the AI based phashes detect some old lady's Tomato plants as weed and we have an entire DEA paramilitary unit raiding her home...

kafrofrite 3y ago

My two cents reg. this. Creating backdoors that allow encryption schemes to be subverted is _fundamentally_ going to cause harm on the internet, and eventually fail the weakest users/those that need privacy/security the most. A mechanism that can subvert cryptographic protocols can be used by any party, including oppressive regimes, private entities etc. that have the resources/will/knowledge to use the backdoor etc. Backdoors harm both the trust on the web (which can have an impact on economic transactions among many others) and the people that need security/privacy the most. In the meantime, criminals will wise up and move their operations elsewhere where no backdoors exist. We basically end up with a broken internet, we are putting people in harm's way and the criminals we are targeting are probably updating their OPSEC/MO not to rely on E2EE.

Manuel_D 3y ago

> The other side of the coin is that criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to. ...regardless of whether Apple rolls out E2EE right? End to end encryption is available through a whole host of open-source tools, and should Apple deploy CSAM scanning the crooks will just migrate to a different chat tool.

thinking_ape 3y ago

I’m sorry that there are victims, and it’s a horrible crime but no matter what I don’t believe my privacy and my freedoms should be sacrificed for this. A small percentage of people are involved in this crime and subjecting every single person to illegal searches and possibly getting wrongly identified or nation states using this to imprison its enemies is wrong. Sometimes there is no solution that satisfies everyone and sometimes the only good solution is the least shittiest solution but still a shitty solution. In this case, I believe that my and everyone else’s freedoms and privacy are worth it and we should instead spend much money time and effort trying to catch these criminals instead of scanning everyone’s phones which won’t ultimately work.

deleted 3y ago

[deleted]

jahewson 3y ago

The destruction of freedom for a good cause is still the destruction of freedom.

rockemsockem 3y ago

So these people who are coordinating and transferring CSAM are presumably bringing others into the fold to more effectively distribute things. Otherwise digital technology would not make distribution and coordination easier. So law enforcement just needs to infiltrate these groups exactly the same way that they did in the past. The only difference is they don't even need to meet these creeps face to face until they arrest them. They can do it safely behind a screen and they can also scale their own efforts far more effectively.

nullc 3y ago

> The other side of the coin is that criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to. You mean at rates greater than in the 1970s when it was legal in the US and there were a bunch of commercial publications creating and distributing it (prior to the "Protection of Children Against Sexual Exploitation Act" in 1977 (which targeted the production) and "Child Protection and Obscenity Enforcement Act" in 1988 (which targeted the distribution))? I find that doubtful. > Does the industry feel that it has any responsibility at all to do so? Why should it? Should every home builder and landlord install a camera in every bedroom to stream back to the government to be sure that no children are being molested? Why not? It's technologically possible today. The fact that garbage people do garbage things to vulnerable people doesn't mean that the entire world must be commandeered to stop it. E2E encryption has replaced a lot of in person meeting that also would have been secure against dragnet surveillance. We shouldn't lose our privacy just because it became technically possible to take it away, no different than the cameras in bedrooms mentioned above.