34 ms·

Apple enabling client-side CSAM scanning on iPhone tomorrow

• ostenning 5y ago
  Now Pegasus can really cause some damage!
• robbiet480 5y ago
  This has worrying privacy implications. I hope Apple makes a public announcement about this but wouldn’t be surprised if they don’t. I also would expect EFF will get on this shortly.
  • zepto 5y ago
    What are the implications?
    • robbiet480 5y ago
      To quote another tweet from Matthew Green, the author of the Twitter thread (https://twitter.com/matthew_d_green/status/1423110344730349574?s=21 https://twitter.com/matthew_d_green/status/14231103447303495...): > Regardless of what Apple’s long term plans are, they’ve sent a very clear signal. In their (very influential) opinion, it is safe to build systems that scan users’ phones for prohibited content. > That’s the message they’re sending to governments, competing services, China, you.
      • zepto 5y ago
        Is it? That’s just something the tweets have read in. The message could equally well be ‘We won’t become an easy political target by ignoring a problem something most people care about like child porn, but we are going to build a point solution to that problem, so the public doesn’t force us to bow government surveillance requests.’ It’s easy to nod along with an anti-Apple slogan, but we need to consider what would happen if they didn’t do this.
        • resfirestar 5y ago
          If Apple thought this kind of dragnet was a losing political fight that tells me they've become too weak to stand up to unreasonable government demands. Where is the company that won the public opinion battle over unlocking a mass shooter's phone?
          • zepto 5y ago
            This isn’t a government demand. This is something the public cares deeply about, and Apple is solving it their own way. Public opinion is not in favor of giving safe harbor to pedophiles and child pornographers, and I can’t see why anyone would even want Apple to fight that battle.
            • resfirestar 5y ago
              Not sure where you got that information. I haven't seen any official announcements, so I assumed based on it being US-only and rolled out with no fanfare (except critical press stories citing unnamed sources) that it's something the FBI asked for.
              • zepto 5y ago
                Not sure where you got that - seems like you’ve just made up an explanation that it’s an FBI demand out of whole cloth. What we do know is that it is a proprietary solution using a proprietary hash which only applies to Apple products and that Apple has always presented themselves as a family friendly company that doesn’t support criminal use cases. Everything points to this being something Apple thinks needs to be solved before the public asks why they haven’t. If it was a government demand, we’d presumably see Google responding to it too. You still haven’t explained why not working to deter pedophiles and child pornographers, is an important battle for them to fight. They might think it’s a problem they want to solve on their own terms, and that would seem to be what they have done here. Not solving a problem people care about just because the government also cares about it seems illogical. I think a lot of people like the idea of corporations taking responsibility for the social problems they cause without needing to be forced to do so by the government.
    • 13of40 5y ago
      False positives, what if someone can poison the set of hashes, engineered collisions, etc. And what happens when you come up positive - does the local sheriff just get a warrant and SWAT you at that point? Is the detection of a hash prosecutable? Is it enough to get your teeth kicked in, or get you informally labeled a pedo by your local police? On the flip side, since it's running on the client, could actual pedophiles use it to mutate their images until they can evade the hashing algorithm?
      • zepto 5y ago
        False positives are clearly astronomically unlikely. Not a real issue. Engineered collisions seem unlikely too. Not impossible. Unless there is a straight up cryptographic defect in the hash algorithm, it seems hard to see how engineered collisions could be made to happen at any scale.
        • SilverRed 5y ago
          At Apple scale, a once in a million issue is going to ruin the lives of 2000 people. A false positive here is not a mild inconvenience. It means police raiding their house, potentially damaging it, seizing all of their technology for months while it is analyzed, and leaving these people highly stressed while they try to put their lives back together. This isn't some web tech startup where a mistake means someones tshirt got sent to the wrong address. Peoples lives will quite literally be ruined over mistakes here.
          • zepto 5y ago
            > once in a million issue Is it a once in a million issue? The collision rate matters. It could easily be much higher and then it wouldn’t matter that it was being used at Apple’s scale.
        • 13of40 5y ago
          If this was the kind of hash where flipping one bit of the input completely scrambles the output, the bad guys would just flip one bit of the input to evade it. Obviously a PhotoDna type of hash is going to be be easier to cause a collision with because they're averaging out a ton of the input data. According to Wikipedia the classic way to do it is convert it to monochrome, divide it into a grid, and average the shade of each of the cells. If they're doing that you could probably just pass in that intermediate grid and it would "hash" to the same result as the original picture with no porn present.
    • tomaskafka 5y ago
      A country can collect a list of people sharing any content they put on a hash list. Like gay porn, 'save Khashoggi' meme, or a photo from documentary about missing Uighurs. It's hard to imagine how this could be misused, right?
      • zepto 5y ago
        That seems like a real problem, and of course it could be misused, however nothing so far revealed actually tells us whether it is possible. E.g. how the hashes are computed, where they come from, and what happens when a positive match is detected. Until we have a clear understanding of these things, the rest is just speculation.
        • tomaskafka 5y ago
          The hashes will of course be provided by local governments, who have the ultimate authority (because they can forbid Apple to sell there, and Tim Cook never says no to money).
          • zepto 5y ago
            > Tim Cook never says no to money Money citizens of those countries want to give him.
• ajsnigrutin 5y ago
  So if I understand correctly, they want to scan all your photos, stored on your private phone, that you paid for, and they want to check if any of the hashes are the same as hashes of child porn? So... all your hashes will be uploaded to the cloud? How do you prevent them from scanning other stuff (memes, leaked documents, trump-fights-cnn-gif,... to profile the users)? Or will a huge hash database of child porn hashes be downloaded to the phone? Honestly, i think it's one more abuse of terrorism/child porn to take away privacy of people, and mark all oposing the law as terrorists/pedos. ...also, as in the thread from the original url, making false positives and spreading them around (think 4chan mass e-mailing stuff) might cause a lot of problems too.
  • willis936 5y ago
    >So... all your hashes will be uploaded to the cloud? That isn't how I interpret "client-side". The privacy implications are far more subtle.
    • dannyw 5y ago
      It's still really, really bad. It always starts with child porn, and in a few years the offline Notes app will be phoning home if you write speech criticising the government in China. This technology inevitably leads to the sueveillance, suppression and murder of activists and journalists. It always starts with protecting the kids or terrorism. Perceptual hashes like what Apple is using are already used in WeChat to detect memes that critique the CCP. What happens on local end user devices must be off limits. It is unacceptable that Apple is actively implementing machine learning systems that surveil and snitch on local content.
      • srswtf123 5y ago
        I agree with you 100% — the only solution I’ve found workable is limiting my use of the technology itself as much as possible.
        • ajsnigrutin 5y ago
• jhayward 5y ago
  I wonder when I gave Apple permission to do this?
  • TedDoesntTalk 5y ago
    I’m sure It’s in the TOS.
    • dannyw 5y ago
      To start scanning my local photos on my local phone and report back?
      • Causality1 5y ago
        I'm fairly sure the ToS says Apple can do whatever it wants whenever it wants as long as it isn't blatantly illegal. Recall when they decided everyone needed a Bono album on their phone without consent.
      • dylan604 5y ago
        Apple has been scanning your photos locally for quite some time now. They've been detecting faces, making collections, finding pets, etc. This is just another step in what they have already been doing.
        • kickopotomus 5y ago
          Processing data locally and maintaining information locally is wholly different from reading (supposedly encrypted) information from your device and reporting it back.
    • smnrchrds 5y ago
      Maybe the part in every TOS that says the company can change the TOS at any time without warning and you should regularly check the TOS page and stop using the service if you saw a change and didn't like it?
  • 5y ago
• BluSyn 5y ago
  I understand the hesitation here, but fundamentally this is like trying to close pandoras box. If something is technically possible to do AND governments demand it be done, it will be done. If not by Apple, by someone else. Rather than complain about it, I am interested in what alternative solutions exist, or how concerns regarding privacy and abuse of this system could be mitigated.
  • wmf 5y ago
    Apple beat government spying in the past so I don't see why they can't again.
  • Spivak 5y ago
    It’s not trying to close Pandora’s box. It’s liability limitation. They’re effectively saying that if you want to distribute CP don’t do it on an iPhone.
    • ajsnigrutin 5y ago
      They're basically saying, that they're watching all your multimedia, to see if maybe you're distributing child porn. This is like doing rectal exams on everyone, every day, because someone might be hiding drugs there.
  • ssklash 5y ago
    I don't understand this argument at all. Look at the Clipper Chip debacle in the 90s. It was technically feasible and the government very much wanted to do it. And the reason they didn't is push back from the public, saying this is a bad idea that can easily be misused, even if it does make some law enforcement things easier. I don't see how this is any different. Sacrificing the privacy of the many to help catch a relatively small amount of (admittedly some of the worst possible) criminals, while simultaneously enabling yet more effective surveillance and oppression by those inclined governments (of which there are plenty) is a pretty terrible idea. Eliminating the 4th amendment or mandating clear walls sure would make the cops' job easier. But no one thinks that's even a remotely good idea.
• vmception 5y ago
  That tweet thread is saying it will scan for hashes client side and upload the result, circumventing E2E encryption, but then says theyre just going to do it on your icloud backups because they dont have E2E encryption, so which is it? All?
  • pseudalopex 5y ago
    It doesn't say that. Today Apple servers scan uploaded photos. Tomorrow Apple phones will scan uploaded photos. The next day Apple phones could scan not uploaded photos.
    • neop1x 5y ago
      The next day Apple phones could periodically listen to the environment via the internal microphone. The next day Apple phones could take and upload photos of the environment by itself.
• vmception 5y ago
  Sucks since we pay for icloud backups and get the lesser service
  • viktorcode 5y ago
    You may turn off iCloud backups and use local backups instead.
  • quenix 5y ago
    This isn't even tied to iCloud, they can scan your device even if you don't use iCloud. iOS downloads CSAM scanning code from Apple and runs it locally—hence, client-side tool.
• edge17 5y ago
  I'm assuming android/google must do this already?
  • wmf 5y ago
    Android uploads everything to the cloud and it gets scanned there. But maybe in the future the Tensor SoC can rat you out client-side.
    • dannyw 5y ago
      No, it doesn't if you choose to keep it local.
• nbzso 5y ago
  Since Snowden I use my phone in minimalistic way. Phone calls. Minimal texting. No games. Banking apps if necessary. Treat your phones as an enemy. Use real computers with VPN and software like Little Snitch when online. Use cameras for photography and video. The benefits of this approach are immense. I have long attention span. I don't have fear of missing out. If governments wan't the future to be painted by tracing and surveillance mediated towards people trough big tech - lets make it mandatory by law. And since big tech will reap benefits from the big data they must provide phones for free. :)
  • gruez 5y ago
    >Treat your phones as an enemy. Use real computers with VPN and software like Little Snitch when online. I'm assuming your "real computer" is a mac (since little snitch is mac only). What makes you think apple won't do the same for macos? Also, while you have greater control with a "real computer", you also have less privacy from the apps themselves, since they're unsandboxed and have full access to your system.
    • nbzso 5y ago
      Not the right logic here. Check your idea more seriously. Mac os is just an example.
      • gruez 5y ago
        Your advice might be sound with the proper operating system choice, but the fact that you made such a glaring error in your initial comment makes it hard to take you seriously. It also brings into question whether you actually have a good understanding of privacy/security, or are just LARPing.
        • nbzso 5y ago
          Error? Seriously? Did you make a wrong assumption that I care? About karma? I share my experience and my point of view. Nothing more, nothing less. And the most important point is not technical. The most important point is personal habits. To overcome smartphone addiction. We put our lives, thoughts and photos on closed technology and have false expectations that someone will care for our security or wellbeing. On a serious note: If you want privacy/security as a semi-professional use: No mobile phone. https://www.qubes-os.org/ https://www.qubes-os.org/ with https://www.qubes-os.org/doc/certified-hardware/ https://www.qubes-os.org/doc/certified-hardware/
• webmobdev 5y ago
  Wow, this explains so much on why the Indian Government "withdrew" a letter seeking Apple's compliance with new IT surveillance rules - https://thewire.in/government/centre-withdrew-letter-seeking-apples-compliance-with-new-it-rules https://thewire.in/government/centre-withdrew-letter-seeking... ... The indian government has recently introduced new laws that give them power to dictate terms over many online platforms and broaden their surveillance powers over online social media and messenger platforms. One of the laws dealing with messenger platforms requires the platform to track shared content, especially "origin of content" (first originator) of any content that is shared through their network. (Facebook / WhatsApp has already gone to the court to challenge this, as it claims that they would need to break end-to-end encryption for this and it thus violates indian privacy laws). Apple's iMessage platform has more than 25 million users, and thus should come under the ambit of this law. But strangely, the indian government seems to have given them an "exception" .... and now we know why.
• etaioinshrdlu 5y ago
  Also, if they send perceptual hashes to your device - it's possible images could be generated back from those hashes. These aren't cryptographic hashes, so I doubt they are very good one-way functions. Another thought - notice that they say "if too many appear". This may mean that the hashes don't store many bits of information (and would not be reversible) and that false positives are likely - ie, one image is not enough to decide you have a bad actor - you need more. But at Apple's scale, statistically, some law-abiding users would likely get snagged with totally innocent images. Just a bad idea all around.
  • benrbray 5y ago
    It's also just plain absurd. Hundreds of pictures of my own children at the beach in their bathing suits? No problem. Hundreds of photos of other peoples' children in bathing suits? Big problem. Of course, the algorithm is powerless to tell the difference.
    • roberttod 5y ago
      I believe it's built on hashing, so it'll only find images in the db they have with already known content. Your own photos won't get mixed up.
      • benrbray 5y ago
        Ah, I guess unsurprisingly the twitter thread is light on details. I saw "perceptual hash" which I usually interpret to mean some kind of feature-based semantic hash that is not as sensitive to small edits. Even if it isn't currently used, the door is open for it to be implemented in the future.
        • nicce 5y ago
          Perceptual hashing would make bit more sense. Flip one bit from the image to avoid detection otherwise.
  • kaba0 5y ago
• 3pt14159 5y ago
  I'm really conflicted about this. For context, I deeply hate the abuse of children and I've worked on a contract before that landed 12 human traffickers in custody that were smuggling sex slaves across boarders. I didn't need to know details about the victims in question, but it's understood that they're often teenagers or children. So my initial reaction when reading this Twitter thread was "let's get these bastards" but on serious reflection I think that impulse is wrong. Unshared data shouldn't be subject to search. Once it's shared, I can make several cases for an automated scan, but a cloud backup of personal media should be kept private. Our control of our own privacy matters. Not for the slippery slope argument or for the false positive argument, but for its own sake. We shouldn't be assuming the worst of people without cause or warrant. That said, even though I feel this way a not-small-enough part of me will be pleased if it is deployed because I want these people arrested. It's the same way I feel when terrorists get captured even if intelligence services bent or broke the rules. I can be happy at the outcome without being happy at the methods, and I can feel queasy about my own internal, conflicted feelings throughout it all.
  • drvdevd 5y ago
    Since you worked on an actual contract catching these sorts of people you are perhaps in a unique position to answer the question: will this sort of blanket surveillance technique in general but also in iOS specifically - actually work to help catch them?
    • jsjohnst 5y ago
      I have direct knowledge of examples of where individuals were arrested and convicted of sharing CP online and they were identified because a previous employer I worked for used PhotoDNA analysis on all user uploaded images. So yeah, this type of thing can catch bad people. I’m still not convinced Apple doing this is a good thing, especially on private media content without a warrant, even though the technology can help catch criminals.
      • disiplus 5y ago
        now im afraid, i have two young children < 5 years old. i have occasionally took pictures of them naked with some bumps on the skin or mosquito bite and sent them to my wife over whatsapp to look at and decide do we need to send them to doctor, do i have to fear now that i will be marked as distributing CP.
• secondo 5y ago
  It's quite easy to extrapolate this and in a few steps end up in a boring dystopia. First it's iPhone photos, then it's all iCloud files, that spills into Macs using iCloud, then it's client side reporting of local Mac files, and somewhere along all other Apple hardware I've filled my home with have received equivalent updates and are phoning home to verify that I don't have files or whatever data they can see or hear that some unknown authority has decided should be reported. What is the utopian perspective of this which counterbalances the risks for this to be a path worth taking?
  • raxxorrax 5y ago
    > What is the utopian perspective of this You will make Apple tons of money.
  • Santosh83 5y ago
    > What is the utopian perspective of this which counterbalances the risks for this to be a path worth taking? Apple takes care of everything for you, and they have your best interests at heart. You will be safe, secure, private and seamlessly integrated with your beautiful devices, so you can more efficiently consume. What's not to like about a world where child crime, terrorism, abuse, radical/harmful content and misinformation can be spotted in inception and at the source and effectively quarantined?
    • SilverRed 5y ago
      No one here has a problem with the worst criminals being taken out. The problem is the scope creep that always comes after. In 2021 and 2020 we saw people being arrested for planning/promoting anti lockdown protests. Not for actually participating but for simply posting about it. The scope of what "harmful content" is is infinite. You might agree that police do need to take action against these people but surely you can see how the scope creeped from literal terrorists and pedophiles to edgy facebook mums and how that could move even further to simple criticisms of the government or religion. It's difficult to say how we draw the line to make sure horrible crimes go punished while still protecting reasonable privacy and freedom. I'm guessing apples justification here is that they are not sending your photos to police but simply checking them against known bad hashes and if you are not a pedophile, there will be no matches and none of your data will have been exposed.
• ulfw 5y ago
  I'm a little bit confused here and hope maybe some of you can clear this up. My parents took lots of photos of me as a baby/small child. Say lying naked on a blanket or a naked 2yr old me in a kiddie pool in the summer in our backyard. Those are private photos and because it was the 1970s those were just taken with a normal non-digital camera. They were OBVIOUSLY never shared with others, especially outside immediate family. Transform that into the 2020s and today these type of pictures would be taken with your iPhone. Would they now be classified as child pornography even though they weren't meant to be shared with anyone nor were they ever shared with anyone? Just your typical proud parent photo of your toddler. Sounds a bit like a slippery slope, but maybe I am misunderstanding the gravity here. I'm specifically highlighting private "consumption" (parent taking picture of their child who happens to be naked as 1yr olds tend to be sometimes) vs "distribution" (parent or even a nefarious actor taking picture of a child and sharing it with third parties). I 100% want to eliminate child pornography. No discussion. But how do we prevent "false positives" with this?
  • pseudalopex 5y ago
    The idea is it detects specific images humans classified. Not any unclothed child.
    • Santosh83 5y ago
      So far. Many websites already use NN trained to detect any nudity. It is only a matter of time before it lands on all consumer computing devices. The noose will keep on tightening because people keep debating instead of protesting.
      • pseudalopex 5y ago
        Nudity detectors are decades old. People use them when they want to block any nudity. Making iCloud servers scan for nudity would have been simpler than making this new system. And pornography is easier to access than ever for most people.
  • akersten 5y ago
• Shank 5y ago
  I was under the impression that one of the reasons why these tools aren’t available for public download is because the hashes and system can be used to design defeat mechanisms? Doesn’t this mean that someone who has an image and a jail broken device can just watch the system, identify how the photo is detected, and modify it so that it doesn’t trip the filter? PhotoDNA and systems like it are really interesting, but it seems like clientside scanning is a dangerous decision, not just from the privacy perspective. It seems like giving a CSAM detector and hashes to people is a really risky idea, even if it’s perfect and it does what it says it does without violating privacy.
  • pornel 5y ago
    I see it as a huge risk too. If the algorithm and the blocklists leaked, then not only it would be possible to develop tools that reliably modify CSAM to avoid detection, but also generate new innocent-looking images that are caught by the filter. That could be used to overwhelm law enforcement with false positives and also weaponized for SWAT-ing.
    • pornel 5y ago
      Fortunately, it seems that matching is split between client-side and server-side, so extraction of the database from the device will not easily enable generation of matching images. https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
• h_anna_h 5y ago
  It is lovely when your "own" device is working against you to catch if you are in possession of illegal numbers https://en.wikipedia.org/wiki/Illegal_number https://en.wikipedia.org/wiki/Illegal_number. And surely we can trust Apple that it will only be used for this kind of content instead of for example government leaks.
• sebyx07 5y ago
  4 good reasons why you should buy a iphone - (Check) fragile - (Check) best monopoly app store - (Check) high price - (Check) phones the police in case of a sha collision
  • SilverRed 5y ago
    I seriously doubt Google is going to have a different opinion on this. They already scan google drive content for mere copyright violations.
    • mhitza 5y ago
      I just assumed the entire FAANG group scanned user content for CP already. I mean, EU recently passed a vote that extended the permission for companies to be able to scan and report user content for this purpose without it being considered a privacy violation [1] (first introduced in 2020). And I recall MS[2]/Google[3] being open about this practice way in the past. Personally I somehow doubt that MS/Google weren't scanning private content (aka not shared) for this type of material. But can't have transparency with these behemoths. [1] https://www.euronews.com/2021/07/07/eu-adopts-temporary-rules-to-detect-and-report-online-child-abuse https://www.euronews.com/2021/07/07/eu-adopts-temporary-rule... [2] https://www.theverge.com/2014/8/7/5977827/microsoft-tips-off-police-for-child-porn-arrest https://www.theverge.com/2014/8/7/5977827/microsoft-tips-off... [3] https://www.theverge.com/2014/8/5/5970141/how-google-scans-your-gmail-for-child-porn https://www.theverge.com/2014/8/5/5970141/how-google-scans-y...
• skee_0x4459 5y ago
  how the fuck am i supposed to know if that image i downloaded from some random subreddit is of a girl who is 17.98 years old? how long until we just use a NN to identify images of children automatically? she looks pretty young so i guess you will get disemboweled alive in prison? what is stopping someone from planting an image on your phone or a physical picture somewhere on your property? im so tired of this fucking dogma around child porn. you can always identify the presence of dogma by the accompanying vacuum of logic that follows in its wake. a teenage girl can go to jail for distributing pictures that she took of herself. do i even need to say more?
• somuchlan 5y ago
  Well this really debunks my common phrase “Apple is a Privacy company, not a Security company” I can’t say I’m surprised they are implementing this (if true), under the radar. I can’t imagine a correct way or platform for Apple to share this rollout publicly. I’m sure nothing will come of this, press will ignore the story, and we all go back to our iPhones
  • no_time 5y ago
    Apple, like all companies is a Money company.
    • mdp2021 5y ago
      And a "subject to other powers" company.
    • account42 5y ago
      This is really a pointless comment - yes all companies are ultimately there to make money, but that does not mean always blindly doing whatever makes the most money in the short term. Clearly apple sees value in marketing themselves as privacy friendly.
  • marticode 5y ago
    Apple only claim to care about privacy because they couldn't manage to compete with Google on running ad-serving cloud service. Since they couldn't sell ads in meaningful number, they figured they might as well brag about it. But Apple iCloud for ex doesn't intrude on your privacy any more or less than Google Photos.
• fencepost 5y ago
  For those not familiar with the acronym, CSAM = Child Sexual Abuse Media
• js2 5y ago
  I'm not worried about China. I'm worried about the U.S. This is a step along the path to the Buttle/Tuttle dystopia that Brazil warned us about.
  • KirillPanov 5y ago
    I've filed a twenty seven B stroke 6 reporting this comment.
• akersten 5y ago
  I really have to wonder why Apple chose to do this. As far as I know, this kind of scanning is not legally mandated. So, either they think that this will truly make the world a better place and are doing it out of some sense of moral responsibility, or they've been pressured into it as part of a sweetheart deal on E2E ("we won't push for crypto backdoors if you'll just scan your users' phones for us"). Either way it doesn't thrill me as a customer that my device is wasting CPU cycles and battery life under the presumption that I might possess data my current jurisdiction deems illegal. For all the acclaim privacy-forward measures like GDPR get here, I'm surprised there isn't more outright repudiation of this frankly Orwellian situation.
  • tomaskafka 5y ago
    Sales on China and wealthy arabic totalitarian regimes. Customer asks, customer gets.
• tinus_hn 5y ago
  According to this this has been a thing for quite a while: https://fightthenewdrug.org/apple-fights-child-porn-by-scanning-users-uploaded-icloud-photos/ https://fightthenewdrug.org/apple-fights-child-porn-by-scann...
• deleted 5y ago
  [deleted]
• akersten 5y ago
  If you like this, I have some other innovations that you may be interested in: * A car that automatically pulls over when a police cruiser attempts to intercept you * A front door that unlocks when a cop knocks * A camera that uses AI to detect and prevent the photography of minors, police, and critical infrastructure * A Smart TV that counts the number of people in your living room to ensure you aren't performing an unauthorized public broadcast of copyrighted content Surely, at least one of those sounds ridiculous to you. As well-intentioned as this scanning may be, it violates a core principle of privacy and human autonomy. Your own device should not betray you. As technologists, just because we can do something doesn't mean we should.
  • nostromo 5y ago
    No need for hypotheticals, 2020 was a huge win for the police state. The UK and Israel allowed cops to monitor cell phone locations to crack down on unlawful gatherings in private homes. https://www.theguardian.com/world/2020/mar/17/israel-to-track-mobile-phones-of-suspected-coronavirus-cases https://www.theguardian.com/world/2020/mar/17/israel-to-trac... The problem with allowing this is that you’re paving the way for future tyrants to use it against us.
    • hulitu 5y ago
      > The problem with allowing this is that you’re paving the way for future tyrants to use it against us. It's funny how everybody talk about the future. This is happenning now. Remember how a certain german guy took the power some 90 years ago ? He was elected.
      • leipert 5y ago
        Not your point, but: Technically he was Austrian by birth, stateless sind since 1925 and tried at least 7 times to get German citizenship before elected in 1932.
      • sharken 5y ago
        I trust we live in a different world today, but it is concerning to read about mandated vaccines to get a job and mandated contact-tracing in countries such as Germany.
• JackeJR 5y ago
  The slippery slope argument is that the use of this method on private files, i.e. not shared with others except for the service provider can legitimise the expansion of such scamming scopes. While this argument can and have indeed happened in other instances, this is akin to saying that we should not give anyone any powers to do anything because it is a slippery slope that they can use it to do bad things. What then sets out the difference between what a slippery slope and a non-slippery one is? Checks and balances and the history of USA have shown that this is indeed what can reign in the worst instincts of any single entity. History of course have also shown when these failed and these should serve not as a reason to reject the idea of checks and balances but as acknowledging it's imperfection and think of ways to mitigate it.
  • Calvin02 5y ago
    I think the checks and balances are pretty fragile and very susceptible to public opinion. Two instances: 1) Post 9/11 Patriot Law 2) McCarthy era: https://www.e-ir.info/2011/11/03/the-extraordinary-injustice-of-mccarthy’s-america/ https://www.e-ir.info/2011/11/03/the-extraordinary-injustice...
    • JackeJR 5y ago
      Sure. And as I mentioned, there will be screw ups along the way. As with any new capability/tech be it nuclear power or recombinant DNA or ability to locate CP, there can be legitimate uses that we can rally behind and ways for them to be abused. Checks and balances are never a done deal. If we reject checks and balances and as a result reject new tech because of abuse potential, how then should we as a civilisation advance?
• deleted 5y ago
  [deleted]
• lenkite 5y ago
  This is terrifying. The possibilities of extraordinary abuse are endless. What's surprising to me is the complete lack of media focus on this topic ? Why isn't this being hotly debated on TV ? Scanning people's photos is just OK now ? Back to an Android phone, once I confirm this story is true.
• anonymouswacker 5y ago
  Soon enough this will scope creep into anything containing what a puritanic neoliberal corporation considers contraband. Time for a serious look at Linux phones.
  • yosito 5y ago
    > Linux phones Contraband detected
• rinron 5y ago
  This matches up with how I view Apples corporate thinking. "we know what's best" "the consumer is not to be trusted". Apple limits access to hardware, system settings, they block apps that don't meet moral standards, are "unsafe", or just might cause apple to not make as much money. They do not significantly care what people say they want after all they know best. A lot of people love not having options and having these decisions made for them. I would never want a device like that or with something that scans my device, but I think the vast majority of their customers if they even hear about it will think "I trust apple, they know what's best, it wont affect me" Im ok with apple doing it because i think most apple users will be ok with it. I would not be ok with it if all Android devices started doing it though.
  • jimbob45 5y ago
    That’s sort of the genius of Apple though, isn’t it? They make products that hold your hand tighter than any designer but their marketing department obscures that fact as much as possible with their “think different” campaigns. It’s less “I trust Apple” and more “if I really cared, I’d have bought from another designer”.
  • pndy 5y ago
    This isn't exclusive to Apple - Microsoft recently decided that starting from August Windows Defender will have the option for blocking PUAs enabled by default for those users who doesn't have other third-party security software [1]. This also I belive falls under "we know what's best" and "the customer is not to be trusted" or "is too stupid to run things by on its own". This does looks good on paper - caring for customers and their security, peace of mind but tomorrow it might be a total vendor-lock with no ways of installing any other software than one approved by the corporate entities. [1] - https://www.ghacks.net/2021/08/03/windows-10-blocks-potentially-unwanted-apps-by-default-now/ https://www.ghacks.net/2021/08/03/windows-10-blocks-potentia...
    • kaba0 5y ago
      I don’t see why a circumventable default block of random executables is bad. People (even technically adept ones, who are a small minority) are very easy to fool. Depending on how easy it is to allow execution of such a program (which is a UX problem) it can indeed be what prevents a botnet on a significant number of computers.
• AHOHA 5y ago
  >Twitter for iPhone Classic
• KirillPanov 5y ago
  Could we get a more useful link here, people? CSAM? According to Google that's Confocal Scanning Acoustic Microscopy. Or something. And what with the tweeters? I think my laptop just gave me thighburns from the CPU bloat that clicking on that link caused. Eight seconds to render the page? Why do folks still use this twerker website?
  • pas 5y ago
    CSAM = Child Sexual Abuse Material
  • quenix 5y ago
    If your machine takes 8 seconds to render a Twitter page, perhaps you need a new machine.
    • jason0597 5y ago
      Or maybe the web is getting too bloated? https://idlewords.com/talks/website_obesity.htm https://idlewords.com/talks/website_obesity.htm
      • quenix 5y ago
        Those two things aren't mutually exclusive.
        • KirillPanov 5y ago
          But only one of those things is relevant to the discussion. Hint: the relevant thing wasn't to do with my laptop.
• bennyp101 5y ago
  Will this work differently depending on what country you are in? For instance, back in 2010 there was that thing about Australia ruling that naked cartoon children count as actual child porn. [1] It's perfectly legal elsewhere (if a bit weird) to have some Simpsons/whatever mash-up of sexualised images, but if I flew on a plane to the land down under, would I then be flagged? edit: If this is scanning stuff on your phone automatically, and you have whatsapp or whatever messenger set to save media automatically, then mass texting an image that is considered 'normal' in the sender country, but 'bad' in the recipients, you could get a lot of people flagged just by sending a message. [1] https://arstechnica.com/tech-policy/news/2010/01/simpsons-powerpuff-girls-porn-nets-jail-time-for-australian.ars https://arstechnica.com/tech-policy/news/2010/01/simpsons-po...
  • konschubert 5y ago
    Get ready for the witch hunts
  • deleted 5y ago
    [deleted]
• deleted 5y ago
  [deleted]
• deleted 5y ago
  [deleted]
• implying 5y ago
  A new aspect of this is that because this is self-reported, and the end goal is to involve the criminal justice system, there is now (essentially) an API call that causes law enforcement to raid your home. What would be the result of 'curl'ing back a few random hashes as positives from the database? Do I expect to be handcuffed and searched until it's sorted out? What if my app decides to do this to users? A malicious CSRF request even?
  • srj 5y ago
    A report to the cybertips line does not equal a police raid. Unfortunately the scale of the problem and the pace of growth is such that only the worst of the worst content is likely to be prosecuted.
    • the_other 5y ago
      If a phone calls the API "hello, I found some porn here" the phone (and/or it's owner) become a "person of interest" very quickly. (I'll wager) The majority of these calls will be false positives. Now a load of resources get deployed to keep an eye on the device's owner, wasting staff time and compute, wasting (tax funded) government budget that could have gone towards proper investigation.
      • deleted 5y ago
        [deleted]
    • TheFreim 5y ago
      Yeah and sadly many of those who are consumers of illicit content get away with it because it's much more important to target the creators. The unfortunate reality of finite resources.
• chobytes 5y ago
  Ive already been itching to de-cloud, and de-tech my life. If were already getting to this stage of surveillance I guess thats just another sign I should be getting on top of it. Today its csam. Tomorrow "misleading information". etc.
  • majjam 5y ago
    Im looking to do the same - this pandemic has made me feel quite claustrophobic about the encroachment of tech, and work into my personal life. Im planning on getting a dumb-ish nokia phone and leaving my smartphone at home to try and wean myself off it. What are your plans?
    • chobytes 5y ago
      Similar. Ive been looking into "dumb phone" options, and moving many tasks to old school solutions. Eg desk calendars, notepads, physical books, etc. For tech stuff I do need Id like to pick dedicated "appliance like" tools, but each of those requires some research.
• choeger 5y ago
  Sorry to say that, but stuff like this has to happen at some point when people don't own their devices. Currently, nearly no one owns their phone and at least EU legislation is underway to ensure that it stays this way. The next step will be to reduce popular services (public administration, banking, medicine) to access through such controlled devices. Then we are locked in. And you know what? Most people deserve to be locked in and subject to automatic surveillance. They will wake up when their phone creates a China-Style social score automatically, but then it will be far too late. It's a shame for those people that fought this development for years, though. But the "I have nothing to hide" crowd deserves to wake up in a world of cyber fascism.
  • bruce343434 5y ago
    Yeah but I don’t and I don’t like being collateral damage.
  • no_time 5y ago
    >EU legislation is underway to ensure that it stays this way. which one?
    • choeger 5y ago
      Devices with radio capabilities (i.e., all mobile devices) must be designed to prevent executing "unauthorized" software.
      • no_time 5y ago
        I'm asking for a specific proposal. I'm interested in reading it. Searching for "EU legislation unauthorized software" yields no results.
      • mdp2021 5y ago
        What do you mean, authorized by whom? The software applications I run have not been greenlit by any third party. Think about a model in which you code, than wait for authorization before deployment... The worst pulp novel. What exactly are you talking about, the OS?
• derefr 5y ago
  Huh. I always took the cynical view and assumed that this was something every proprietary OS was already doing, and that this was part of why dark-web die-hards were so insistent on using TAILS. Guess "not yet." On another note—OSes may only be starting to do this, but that same cynicism still leads me to presume that arbitrary closed-source third-party apps — or even closed-source binary distributions of open-source apps (e.g. the App Store versions of FOSS software) — could have been quietly scanning the files people are passing them for CSAM for years now, without telling users about it. It always seemed to me like the kind of thing it'd make sense to quietly slip into a media player like VLC.
• VoodooJuJu 5y ago
  Even when this reaches its final conclusion, policing copyrighted and political content, people will still be content to use their i-spy-devices. The future is grim; it's now.
  • sharken 5y ago
    The idea of this scanner is good, but as you say it's almost certainly bound to evolve into something else. A court order should be mandatory, not this blanket scanning. Thankfully, there is the option to steer clear of Apple devices.
• figassis 5y ago
  And with this, the fear politics are in effect. Just from reading the comments it seems one can no longer be 100% sure their phone is clean. So people will live in constant fear that on some random Tuesday the cops will come knocking, your reputation will be destroyed and in the end when you’re cleared, you will have incurred incredible financial and mental costs. This is just aside the fact that your phone should be your phone and no one should be allowed.
• SrslyJosh 5y ago
  This will go great with zero-click iMessage exploits like this one: https://9to5mac.com/2021/07/19/zero-click-imessage-exploit/ https://9to5mac.com/2021/07/19/zero-click-imessage-exploit/ Edit: Actually, this won't even require an exploit if they also scan media for people who have enabled "iMessage in iCloud". Just send someone an image in the DB (or an image that's been engineered to generate a false positive) and wait for them to get raided.
  • drvdevd 5y ago
    One could envision false positive images that don’t even display in iMessage when sent or that are nested in other file types, etc.
  • tomaskafka 5y ago
    Yup, there is now a single API call for planting an evidence onto dissident's phone and sending a SWAT team to retrieve him. Authoritarian regimes love this.
• SrslyJosh 5y ago
  Betcha the Chinese government already has their own DB of hashes that they want to scan for.
• DavideNL 5y ago
  This is just horrible… the people who actually abuse children and download such photos will now stop using Apple devices, and now the rest of us is vulnerable to misuse/abuse/corruption. Instead of specifically targeting suspects, everyone is surveyed by default. Welcome to a world of mass surveillance.
• max_ 5y ago
  On the light side perceptual hashing is actually very interesting technology. If you're interested I suggest you also have a look at photo DNA[0] [0]: https://www.microsoft.com/en-us/photodna https://www.microsoft.com/en-us/photodna
• viktorcode 5y ago
  What I find disturbing is that almost all commenters here took that rumour for a fact. There's nothing to substantiate it, there's no evidence of scan actually happening, and there's no historical precedence of similar thing done by Apple. And yet, people working in tech with supposedly developed critical thinking took the bait. Why? Is it simply because it fits their world view?
  • digisign 5y ago
    The world has changed. May not be completely accurate yet, but fits 2021 like a glove: https://sneak.berlin/20210202/macos-11.2-network-privacy/ https://sneak.berlin/20210202/macos-11.2-network-privacy/ https://sneak.berlin/20201112/your-computer-isnt-yours/ https://sneak.berlin/20201112/your-computer-isnt-yours/
  • kylehotchkiss 5y ago
    I found another source (https://www.macobserver.com/analysis/apple-scans-uploaded-content/ https://www.macobserver.com/analysis/apple-scans-uploaded-co...) saying apple was already running these scans on iCloud using homomorphic encryption… in 2019. It doesn’t really make sense for them to run it on device. Apple has the keys to unlock iCloud backups on their server and a sizable portion of users have those enabled, so why bother to run these on device? I’m not sure if it’s a rumor or not but there was a thread on HN the other day about Facebook exploring homomorphic encryption for running ads on WhatsApp and I wonder if wires got crossed?
    • pseudalopex 5y ago
      The article said they could see Apple using homomorphic encryption. Not currently. Scanning on phones is a step toward end to end encryption for photos probably. Or scanning not uploaded photos. Or both.
  • drvdevd 5y ago
    You’re right of course but I think in this case it was due to the reputation of the poster on Twitter. At least, that’s the only reason I would take this rumor seriously. But yeah, a rumor is a rumor still.
• vincnetas 5y ago
  This might be unpopular opinion but catching people sharing CP images is like catching end users of drugs. Yes it's illegal but the real criminals are the ones producing drugs. But it's very difficult to get to them, so you just arrest end users. Another side note is about near future when someone comes up with synthetic CP images, will they also be criminalised?
  • aemreunal 5y ago
    It's not just unpopular, it's also wrong: when you use drugs, you are almost entirely harming yourself (leaving aside funding all sorts of illegal activities, just focusing on the act itself). When you propagate CSAM material, you are causing psychological harm to the victims, plus can cause them to physically harm themselves or get harmed by others. So you are a criminal, harming a victim as well. You can read about this directly from a victim via this NYT article: https://www.nytimes.com/2020/12/04/opinion/sunday/pornhub-rape-trafficking.html https://www.nytimes.com/2020/12/04/opinion/sunday/pornhub-ra...
    • handoflixue 5y ago
      How would a victim of CSA ever find out that I downloaded a particular file? Surely the harm there is caused by the distributor, not the consumer. Conversely, when I use drugs, I'm paying someone, so I'm actually directly funding criminals. Depending on the country and the drugs, this is often putting cash in the hands of a very violent cartel.
      • h_anna_h 5y ago
        You are correct, however vincnetas made the comparison between distributing CSAM and buying drugs, it is not aemreunal's fault, you are replying to the wrong person. A comparison that would make better sense would be to compare these in possession of CSAM and these that buy drugs.
  • tpxl 5y ago
    Consumption encourages production.
• yosito 5y ago
  What happens when a theocracy demands that Apple check for hashes of images that disrespect their prophet? To me this sounds potentially more scary and distopian than surveillance in China. But if I'm honest, I don't know that China isn't scanning citizens' devices for illegal hashes.
• mikkelam 5y ago
  Honestly, it's probably about time to switch to GrapheneOS or LineageOS
• makach 5y ago
  Well, this is very problematic for a privacy concerned company. Under no circumstances do I want Apple to scan my private files/photos, aspecially so if it means that an alarm can allow someone to determine if it is a positive or a false positive. Also, this functionality isn't something they should be able to implement without telling their end users. It is also problematic because it will just make the cyber criminals more technical aware of what counter measures they must take to protect their illegal data. The consequence is very bad for the regular consumer: the cyber criminal will be able to hide, and the government has the possibility to scan your files. End consumer lose, again.
• keyme 5y ago
  Dear humans, 1) You willingly delegated the decision of what code is allowed to run on your devices to the manufacturer (2009). Smart voices warned you of today's present even then. 2) You willingly got yourself irrevocably vendor-locked by participating in their closed social networks, so that it's almost impossible to leave (2006). 3) You willingly switched over essentially all human communication to said social networks, despite the obvious warning signs. (2006-2021) 4) Finally you showed no resistance to these private companies when they started deciding what content should be allowed or banned, even when it got purely political (2020). Now they're getting more brazen. And why shouldn't they? You'll obey.
  • ben_w 5y ago
    (1) happened with the first multitasking OS, or possibly when CPUs got microcode; Android and iOS are big increases in freedom in comparison to the first phones. (2) and (3) are bad, but a tangential bad to this: it’s no good having an untainted chat layer if it’s running on an imperfect — anywhere from hostile to merely lowest-bidder solution — OS. (And most of the problems we find in software have been closer to the later than the former). (4) for all their problems, the American ones held off doing that until there was an attempted coup, having previously resisted blocking Trump despite him repeatedly and demonstrably violating their terms.
    • majewsky 5y ago
      Re (1): That's technically true, but missing the point when viewed holistically. Those first feature phones were mostly just used to make quick calls to arrange an appointment or discuss one or two things. They were not a platform to mediate a majority chunk of our social lifes like today's phones are.
      • ben_w 5y ago
        That also seems to miss the point, as for most of the stuff you’re describing the phone is a thin client and the computation is on a server, and that would still be true even if the phones themselves ran only GPL-licenced code and came with pre-installed compilers and IDEs.
• tomytosian 5y ago
  soon youll be in jail for a meme against the leading political party and the leading line of thought that the press carries.
• SXX 5y ago
  It's funny to see anyone here could find this acceptable. I wonder what's comments would be after Apple start to scan phones for anti-censorship or anti-CCP materials in China. Or for some gay porn in Saudi Arabia. Because you know in some countries there are materials that local government find more offensive than mere child abuse. And once surveillance tech is deployed it's certainly gonna be used to oppress people.
  • meibo 5y ago
    Won't anyone think of the children! And Tim Cook personally promised to not look at anything in my unencrypted iCloud backup, they really care about privacy! And you can be sure that there's no way for the PRC, that already runs its own iCloud, to use this. America's favorite company wouldn't allow that.
  • croes 5y ago
    If you don't, you are suspicious.
  • deleted 5y ago
    [deleted]
  • azalemeth 5y ago
    > Because you know in some countries there are materials that local government find more offensive than mere child abuse. And once surveillance tech is deployed it's certainly gonna be used to oppress people. In Saudi, Bahrain, and Iran there is no minimum age of consent – just a requirement for marriage. In Yemen, the age of consent for women is 9 (but they must be married first). In Macau, East Timor, and UAE, it's 14. [1] I would allege that in all of those states they would probably find the perceptual hash of government criticism far more important to include on the "evil material" database than anything else. [1] https://en.wikipedia.org/wiki/Ages_of_consent_in_Asia https://en.wikipedia.org/wiki/Ages_of_consent_in_Asia
  • antihero 5y ago
    Exactly, now Apple has this tech, shady governments know they can mandate Apple to use it for their own databases and Apple will have to do this if they want to keep operating within a territory.
• kota2 5y ago
  If you have not already, please check out the wonderful alternatives to iOS/Android that do not trample over your privacy, see https://grapheneos.org https://grapheneos.org or Debian for mobile (which is still very much a WIP). It is simply unacceptable for a company or government to scan your property that you not allow.
• hughrr 5y ago
  Ah I’m done now. That’s one step too far. When something overreaches into my data and risks integrity and privacy problems it’s game over. So which Linux desktop sucks the least at the moment?
• gorgoiler 5y ago
  Every so often I feel a wave of revulsion that the computer I use the most — my iPhone — is an almost completely closed system controlled by someone else. Contrast this with my desktop where, in the press of a few buttons, I am presented with the source code for the CPU frequency scaling code. Bring on the Linux phones.
  • majjam 5y ago
    Can you recommend one?
    • GeckoEidechse 5y ago
      Either a PinePhone or a Librem 5. There's not much more choice than that atm.
      • kaba0 5y ago
        And neither are usable as daily driver for the majority of people. I really don’t want to wait half a minute for taking a picture, and would prefer if the phone would not be untouchably warm after the fact.
      • alecco 5y ago
        Some people argue Google Pixel (and Chromebooks) can be fully controlled with a custom key.
        • SXX 5y ago
          Do they have baseband isolation? Without an option to at least isolate this black box all else is worthless.
• thepimp32 5y ago
  Great. I'm curious what would happen if you have auto-save received images enabled in WhatsApp and someone would spam you with child pornography images.
• deanclatworthy 5y ago
  The terrifying part about this is potential abuse. We have seen people arrested for having child porn in their web cache just from clicking on a bad link. I could inject your cache with any image I want using JS. Presumably the same could apply to your phone. Most messengers save images automatically. I presume the images are immediately scanned against hashes once saved. And the report is immediately made if it passes the reported threshold. There’s no defence against this. Your phone number is basically public information and probably in a database somewhere. You have no protection here from abuse, if you’re a normal citizen. I bet most people don’t even turn the auto save setting off on WhatsApp.
  • sildur 5y ago
    I've read that people intentionally bomb telegram groups with CP. Telegram automatically downloads shared images. This is going to be a crapfest.
    • ornornor 5y ago
      Maybe this is a necessary crapfest though, to demonstrate how absurd these measures are.
• srmarm 5y ago
  You demo this tech working with child porn, it maybe shows it's worth with some Isis training videos but before long China will be demanding access on their terms as a condition of accessing their markets. And at that point the well meaning privacy advocate who worked hard to get some nice policies to protect users is booted off the project because you can hardly tell the shareholders and investors who own the company that you're going to ignore $billions in revenue or let your rival get ahead because of some irrelevant political movement on the other side of the world. It's happened plenty of times before and it'll happen again.
• trangus_1985 5y ago
  This is horrifying. Does this only affect iMessage, or the photos library? Is it remote? Does it require physical access? As I understand it: it's a tool (that sends a command of some sort) that compels an iphone to perform the hashing match operation, and output results. Is that correct? Does it notify the user? If I had to build it within apple's privacy framework, that'd probably be my approach: remote command causes sepos unlock of photos library (even running the job on sepos?) to do photo scanning. sepos returns hashes that match
• gizdan 5y ago
  I'm gonna go out on a limb here. At the end of the day laws are relative so to say. The thought behind such a system is noble indeed, but as we've seen, anything any government gets their hands on, they will abuse it. Classic example being PRISM et al. In theory it's great to be able to catch the bad guys, but it was clearly abused. This is from countries that are meant to be free, forward thinking etc, not any authoritarian regimes. People in this thread are asking what Saudi Arabia, China etc will do with such power that Apple is adding, you bet your ass that they'll use it for their own gain. I want to believe in such systems for the good. I want child abusers caught. But a system that equally can be abused by the wrong people (and I guarantee you that will be western countries too) ain't it.
  • fragileone 5y ago
    It's not even hypothetical, it's already known that Apple has to use servers operated by China for their operations there [1] so this capability will be fully within their hands now too to arbitrarily censor and report iPhone users for any material they want to disallow. [1] https://www.businessinsider.com/apple-data-china-censors-apps-nyt-2021-5?op=1&r=US&IR=T https://www.businessinsider.com/apple-data-china-censors-app...
• tomaskafka 5y ago
  Saudis: We want a list of everyone who ever shared a photo of Khashoggi (no matter in which app). Apple: Say no more, here they are. Hope you won't imprison all of them, as that would decrease our services revenue substantially, lol. Also Apple: Privacy is a human right, buy more iphones.
• f6v 5y ago
  What about people storing their children's photos on the iPhone? How would a system differentiate those?
• deleted 5y ago
  [deleted]
• r721 5y ago
  Follow-up thread: https://twitter.com/matthew_d_green/status/1423091097933426692 https://twitter.com/matthew_d_green/status/14230910979334266...
• layoutIfNeeded 5y ago
  If you really believe this will stop at child porn, then I have a bridge to sell you.
• andy_ppp 5y ago
  How easy is it to generate an image that has the same “perceptual hash” or whatever that are calling it? My guess is it has to be easier than cracking a non fuzzy hash? Do we know the algorithm they are using?
  • r721 5y ago
    No, it's undisclosed: >Hashes using a new and proprietary neural hashing algorithm Apple has developed, and gotten NCMEC to agree to use. >We don’t know much about this algorithm. What if someone can make collisions? https://twitter.com/matthew_d_green/status/1423079258516344833 https://twitter.com/matthew_d_green/status/14230792585163448...
    • andy_ppp 5y ago
      How absurd, insecurity through obscurity.
    • hda2 5y ago
      > neural Oh boy..
• danielktdoranie 5y ago
  Source: "trust me bro". By all means don't let me interrupt the circle jerk.
• danielktdoranie 5y ago
  source: "trust me bro".
• bitcurious 5y ago
  Remember earlier this year when bing.com returned an empty page for the image search of tank man? We’re now moving towards a world where your phone can show you that same blank page.
• JanneVee 5y ago
  So what would the process be for example if a unwitting parent or relative has pictures of a victim on their iPhone that is perceptually similar of their loved one being molested at day care something like that?
• intunderflow 5y ago
  Apple have separate operations in China, how long until anti-party content is on the hash list over there
• wayneftw 5y ago
  What if I just don't want this feature using my battery time or my network bandwidth? They have to download the hashes in order to compare them, I wonder if a pihole could help here?
• hamilyon2 5y ago
  So, while everyone discusses how horrible future dystopia will be. I worry about the method itself: will a simplest of firewalls be very effective against this? The one that forbids any communication except with few hosts, like a pair of them?
• vslira 5y ago
  I won't get into the CSAM discussion but for anyone that has a stash of non-DRMed content I think it's a good idea to look into alternatives to Apple devices. Sooner rather than latter the same kind of system will be auto-deleting or alerting authorities about copyrighted material and I doubt that too much care will be taken to ensure that you didn't actually have a right to those copies.
• deleted 5y ago
  [deleted]
• Angostura 5y ago
  So many questions that akes this Tweet look odd. It's a "Client side tool" - so what? An app you install? That law enforcement can install? That Apple can silently install. It lets "Apple Scan"? So Apple is going to proactively scanning your photos using a tool then install? So many questions about this. It doesn't add up.
• hanselot 5y ago
  Tomorrow's headline: Unknown drive for freedom phone among the youth as FAANG roll out CSAM.
• hexo 5y ago
  Thank for letting me know. Now I know I'm not getting Apple smartphone.
• PostThisTooFast 5y ago
  Whatever THAT is.
• croes 5y ago
  Can we get a statistic how many people switch to Android tomorrow, separated by profession, politicians for instance?
• croes 5y ago
  iPhone only? Why not iPad and iMac too?
• zomglings 5y ago
  How do they determine if an image is child porn? My wife has an iPhone and we take pictures of our baby daughter on it, sometimes in diapers and sometimes naked. Our intentions are not pornographic but now I am worried about apple's algorithm flagging them as such. Has apple published its training data somewhere?
  • chimen 5y ago
    It just gathers hashes without judging or interpreting. This is the first phase. When a child porn picture is discovered and inquired about, they just compare the hash of it with what they have on database and see who had that picture on their phone as well, allowing them to build a nicely timelined trace of it and even discover the first source.
    • zomglings 5y ago
      Wouldn't their scanner be defeated by a tool that randomly modified rgba values for a small percentage of pixels in an image? This is something that child porn enthusiasts using apple would invest their time and money into. A regular person would not think to do this (yet).
• pcdoodle 5y ago
  I am done buying apple products. This is the final straw.
• chimen 5y ago
  These days you can trump on anyone's (right to) privacy, freedom or general rights for that matter with just 2 keywords: - terrorism - child pornography Try to protest it and you will be prompted with a nice "do you have anything to hide?" question by the masses. The advertised intention of these tools could not be farther from the truth and people happily fills their pockets at each new launch.
• option_greek 5y ago
  This will end being a nightmare due to false positives especially for parents with kids.
• oever 5y ago
  General Failure reading drive A:
• eitland 5y ago
  Ok. I can say this since I don't have anything to hide (edit: 1. that I am aware of and 2. yet). I switched to the Apple ecosystem 2 years ago and have been extremely happy. I couldn't see a single reason to switch back. Today that reason came. What goes on on my phone is my business. I guess fairphone next. Again, I think have nothing to hide now so I can sat this loud and clear now. Given what recent elections have shown us we cannot know if I have something to hide in a few years (political, religious? Something else? Not that I plan to change but things have already changed extremely much since I was a kid 30 years ago.)
• pokot0 5y ago
  Everyone worried about Apple, but Apple is company and follows the laws of the countries it operates in. If this is legal, or even more required by the goverment agencies: Apple will comply. And there is nothing wrong with it. If you don't like the laws of your country, you need to work for that, not just go after Apple on social media.
• egberts 5y ago
  Soviet Apple sees you.
• alsetmusic 5y ago
  I scanned the comments to find out who this person is and how they would have any inside info and found nothing. Why is this person’s claim being taken at face value? Before debating the merits of Apple scanning photos / hashes, why does anyone believe this is true?
• devwastaken 5y ago
  This will be used for anti-piracy, government censorship, and targeted attacks, as always. There's no such thing as "were only scanning for CP". By creating the tool the company can be compelled to use the tool in other ways by U.S. or foreign governments. Apple already complies with anti-lgbt countries and will change their app store to suite each one of them. What happens when they're required to also scan for LGBT materials? They'll comply, because apple doesn't actually have morals. Ontop of this, it gives apple far too much power. What happens when someone they don't like owns an iphone? They can pull an FBI and put the content onto the device, and having it then "automatically detected".
• somedude895 5y ago
  Will they have to update their EULA or something before they have it communicate to their servers? I hate everything about this and would like to know when it actually happens. So far it's just a rumor.
• jl2718 5y ago
  I would like to hear the strongest case for the privacy trade-off. How many more children will be physically recovered versus existing methods? What is the reduction in money flow to abduction activities? This might be naive, but I would guess that the best way to fight this kind of thing is to let people know more of the case details. People would protect themselves, find the crimes, and stop unwittingly supporting them. For instance, if it can be shown that cryptocurrency or encrypted messengers are used to a significant extent, the community will either find a technical solution, or stop using it.
• jpe-210 5y ago
  How does one own/use an iPhone and help mitigate any issues from this? How does one help prevent this kind of sneaky photo crawling? I feel like in order to prevent people from spying on me I have to change _everything_ I do on my phone/computer.
• sydthrowaway 5y ago
  Why do people think this will result in immediate abuse by corrupt governments as opposed to any other Apple service? Just because anime avatar twitter says so?
• vegetablepotpie 5y ago
  How likely are perceptual hashes to give a false positive? If I take a picture of a tree, how likely is it that a few pixels are going to line up just right in a hashing algorithm and say it might be child porn? How likely is it that law enforcement is going to understand the limitations of this technology? How likely is it that the judicial system will understand the limitations? I can see law enforcement showing up at my door one day with a search warrant demanding to have a look around, and I would have no idea why they’re there, but they’ll want to look through all my personal belongings. Worse yet, I might come home from work one day, see my windows broken, see my place has been ransacked and my computers are missing. I would call the police to report a burglary only to hear than that I’m under investigation and they need me to give them the key to decrypt my hard drives.
• Cyril_HN 5y ago
  Mac up next, I bet.
• LinuxBender 5y ago
  Does this not encourage spawning a new arms race? New or modified apps that randomly change hashes of multimedia files as they are stored? If the CSAM DB is just simple hashes like sha256, md5/md4, etc.. then evading detection would be trivial. Or would Apple block applications that could rewrite randomized data into files? People don't have to be against CSAM to dislike something scanning their devices and many developers love puzzle challenges. I assume perhaps incorrectly that whatever app is doing the scanning could potentially also accept additional hash DB's, allowing Apple to enable categories to detect on per region. One of the iPhone emulators should facilitate reverse engineering the application.
  • James-Livesey 5y ago
    The hash is a pictorial representation of the image, and not quite a checksum of the raw file data (like MD5 etc.). I would expect that even photos of printed photos would still have the same pictorial hash (if the photos are properly aligned), where obviously the cryptographic hash would be much different (since it's not an exact replica of the original image) but in the ML's eyes (bearing in mind the pictorial hash is generated through machine learning afaik), there would be a very strong match between visually similar images. I suppose that it's a bit like when you do a reverse image search on your favourite search engine. When you upload an image, the engine will try and find images that the ML thinks look the same, even if the bits and bytes that make up the file are different. From what I can see, the similarity detection will be much more specific so as to not generate false positives. As you theorise though, it might be possible to modify images to evade detection if the hash's match specificity is high enough. All bearing in mind that the pictorial hash also is supposedly designed to be a one-way function to ensure that those who know file hashes don't know what the original contents of the file are.
• at_a_remove 5y ago
  I have so many questions about the implementation details. 1) Does this work only on iPhones or will it be iPads, as well? 2) Is this part of a system software update? I wonder if that will show up in the notes and how it would be spun. "In order to better protect our users ..." 3) If it is part of the system software update, will they be trying to make it run on older iDevices? 4) Is it just photos in your photo bin, iCloud, or does it start grabbing at network drives it attaches to? I could see the latter being prone to blowing up in their proverbial faces. The Four Horsemen of the Infocalypse ride again!
  • 1--6zVa-E 5y ago
    I’m certain if it gets added to iPhones, it’ll get added to iPads. iPadOS is iOS.
• bitL 5y ago
  OK, I guess the temperature is high enough after ~20 years, frogs are boiled, it's time to move in and consolidate power everywhere.
• fumar 5y ago
  Can activist shareholders mitigate changes like CSAM scanning?
• shuckles 5y ago
  The comments here suggest very few on HN have run a public service that allows arbitrary upload of photo and video content.
• bississippi 5y ago
  Been scanning the answers for a solution, I am 100% on Apple ecosystem and I am all for protecting little children but as others have pointed out nothing is stopping them from colluding with the government in future to say scan for political dissent: - weaponizing memes against politicians as “hate speech” - memes against government suppressing dissent - and we can look what happened to Uighurs Can Samsung phones that are de-googled work ? I am specifically interested in a new phone that Samsung launches, can it be degoogled ?
• protomyth 5y ago
  So, the Chinese government could force Apple to add the hash for the Tienanmen Square tank man picture and find it on all iPhones?
• cptskippy 5y ago
  CSAM also means Cyber Security Asset Management.
• Xamayon 5y ago
  Isn't there the potential for abuse of this to track things like who you talk to in private? Even if the images on your phone do not contain CSAM, the hashes of all your images would need to be shared with Apple, the NCMEC, and who knows what other quasi-gov't agencies. All it would take to build massive graphs of who talks to who, etc is to match up those hashes. It doesn't matter if they have no idea what image the hashes correspond to... If they then take the simple step of generating hashes for common images found online, they could even track what sites you browse and such. Ignoring the potential for false positives and other negative side effects of the main goal, this is a horrific breach of privacy. If you honestly think the gov't won't abuse any and all data they collect this way, I don't know what to say...
• abstractbarista 5y ago
  This is terrifying. I'd be pissed if I owned any Apple hardware. Encrypt your systems and run your own private cloud. Privacy is about the only thing keeping innocent people free in today's world.
• kyralis 5y ago
  Is there any actual evidence presented here? This is someone who's repeating "But it’s important to remember that this is the same company that won’t encrypt your iCloud backups because the FBI put pressure on them.", for which there is also no evidence and quite a lot of testimony to the contrary. His credibility seems questionable in the absence of evidence.
• specto 5y ago
  I think a lot of people are missing why apple is doing this now. They're doing this because they have a fairly secure eco system. They have also create a proxy that makes it difficult (impossible according to them) to know the client. More than likely this was implemented so when it does go to congress, they can say look we implemented a system. Otherwise the DOJ will continue to push for no encryption or backdoor encryption. There's no winning here.
• vincnetas 5y ago
  Another real case is how they will handle photos of my own naked kids, of which i have plenty of photos, because it's quite natural for my kids to be running around and playing naked. And i want to capture the moments, not the nudity. And also i have very close friends who are visiting us with they kids and for us it's ok to see each other children playing naked. We sometimes even share photos of nice moments with our kids, where kids sometimes happen to be naked. Is this already CP? Our kids are 3 and 5 years.
• maCDzP 5y ago
  I really hope Apple have addressed hash collision. Otherwise we are going to have a bad time.
• u4918 5y ago
  In Apple's public comments they repeatedly say things like "Apple can't access metadata or visual derivatives for matched CSAM images until a threshold of matches is exceeded for an iCloud Photos account." If they are only concerned with iCloud accounts then... why not scan in the cloud? Can anyone explain to me why client-side scanning is actually needed here? As far as I'm aware, Apple only E2E encrypts iMessages, not iCloud photos or backups. US politicians (to say nothing of countries with less individual freedoms) already openly pressure tech companies to censor specific content. And tech companies will do so even to the point of blocking direct, private, 1-1 messages sharing specific news. In that light, Apple's crossing a line to client-side scanning seems deeply concerning. I don't see how keeping this as narrowly-targeted as it's being advertised would ever be possible or even intended.