34 ms·

The deceptive PR behind Apple’s “expanded protections for children”

• ur-whale 5y ago
  > The Deceptive PR Tautology
• querez 5y ago
  I have a newborn at home, and like every other parent, we take thousands of pictures and videos of our newest family member. We took pictures of the very first baby-bath. So now I have pictures of a naked baby on my phone. Does that mean that pictures of my newborn baby will be uploaded to Apple for further analysis, potentially stored for indefinite time, shared with law enforcement?
  • deleted 5y ago
    [deleted]
  • baal80spam 5y ago
    No, it doesn't work like that.
    • zionic 5y ago
      Yes it does, it uses fuzzy perceptual hashes not crypto hashes. So if your innocent baby pic looks similar enough to a previously tagged child abuse image then YES, it will flag you and send a copy to the feds. And before you correct me, the Apple employee will see a picture of your naked baby and hit “forward to NCMEC”, which… upon investigation is actually just the feds
      • outworlder 5y ago
        IF there are multiple matches, IF it's going to icloud, THEN a 'derivative image' will be show for screening and IF deemed to be warranted, sent to NCMEC.
      • nicce 5y ago
        Perceptual hashes are extremly accurate. You might need a twin and top of that somehow identical environment almost in pixel level. Are news filled with false-positive accusations by PhotoDNA, flagging wrong images in Google, Facebook, Instagram etc.?
        • heavyset_go 5y ago
          > Perceptual hashes are extremly accurate. No, they are not. Ask anyone who has worked in this space[1][2], including myself. They are incredibly common. Two images that kind of look like one another will have similar or the same hashes. That is the point of perceptual hashing. [1] https://news.ycombinator.com/item?id=28091750 https://news.ycombinator.com/item?id=28091750 [2] https://news.ycombinator.com/item?id=28110159 https://news.ycombinator.com/item?id=28110159
  • slownews45 5y ago
    If you don't choose upload to icloud, no upload to apple at all. If you do choose icloud upload (most do), they were being uploaded already and stored and may be available to law enforcement. If you do upload to icloud, NOW they will be screened for matches with "known" images in a database, and if you have more than a threshold number of hits, you may be reported. This will happen on device. Apple will also scan photos in their cloud system as well from what I can tell (though once on device is working less should land in cloud). Note that it is HIGHLY likely that google photos / facebook / instagram and others will or are already doing similar scanning and reporting. I've heard millions of reports go in a year.
    • cwkoss 5y ago
      Arent the perceptual hashes based on a chunk of the image? I wonder what the false positive rates are for: - A random image against the DB of perceptual hashes - Images of a baby's skin against the DB of perceptual hashes It seems like the second would necessarily have a higher false positive rate: similar compositions (contains baby's skin) would more likely have similar chunks. Is it just a little higher or several orders of magnitude higher? I know hash collisions are rare, but wonder how much rarity of collisions decreases with perceptual hashes.
      • fossuser 5y ago
        It's two factors, both the match on an image hash and an unknown threshold of matches at which point the data gets sent up. If the threshold is not met then nothing gets notified (even if there is a match). Arguably this is why this approach is better for privacy. Cloud matches would not be able to have this extra threshold (in addition to this model allowing e2ee on the cloud in the future). I'd also like to know more about the specifics here, my guess is that threshold value is pretty high (their 'one in a trillion' comment not withstanding). It's probably targeting large CSAM dumps of matches which would not get flagged by different images.
      • slownews45 5y ago
        Absolute - I think this is one of two key questions for me. That is why I put "known" in quotes. It can't be an exact match because it has to handle cropping, rotation, resize etc. Images then do get a manual review before a report is made which is good and may help provide feedback on alogs being used. Going to be hard though for apple to set the second factor to high - I'd say 5 maybe? It's hard to say you had matches on potential CASM and ignored them I'd think.
  • fossuser 5y ago
    It's worth reading this, which is basically the only good reporting I've seen on this topic: https://daringfireball.net/2021/08/apple_child_safety_initiatives_slippery_slope https://daringfireball.net/2021/08/apple_child_safety_initia... There are legitimate things to be concerned about, but 99% of internet discussion on this topic is junk.
    • Spooky23 5y ago
      The EFF wrote a really shitty hit piece deliberately confused the parental management function with the matching against hashes of illegal images. Two different things. From there, a bazillion hot takes followed.
      • dathinab 5y ago
        Two different things which are sold as one package by Apple. Two different things which both are known to be prone to all kind of miss-detection.
      • rootusrootus 5y ago
        Yeah I found the EFF's piece to be really disappointing, coming from an organization I'm otherwise aligned with nearly 100% of the time.
        • shapefrog 5y ago
          EFF today is really not the organisation it was just a few years ago. I dont know who they hired badly, but the reasoned takedowns have been replaced with hysterical screaming.
          • matwood 5y ago
            > hysterical screaming Given the political/societal climate, it probably gets them more donations.
          • xdennis 5y ago
            > hysterical screaming Ah, I see you work for Apple, given the language ("the screeching voices of the minority").
  • samename 5y ago
    Unless those pictures are also in the NCMEC database, there won’t be a match.* * As addressed in the comments below, this isn’t entirely true: the hash looks for visually similar picture and there may be false positives.
    • gambiting 5y ago
      Absolutely not true. Apple is using a similarity based hash, so if the NCMEC database contains a picture that's similar to one that you have, it could produce a match even if it's not the same. Apple says this isn't an issue, because a person will look at your picture(yes, a random person somewhere will look at the pictures of your newborn) and judge whether they are pictures of child abuse or not. If this unknown person thinks your picture shows child abuse, you will be reported to NCMEC and then what happens is unknown - but likely that it would result in some legal action against you.
      • lawkwok 5y ago
        Keep in mind, this manual review only happens after Apple’s system detects multiple occurrences of matches. Until that point, no human is alerted of matches nor does anyone see how many matches there have been. In a TechCrunch interview Apple said that they are going after larger targets that are worth NCMEC’s time.
        • zionic 5y ago
          Parents take a lot of photos of their kid. Like, lots.
          • simondotau 5y ago
            How many of them include erect adult penises and active participation in sex acts? Apple's on-device list of hashes only includes images which have been classified "A1" under the CSAM categorisation scale. If any other photographs are accidental hash collisions to these images, it's going to be pretty damn obvious to the human reviewer.
  • deleted 5y ago
    [deleted]
  • stevenicr 5y ago
    from my current understanding - that does occur with m-soft one-drive which is a default in many systems), but not the hash-looking thing apple is currently proposing.
  • dathinab 5y ago
    Unlikely except if you send them to a iphone which is registered with a "child" account. Apple uses two different approaches: 1. Some way to try to detect _known_ child pornographic material, but it's fuzzy and there is no guarantee that it doesn't make mistakes like detecting a flower pot as child porn. But the chance that your photos get "miss detected" as _known_ child pornographic material shouldn't be too high. BUT given how many parents have IPhones it's basically guaranteed to happen from time to time! 2. Some KI child porn detection on child accounts, which is not unlikely to labile such innocent photos as child porn.
    • fossuser 5y ago
      Even in the child account case it's not sent to Apple - it alerts parent accounts in the family. It's also just nudity generally, more akin to garden variety parental control content filtering. The child account iMessage thing is really entirely separate from the CSAM related iCloud announcement. It's unfortunate people keep confusing them.
  • s5300 5y ago
    So... this is the way I understand it, which the general public will never have the attention span to understand, so it doesn't fucking matter one bit. LEO's/FBI/every other institution/group that deals with child pornography and abuse have teams that go through a near infinite amount of pictures and videos of CP/etc. These are then marked by said people as either - yes, CP/Abuse/etc - or marked false positive. Once marked as what they're after, they're uploaded to a shared database between all groups involved. Only what is in these worldwide national databases is what's going to be checked against. Your new pictures of your children will have obviously never made their way to any of these groups as they've never been shared/distributed in any areas of the internet/etc these people work in to track down trafficking rings (well, I'd hope you're not selling pictures of your children to them). This is the way I understand it. I admit I haven't looked into it that much. If it's anything different than what I've said, then yeah, it's probably fucked. I don't get what people don't understand about checking against a database though. No, your new pictures of whatever are not in this pre-existing database
  • bouncycastle 5y ago
    My understanding is that you should not upload these photos to the cloud anyway. The cloud is not your computer and who knows, maybe apple engineers might be snooping on them, or there could be a hack and so on..Putting on the cloud is like sharing with Apple.
  • dev_tty01 5y ago
    No. The CSAM (Child Sexual Abuse Material) scanning is comparing hashes of photos about to be uploaded to iCloud against a specific set of images at NCMEC (National Center for Missing and Exploited Children) which are specific to missing and exploited children. It is not machine learning models looking for nudes or similar. It is not a generalized screening. If enough matched images are found, the images are flagged for manual verification. If the manual verification confirms that the images match specific images in the NCMEC database, law enforcement is informed. Be aware that almost all cloud providers screen photos. Facebook reported 20 million images in 2020, Google reported half a million. Dropbox, Box, and many, many others report images. See https://www.missingkids.org/content/dam/missingkids/gethelp/2020-reports-by-esp.pdf https://www.missingkids.org/content/dam/missingkids/gethelp/... to see a complete list of companies that screen and report images. The other thing Apple announced which is completely separate from the CSAM photo scanning is additional parental controls for the Messages app. If a parent opts in for their under-13 children, a machine learning model will look for inappropriate material and warn the child prior to showing the image. The child is also told that their parent will be flagged if the child looks at it anyway. For 13-18 year olds whose parents opted in, the teen is warned first about the content. If the teen continues past the warning the image is shown and no further action is taken. Parents are not flagged for children 13 and over. As I said, this is a parental control for pre-adult kids. It requires opt-in from the parents and has no law enforcement implications.
    • ok123456 5y ago
      The correct answer is a well qualified "Maybe." The hashes are fuzzy AI generated weights. It's impossible to know what will cause a false-positive.
    • aczerepinski 5y ago
      Comparing hashes reminds me of this announcement from a few years ago that Google had produced a SHA1 collision: https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html https://security.googleblog.com/2017/02/announcing-first-sha... Can you imagine the chaos of a successful collision matching some explicit material being sent as a prank or targeted attack?
  • sandworm101 5y ago
    Yes, if they wind up part of a child porn investigation. Your cloud account gets hacked. Some perv gets your images. He is then arrested and his "collection" added to the hash database... including your family photos. Context often matters more than the nature of the actual content. Police aquire thousands of images with little hope of ever knowing where they originated. If they are collected by pervs, and could be construed as illegal in the hands of pervs, the images become child porn and can be added to the databases.
    • snowwrestler 5y ago
      It’s worth pointing out that this could happen with any Internet-attached photo storage, and pre-dates Apple’s announcement. What Apple announced is a new system for reading the existing hash lists of known CSAM images and doing the comparison on the device as part of the iCloud upload, rather than on the server after upload.
    • nicce 5y ago
      Actually, we don’t know yet whether you can access your photos from the web anymore after this update, because E2EE ”like” implementation. Protocol is rather device specific (while allowing multi-device), so it might not be enough to access or hack iCloud account to access the photos. So, things get complicated.
      • vineyardmike 5y ago
        > because E2EE ”like” implementation. Did apple actually say photos would be e2ee or are we just assuming?
        • nicce 5y ago
          Did you read the spec? This is why the scanning happens on the device.
          • vineyardmike 5y ago
            I don't recall them explicitly saying anything was e2ee except imeessage which is not relevant for this discussion.
  • fortenforge 5y ago
    Lots of people responding to this seem to not understand how perceptual hashing / PhotoDNA works. It's true that they're not cryptographic hashes, but the false positive rate is vanishingly small. Apple claims it's 1 in a trillion [1], but suppose that you don't believe them. Google and Facebook and Microsoft are all using PhotoDNA (or equivalent perceptual hashing schemes) right now. Have you heard of some massive issue with false positives? The fact of the matter is that unless you possess a photo that exists in the NCMEC database, your photos simply will not be flagged to Apple. Photos of your own kids won't trigger it, nude photos of adults won't trigger it; only photos of already known CSAM content will trigger (and that too, Apple requires a specific threshold of matches before a report is triggered). [1] "The threshold is selected to provide an extremely low (1 in 1 trillion) probability of incorrectly flagging a given account." Page 4 of https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
    • deleted 5y ago
      [deleted]
    • deleted 5y ago
      [deleted]
    • shawnz 5y ago
      The 1 trillion figure is only after factoring in that you would need multiple false positives to trigger the feature. It's not descriptive of the actual false positive rate of the hashing itself.
    • FabHK 5y ago
      Probability of a false positive for a given image = p Probability of N false positives (assuming independence) = p^N Threshold N is chosen by Apple such that p^N < 10^-12, or N log p < -12 log 10, or N > -12 log(10)/log(p) [since log(p) < 0, since p < 1]. ETA: Suppose, just for the sake of the argument, that p = 10^-3 (one false positive in 1000, so really quite bad). Then log(p) = -3 log(10), so N > -12 log(10)/(-3 log(10)) = 12/3 = 4. Similarly, if p is one in a million (10^6), then N would be required to be > 12/6 = 2. In practice, I'd expect N to be larger than 4, in other words, Apple being very conservative here. ETA: The above doesn't take into account how many images M you have. The analysis gets more complicated, but N needs to be way larger than 4. I'll think about it some more.
  • 908B64B197 5y ago
    Wait until someone manages to create an image (white noise) that's a hash collision for anything in that database. And then starts spamming random strangers via airdrop. Enjoy explaining why your mugshot and arrest record had these charges attached to it! (Actually, in this case the prosecution would probably use the other pictures on the phone that were not detected by the scanning tool as a way to get a guilty plea deal!)
    • FabHK 5y ago
      It would have to be a number of pictures that are flagged, and after that threshold is exceeded, they (more precisely, their "visual derivative") are reviewed by a human. So, no mugshot and no arrest record, even if you choose to accept any number of pictures sent from random strangers via airdrop.
    • GuB-42 5y ago
      Assuming it is possible (I think it is), there is a manual verification process if you have a match. And obviously, the white noise will be rejected, like all pictures that do not look remotely like the original. But it can be a form of denial of service: saturate the system with hash collisions so that people can't keep up.
  • migueldeicaza 5y ago
    No, this will never be caught. This only catches ownership of illegal photos.
  • darkhorn 5y ago
    Just don't critisize your government in any way. Otherwise they will find anything illegal to arrest you like from crossing street in red to I don't know what. You will be fine becouse there is a legal system and no one can put you into jail for crimes you have not commited. Just look at Julian Assange or random Joe in Belarus who was arrested for wearing red white hat. The justice system always is in the innocent people's side, without exception.
  • devwastaken 5y ago
    Eventually, yes. As we work more towards AI's and "smart" image recognition eventually there will be a system that will have these false positives on innocent images. Current CSAM depends on humans to "verify" the imagery, this is something companies desperately want to get rid of, and so do the employees understandbly so. Nobody wants a job (99.99%) of comparing CSAM. It costs companies money in labor costs, and draws them bad PR when those employees inevitably develop permanent/semi-permanent mental health issues from it. The only reason it hasn't happened yet is because a startup can't just start scanning CSAM. They need the blessing of the feds to do that, which requires political connections, and of course requires competing with companies that already have that blessing - something that politics prevents. PhotoDNA and current CSAM scanning only gets known CSAM, but not new CSAM. The end goal is to detect CSAM before it's ever even distributed, to be "closer to the victim", rather than just those consuming it. Even with current PhotoDNA you can generate hash collisions, which flag the image for review, and a real human compares the material. This is of course subject to change for the reasons stated above. Secondarily, automatic scanning and ID'ing of imagery is how you can easily throw an FBI raid at someone. Apps like Telegram automatically download every image/video in the thread. Ontop of that you can create images that appear different at differing resolutions. At one resolution a harmless meme, at another, CSAM. Meaning that you can again throw an FBI raid at someone using simple tricks.
  • bryanrasmussen 5y ago
    no, because it only catches registered CSAM, however if you sent your pictures to relatives etc. and somehow someone who was into CSAM got one of your pictures and later gets arrested, your photo in their collection could theoretically be registered in the official archives - then you might have something that matches one of the hashes of a known CSAM image in your collection of images (maybe enough matches to have the police come talk to you) on edit: later on of course this will make a great article in some place like the Atlantic with a stolid monochromatic picture of your family in the lead-in and we will all read about it on HN and talk about how this was an obvious problem with the whole system (if it gets posted at the right time and gets enough upvotes).
• seph-reed 5y ago
  I really don't see why the scanning would ever be done on the phone instead of on iCloud if it only affects iCloud images. But I do have guesses why.
  • czzr 5y ago
    Only semi-good reason is it would enable E2E encryption in the cloud while still allowing detection of CSAM.
    • roody15 5y ago
      Except despite this being repeated over and over… Apple has not said anything about E2E
      • czzr 5y ago
        As I said, the design enables this, if Apple chose to do it. It remains to be seen if they will.
        • zionic 5y ago
          The design more plausible enables total device surveillance than questionable iCloud Backups. (I refuse to call a backdoored setup E2EE)
          • zepto 5y ago
            That’s silly. The design is so narrowly tailored to scan for CSAM that nobody can use it for anything else.
            • FabHK 5y ago
              It all depends on what perceptual hashes you use. If Apple can institute a process whereby those are tied to the OS version, but not to the region, then it would be impossible to impose jurisdiction-specific exceptions.
• spoonjim 5y ago
  Any idea why Apple didn’t just implement server side scanning like everyone else?
  • barbazoo 5y ago
    Potentialy to be able to introduce e2e encryption later on.
  • mortenjorck 5y ago
    As covered in other articles, that is exactly what they were doing previously.
    • rootusrootus 5y ago
      I'm not so sure. John Gruber's write-up said that Apple has only sent over a couple hundred reports in the last year to the gov't, compared to over 20 million from Facebook. This suggests to me that Apple's scanning wasn't nearly so widespread.
      • zionic 5y ago
        Because no one in their right mind uploads CP to a cloud service, and apparently pedos abuse Facebook’s easy sign up process to bulk upload CP. Not that it matters when those Facebook sign ups are probably proxied with throwaway emails
    • tpush 5y ago
      No, they did not. That was erroneous reporting by the Telegraph that a lot of outlets copied [0]. The correction: > This story originally said Apple screens photos when they are uploaded to iCloud, Apple’s cloud storage service. Ms Horvath and Apple’s disclaimer did not mention iCloud, and the company has not specified how it screens material, saying this information could help criminals. And from the interview with TechCrunch: > This is an area we’ve been looking at for some time, including current state of the art techniques which mostly involves scanning through entire contents of users’ libraries on cloud services that — as you point out — isn’t something that we’ve ever done; to look through users’ iCloud Photos. [0] https://www.telegraph.co.uk/technology/2020/01/08/apple-scans-icloud-photos-check-child-abuse/ https://www.telegraph.co.uk/technology/2020/01/08/apple-scan...
• deleted 5y ago
  [deleted]
• rootsudo 5y ago
  https://www.missingkids.org/theissues/end-to-end-encryption https://www.missingkids.org/theissues/end-to-end-encryption Geez.
  • still_grokking 5y ago
    The main statement on that site does once more not explain why real criminals just wouldn't use not backdoored software. Indeed, it just looks like another move in the current crypto-wars.
  • deleted 5y ago
    [deleted]
  • commoner 5y ago
    > Do not implement end-to-end encrypted communications for accounts where a user has indicated they are under 18 years old. It's repulsive how the NCMEC is pushing to deprive minors of privacy and agency, while simultaneously claiming to advocate for their benefit.
• Accacin 5y ago
  Eh, I completely agree that this is a step too far, but the solution is so simple. Stop using Apple devices - luckily I switched from iOS to CalyxOS when my iPhone 7 broke earlier this year. Honestly, it wasn't so bad.
  • psychomugs 5y ago
    This is throwing the baby (pictures) out with the bathwater. I am for better or worse deeply rooted in the Apple tree (phone, laptop, tablet, and, recently, watch); for all its occasionally infuriating and arguably stupidly designed warts, the fact that so many features disappear and Just Work is something you can nary say for other ecosystems.
    • hypothesis 5y ago
      That’s the thing, for years I had to tolerate those silly issues from people who are supposed to be best in the industry. There is still no default calculator installed on iPad in 2021! For some people, it’s simply no worth it anymore, after primary commitment is gone..
• EugeneOZ 5y ago
  in "Photos" app, in the bottom right corner there is a "search" icon. When I click it, and entering "beach", I can see photos I've made on the beach (or in the sea, near the beach). What does it mean? My (and your) photos are scanned and analyzed. I've heard literally zero noise about this feature - nobody was complaining (at least not loud enough to let me notice it). So, why the hell all of that fuzz is being raised now? You're (and mine) photos will be scanned and analyzed AGAIN. Not by humans, by algorithms. In some really rare cases they might be checked by humans, but you 100% will not have troubles with the law if photos don't contain CSAM. I have 2 kids and I’m not buying that argument “oh my library of naked photos of my child - I’m in danger”. If you are uploading naked photos of your child to iCloud - it's similar to publishing them. Everything that is uploaded to the Internet, will belong to the Internet, and you don't have so much control of it. If, for some awkward reason, you have sets of naked photos of your child and you want to save them - never ever send them to the Internet. If you think that not-so-experienced users should not know about this rule - I’m pretty sure they don't even know (or care) about this “scandal”. All of that FUD wave is raised by the journalists and echoes on forums like this one.
  • shapefrog 5y ago
    It turns out people liked it when their phone scanned their photos for 'selfie' or 'beach' for them. Apparently tagging 'child porn' on your photos for searching isnt the killer feature someone thought it might be.
    • EugeneOZ 5y ago
      Yeah :) Also, it's funny that Apple here goes for bigger risks: reputation, trust, all of that noise, then risks of false accusations. And for what? To help with stopping the pedophile networks. “But no, wait, they want to use algorithms to scan my photos, it’s a privacy violation...” Just wake up.
      • shapefrog 5y ago
        They can have a full resolution copy of my photo, all 12 million pixels, along with the exact time, location and direction I was facing when I took it... but I draw the line firmly at a hash of it being taken.
• rvz 5y ago
  Yep, that's deceptive advertising on privacy and everyone bought into it and walked straight into the reality distortion field. Another innovative 'gotcha' by Apple. A reminder that they are not your friends.
• anko 5y ago
  From the article; > You could of course say that it’s “a slippery slope” sort of argument, and that we should trust Apple that it won’t use the functionality for anything else. Setting aside the absurdity of trusting a giant, for-profit corporation over a democratically-elected government, And then later it reads > and has previously cancelled their plans for iCloud backups encryption under the pressure of FBI. Isn't the FBI in place because of the democratically elected government? It seems like the for profit organisation is trying to do the right thing, and the government is stopping them. This is the fundamental problem with arguments based on "trust" - the government seems to be doing the wrong thing.
• wpdev_63 5y ago
  I used to always get the latest and greatest iphone but with the politics and everything that's going on why would I want to spend more than the absolute minimum on my cellphone? There are plenty of wholesome things to spend money on other than tech.
• atbpaca 5y ago
  #NotUpdating to iOS15, also #NotUpgrading this time until further notice.
• phkahler 5y ago
  I really don't get all the hype. This is not a backdoor as it's called in TFA. It's not Apple "reaching into your device". It is literally checking for specific images and reporting their presence to Apply if found. It's not using AI to analyze your photos or anything like that. It's looking for specific images, and only prior to uploading them to iCloud. It won't even flag your own nasty images because the hash won't match. Note: The above assume we're talking about a typical hash of data and not an image-analysis "hash" of what it thinks the content it. This is supported by the language they use. Yes, it's a bit big-brother. But I already assume the authorities can fairly easily get ALL your iCloud data if they ask Apple the right way. You know what's creepy AF? Having a private conversation and getting facebook ads the next day relating to the topic. Talk about an acquaintance acting schizophrenic and get ads about medications and treatment for that? Creepy as fuck. And that was on the wifes iPhone - I have Android and didn't get that stuff, but I seem to remember similar incidents where I got ads for stuff talked about. That's serious voice analysis, not just checking a file hash, and it happens when your phone is in your pocket.
  • daveidol 5y ago
    Wait are you saying you get ads based on things you converse about out loud in the real world because your phone is listening to everything in your pocket? You know that is a myth and isn't true, right?
    • shapefrog 5y ago
      You saying one conspiracy is 100% true but another one is laugh out loud impossible?
      • daveidol 5y ago
        Wait, what conspiracy am I saying is 100% true? If your phone was listening to everything you do all the time then you'd need a lot of processing to analyze it on-device and exfiltrate the critical pieces of info (which would be pretty obvious when looking at CPU usage and battery life), or you'd be sending an insane amount of data off over the internet all the time. I'm not saying it's technically impossible - I'm just saying that the ol "Facebook is listening to everything and that's why you get ads for something after you talk about it in real life" is almost certainly a myth with zero actual data to support it other than some anecdotes online.
• severak_cz 5y ago
  > The hypothesis that I have is that Apple wishes to distance itself from checking users’ data. This is best explanation of the whole situation I have read.
  • hu3 5y ago
    And in the process hand over more user data to tyrants. Surely they know this will be abused to check user data before it is uploaded to iCloud. All it takes is a willing government.
    • crummy 5y ago
      How is that different from how things work now?
      • hu3 5y ago
        Before, Apple would only scan data already in the cloud. Now the pandora box has been opened. They are adding capability to scan files on iPhones before it hits the cloud. Any technical or financial excuse they might have used in the past to not scan files locally is now rendered null. Governments can just say: "you know what? scan these arbitrary sets of hashes as well, they are illegal in my jurisdiction and since you've shown that you can, scan them regardless if the user is sending to iCloud or not."
        • nicce 5y ago
          > Any technical or financial excuse they might have used in the past to not scan files locally is now rendered null. This just proves that people don’t understand much about technology in depth. Capability has been there already, for very long time. 99% of their work has gone for implementing that perceptual hashing function and their PSI system. If you want to give excuses, there will be always more. But they are not reasons to prevent this pandora box.
          • hu3 5y ago
• Componica 5y ago
  Imagine taking a photo or have in your gallery a photo a dear leader doesn't want to spread. Ten minutes later you heard a knocking at your door. That's what I'm most worried about, how is this not creating the infrastructure to ensnare political dissidents.
  • psyc 5y ago
    I am profoundly disappointed that almost all of the discussion is about the minutiae of the implementation, and "Hmm.. Am I ok with the minutiae of Apple's specific implementation at rollout?" And almost nobody is discussing the basic general principle of whether they want their own device to scan itself for contraband, on society's behalf.
    • hypothesis 5y ago
      Maybe people realize that’s not a winning strategy and thus keep going back to technical details…
    • InternetPerson 5y ago
      But Apple says, "You need several hash matches to trigger a review." See, that makes it OK!
• balozi 5y ago
  Dear tech users, Associating with some of you has become a liability. One may be smart enough to avoid iPhone and Alexa et al. but what to do when one is surrounded by people who willingly expose themselves to nefarious technology? In short, I don't want pictures of me being hoovered up along with your baby pics from your iPhone.
• deeblering4 5y ago
  What would prevent someone from, for instance, printing off an illegal photo, “borrowing” a disliked co-workers iCloud enabled phone, and snapping a picture of the illegal picture with their camera? On iOS the camera can be accessed before unlocking the phone, and wouldn’t this effectively put illegal image(s) in the targets possession without their knowledge?
  • fortenforge 5y ago
    These illegal photos are not trivial to obtain. Possessing (and here, the printing step necessitates possession) these illegal photos is in and of itself a crime in most relevant jurisdictions. But OK, let's say that you've found a way to get the photos and you're comfortable with the criminal implications of that. At that point why don't you just hide the printed photos in your coworker's desk? My point is that if you have a disgruntled coworker who's willing to resort to heinous crimes in order to screw you over, there's many different things they could do that are less convoluted.
    • tick_tock_tick 5y ago
      It takes 1 minutes on TOR to find enough to get anyone thrown in jail don't make it sound harder than it really is. As for photos vs printing taking a photo reports it for you so you're never involved.
    • makeitdouble 5y ago
      An “interesting” part of this is, up until now these photos had little technical exposure. But now that millions of phones can be affected, creating unrelated pictures that purposefully match these hashes becomes a shinny target.
      • fortenforge 5y ago
        This is untrue. The same photos being used by Apple for this scheme are already in use for CSAM scanning by many major platforms (Google, Facebook, Microsoft). If someone wanted to generate a false positive image, they could already leverage it on these other platforms.
• cebert 5y ago
  As much as the tech and security community has concerns and objections to this policy change on the part of Apple, I’m skeptical there will be any notable impact to Apple’s revenue and future sales.
  • hypothesis 5y ago
    I remember people were saying same thing about Linux on Desktop, yet we have viable alternatives to proprietary OSes. Yes, someone will have to struggle to get us there, but will have alternative if we don’t give up.
• johnvaluk 5y ago
  My common sense is tingling, telling me that Apple's eventual move will be one of malicious compliance, finally implementing e2ee in a way that provides them with culpable deniability and users with a much desired privacy enhancement.
• jliptzin 5y ago
  There is something you can do about it: don’t use Apple products
  • blairbeckwith 5y ago
    That strategy will last ~15 minutes until Google is doing the same thing. Then what? I would argue that what Google is doing already is way more privacy-compromising than this.
    • hirundo 5y ago
      That's a great argument for a Linux phone or de-googled Android build.
    • megous 5y ago
      Then don't use Google products either (or don't use for photography). Seems obvious. "Dumb" phones and "dumb" cameras still exist.
      • blairbeckwith 5y ago
        That is not practical for many people who wish to be a part of society anymore. See: rollout of virtual vaccine passports, etc.
        • postalrat 5y ago
          Virtual vaccine passports will be forgotten long before they affect even a small fraction of people on this planet.
        • megous 5y ago
          You can get a physical one by printing out a QR code, afaik.
    • jjcon 5y ago
      Google couldn’t do it, not effectively anyway because android vendors and variants are decentralised. They could do it for the pixels but that represents less thats half a percent of the market - not to mention that everyone on a pixel could just move to lineageOS if they didn’t like it. That freedom and decentralisation doesn’t exist on Apple (at least yet, maybe the DOJ or congress will regulate them).
• zug_zug 5y ago
  Good thing this didn't exist in 1776, or I'd be living in Great Britain.
• shmerl 5y ago
  This ad seems fitting in the context: https://www.youtube.com/watch?v=tdVzboF2E2Q https://www.youtube.com/watch?v=tdVzboF2E2Q
• tuatoru 5y ago
  Unless Apple can demonstrate that the techniques they are using are intrinsically specific to CSAM and to CSAM only--the techniques do not work for any other kinds of photo or text--slippery slope arguments are perfectly valid and cannot be denied. Apple is a private company and as such its actions amount to vigilantism.
• FabHK 5y ago
  Question: Would Apple report CSAM matches worldwide to one specific US NGO? That's a bit weird, but ok. Presumably they know which national government agencies to contact. Opinion: If Apple can make it so that a) the list of CSAM hashes is globally the same, independent of the region (ideally verifiably so!), and b) all the reports go only to that specific US NGO (which presumably doesn't care about pictures of Winnie the Pooh or adult gay sex or dissident pamphlets) then a lot of potential for political abuse vanishes.
  • trynumber9 5y ago
    Apple said they're only enabling it in the US for now.
  • tylerhou 5y ago
    NCMEC is not exactly a normal NGO; it was opened with Reagan present and Congress frequently gives it funding. It also works with other government organizations on a frequent basis.
    • jhayward 5y ago
      NCMEC isn't really an NGO; it is an agent of the US government, written in to US law.
      • tylerhou 5y ago
        Except it's not a government agency; it's a private non-profit with strong governmental ties. https://www.missingkids.org/blog/2020/four-ncmec-myths https://www.missingkids.org/blog/2020/four-ncmec-myths
• 1vuio0pswjnm7 5y ago
  "The hypothesis that I have is that Apple wishes to distance itself from checking users' data. They've been fighting with the FBI and the federal government for years, they've been struggling with not reporting CSAM content to the NCMEC, they don't want to be involved in any of this anymore." However there is close to zero evidence to support this idea. I was just reading something the other day that directly contradicted this; it suggested the relationship has been excellent save for a single, well-publicised dispute over unlocking an iPhone. In other words, the publicly aired dispute was an anomaly, not representative of the underlying relationship. Even more, unless the pontificator works for Apple or the government, she is not a good position to summarise the relationship. Plainly put, it is not public information. What does such baseless speculation achieve. Is it like spreading a meme. I dont get it. "The worst part is: how do I put my money where my mouth is? Am I going back to using Linux on the desktop (2022 will be the year of Linux on the desktop, remember), debugging wifi drivers and tirelessly trying to make resume-from-suspend work? Am I getting a Pixel and putting GrapheneOS on it like a total nerd? FUCK." Is having a computer with closed source wifi drivers and proper ACPI support more important than having a computer with an open OS that does not include an intentional backdoor. Maybe the problem is not how to put your money where your mouth is, its how to put your mouth where your money is. What does GrapheneOS cost. Maybe this is not about money. Options like GrapheneOS, even the mere idea of GrapheneOS, i.e., that there can be alternatives to BigTech's offerings, get buried underneath Apple marketing. Much of that marketing Apple gets for free. It comes from people who do not work for Apple. Bloggers and others who discuss computers can help change that. They can also help Apple sail through any criticism (and they do).
  • nicce 5y ago
    > However there is close to zero evidence to support this idea But there is. For example Apple used a lot of effort in this area, when they built their hardware security module (HSM), which is basically on every iPhone and iPad. This module is built in such a way, that nobody, not even Apple can access security keys from this device, or reprogram it again. Locked iPhone or password vault stays locked or gets cleaned. One blog about this matter: https://blog.cryptographyengineering.com/2016/08/13/is-apples-cloud-key-vault-crypto/ https://blog.cryptographyengineering.com/2016/08/13/is-apple... How about user tracking? Apple is one of the few companies which is not caught yet by selling data to third parties, nor even collecting more than needed to develop their products. Our new fancy iCloud is maybe the biggest evidence? It is maybe the cleverest way to this date to enable somekind of E2EE while getting limited info about the content. Highly recommed reading that PSI paper.
• Barrin92 5y ago
  >The worst part is: how do I put my money where my mouth is? Am I going back to using Linux on the desktop (2022 will be the year of Linux on the desktop, remember) people really need to retire this meme. On the desktop in particular as a dev environment Linux is completely fine at this point. I can understand people not wanting to run a custom phone OS because that really is a ton of work but for working software developers Fedora, Ubuntu whatever any mainstream distro is at this point largely hassle free.
  • ajsnigrutin 5y ago
    I hate ubuntu from the bottom of my heart, for breaking stuff and changing stuff that used to "just work" all the time, but 99.999% of the time, that means "background stuff", "normal users" never mess around with, and for normal users, a "usb key -> install -> next, next, next -> finish -> reboot" just works.
    • nicce 5y ago
      I used to use Ubuntu for many years, but it became a such bloatware. So many things what you don’t really need. Packages were sometimes also different compared to vanilla Debian. This caused issues in stability (talking more about feature set). Some advanced software just did not work, which worked on equivalent vanilla Debian. I might recommend Ubuntu for very beginner developer, but not to stick with it longer time. It will give you headache. There are also more privacy-friendly distributions.
      • treebot 5y ago
        Are Mac and Windows not also full of an enormous amount of crap that we don't need?
        • nicce 5y ago
          That does sound quite bad, doesn’t it? That some Linux distributions are getting closer to them? We have a freedom, let’s use it. I’m using it for something minimal, like Arch Linux.
• atbpaca 5y ago
  I doubt Apple has not thought about the PR & policy consequences of such an iPhone backdoor. For me, it's even more sad to see Apple using the fight against CSAM, a noble cause, as a shield and a way to convince the masses that breaking its promise to protect privacy is OK. "What happens in your iPhone stays on your iPhone [no longer]". There is no court oversight, no laws, it's automated mass surveillance.
• farmerstan 5y ago
  Whoever controls the hash list controls your phone from now on. Period. End of sentence. Apple has not disclosed who gets to add new hashes to the list of CSAM hashes or what the process is to add new hashes. Do different countries have different hash lists? Because if the FBI or CIA or CCCP or KSA wants to arrest you, all they need to do is inject the hash of one of your photos into the “list” and you will be flagged. Based on the nature of the hash, they can’t even tell you which photo is the one that triggered the hash. Instead, they get to arrest you, make an entire copy of your phone, etc. It’s insidious. And it’s stupid. Why Apple is agreeing to do this is disgusting. And it doesn’t make sense. If I were a pedophile and I took a new CSAM photo, how long would it take for that specific photo to get on the list? Months? Years? As long as pedophiles know that their phones are being scanned, they won’t use iPhones for their photos. And then it will be only innocent people like me that get scanned for CSAM and potentially getting that used against me in the future. If they really cared about CSAM, this feature is useless and stupid. All it does is make regular people vulnerable to Big Brother tactics which we know already exist.
  • fortenforge 5y ago
    There are numerous incorrect statements in your comment. First: Apple has disclosed who gets to curate the hash list. The answer is NCMEC and other child safety organizations. https://twitter.com/AlexMartin/status/1424703642913935374/photo/1 https://twitter.com/AlexMartin/status/1424703642913935374/ph... Apple states point-blank that they will refuse any demands to add non-CSAM content to the lists. Second: Why can't the FBI / CCCP inject a hash into the list. Here's a tweet thread gamifying that scenario: https://twitter.com/pwnallthethings/status/1424873629003702273?s=20 https://twitter.com/pwnallthethings/status/14248736290037022... The short answer is that at some point an Apple employee must visually review the flagged photo, and confirm that it does represent CSAM content. If it does not, then Apple is under no legal obligation to report it. Third: You claim that abusers will simply opt not to use iPhones to distribute their CSAM content rendering the feature useless. This is in fact not how things have played out on other platforms like Google and Facebook that do already scan for CSAM. These organizations report on the order of millions of flagged images per year. [1] Clearly the abusers have simply not moved on to a different platform. [1] https://www.businessinsider.com/facebook-instagram-report-20-million-child-sexual-abuse-images-2021-2 https://www.businessinsider.com/facebook-instagram-report-20...
• atbpaca 5y ago
  when are they going to add this backdoor to MacOS?
• xg15 5y ago
  > In the world of computer security this technology has a name, it’s called “a backdoor.” A well-documented and well-intended backdoor, but still a backdoor. Installed and enabled by default on millions of devices around the world. Sorry, but that backdoor has already existed for a long time. It exists in every IoT gadget, smart car, smart speaker, smart home and other connected device that phones home to its vendor and can receive arbitrary firmware updates. It exists for every app and every desktop software that will automatically update itself in the name of "evergreen software". This is just the first time someone is publicly making use of the backdoor.
• tehjoker 5y ago
  I'd like to point out that the government (and by proxy Apple, companies care even less) doesn't give a shit about children. They are advocating a policy of mass infection, they didn't give a crap about children in Flint drinking toxic water, etc. If they cared about kids, they would care a lot about thinks that physically hurt and kill them. This means we don't have to take their stated reasons for this at all seriously. Apple, if you care about children, you'll pay more than your legally owed taxes and push for improved access to education, nutrition, and free child care. They're only interested in the avenue that coincidentally dramatically increases their surveillance powers and the powers of the government. Weird, can't figure that one out.
  • mackrevinack 5y ago
    apple also has a history of using child labour to build their products, even cases within the last decade
• squarefoot 5y ago
  This whole mess brought back a memory of when I was 4 to 5 years old (so probably 1971). During a summer vacation we were walking at a harbor in Tuscany with my parents and they told me suddenly I had to take a dump. Problem was that there was no bathroom nearby, well it probably was since the place was filled with restaurants, but we were like a hundred meters from the nearest one, which was incompatible with the sudden need of a baby like I was. So my parents quickly found an area with vegetation behind a building, helped me remove my clothes and sit down waiting for me to unload all that stuff. Then my father saw me doing an expression they later described as priceless, so he quickly shouted me to wait, then grabbed his Nikon and shot me a photo. That photo later that year won a prize. Now imagine the same happening today with my dad shooting me a photo using his iPhone, only to trigger a CSAM alert somewhere an probably be investigated for child abuse. Just no thanks. Screw you Apple, and all those who pull your strings into creating this farce.
  • tehnub 5y ago
    There will only be an alert if that photo is extremely similar to an image in the NCMEC database, AND there are numerous other such photos on the account that match. The threshold number of matches to trigger an alert is tuned for a 1/trillion chance of false positive. Furthermore, if you were using say Google Photos to store your images, then you were already subject to this vulnerability.
    • deleted 5y ago
      [deleted]
    • squarefoot 5y ago
      So what if a small circle of people produce their CSAM material by themselves and share only among themselves? None of the pictures is being uploaded to that database, so either the algorithms are really really good at recognizing them, or it will require human intervention, that is, scanning one by one all phones, then deciding which picture matches the criteria and write down the names of the people involved. I can't think of a similar scenario that doesn't imply the total loss of privacy by anyone even remotely linked to one of these people.
• dev_tty01 5y ago
  There is a great deal of misinformation and confusion on this topic. Here is a good interview with Apple's head of Privacy. https://techcrunch.com/2021/08/10/interview-apples-head-of-privacy-details-child-abuse-detection-and-messages-safety-features/ https://techcrunch.com/2021/08/10/interview-apples-head-of-p...
  • resoluteteeth 5y ago
    What part exactly do you think people are confused about?
  • system2 5y ago
    The only information I care about is Apple is putting a trojan in my phone and check anything I do with my phone. That's where it is going. This is all messed up, no random bs article can change what's going on.
• FabHK 5y ago
  From A Concrete-Security Analysis of the Apple PSI Protocol: > Taking action to limit CSAM is a laudable step. But its implementation needs some care. Naively done, it requires scanning the photos of all iCloud users. But our photos are personal, recording events, moments and people in our lives. Users expect and desire that these remain private from Apple. Reciprocally, the database of CSAM photos should not be made public or become known to the user. Apple has found a way to detect and report CSAM offenders while respecting these privacy constraints. When the number of user photos that are in the CSAM database exceeds the threshold, the system is able to detect and report this. Yet a user photo that is not in the CSAM database remains invisible to the system, and users do not learn the contents of the CSAM database. https://www.apple.com/child-safety/pdf/Alternative_Security_Proof_of_Apple_PSI_System_Mihir_Bellare.pdf https://www.apple.com/child-safety/pdf/Alternative_Security_...
• roody15 5y ago
  Apple is not the police. Apple is not an extension of the Unites Government. There is simply no reason for Apple to enable local scanning for any content what so ever.
  • cblconfederate 5y ago
    Apple was (is?) part of PRISM
• ekianjo 5y ago
  > The worst part is: how do I put my money where my mouth is? Am I going back to using Linux on the desktop (2022 will be the year of Linux on the desktop, remember), debugging wifi drivers and tirelessly trying to make resume-from-suspend work? Oh come on. DOn't make it sound like it's that bad. Wifi is a solved problem for a long time now, and you can buy Lenovo, System76 or Tuxedo if you want to make sure 100% things work as expected. Don't be that guy.
  • rambambram 5y ago
    Second this. I use Ubuntu Mate on a RPi4 for a couple of weeks now. All went fine. Last week, I suddenly thought: I should connect my printer as well, and expected to have a slightly harder time, just like setting up my printer on my last Ubuntu pc. Click-click-done. I didn't have a hard time, not even with connecting my printer. I'm almost disappointed a bit, since there's no way I'm a cool computer guy if it's this easy.
  • KronisLV 5y ago
    > The worst part is: how do I put my money where my mouth is? ..., debugging wifi drivers and tirelessly trying to make resume-from-suspend work? Coincidentally, this is actually a good idea. Apart from using supported hardware (that others have checked actually works), contributing fixes for hardware that's not officially supported yet and hasn't been tested would benefit everyone in the future! I remember having to dig through GitHub to find a repository that had the network drivers for my off-brand Chinese/Polish netbook (i'm somewhat poor and/or frugal) and they actually worked and turned a system that would otherwise not have any network connectivity into my daily driver for note taking. Now, the fact that i couldn't automate this lookup process and that there's nothing out there that lets you check for these drivers more easily (think something along the lines of https://appdb.winehq.org/ https://appdb.winehq.org/ but for drivers) or maybe try multiple ones in a row, was disappointing because things felt needlessly hard. However, actually contributing or using the work of others isn't that much of a problem. And, since the whole ecosystem is pretty much open, there's nothing actually keeping one from at least trying to address these problems for their particular configuration, apart from needing to learn how to do so. In a sense, working on open source is exactly putting your money where your mouth is, even if it's just alternative costs.
• id5j1ynz 5y ago
  Apple implemented a backdoor that scans your photos on your device, then alerts Apple and the authorities if there is a match against an un-auditable list of reference photos. Currently it's been activated for CSAM only and only scans photos backed up to iCloud. That's the framing I prefer and which much better explains the issue with it.
• 8bitsrule 5y ago
  "I don’t care what anything was designed to do. I care about what it can do!" <= Gene Kranz in 'Apollo 13'
• GiorgioG 5y ago
  I turned off iCloud photos tonight. F** Apple. If there is a collision, then it gets manually reviewed by a human...so now my private pictures are on display for someone to see who I've not given permission. Just Say No.
• ekianjo 5y ago
  > Am I getting a Pixel and putting GrapheneOS on it like a total nerd? FUCK. Thanks for depicting people who care about privacy and act on their beliefs as "total nerds", that's an encouraging attitude.
  • BiteCode_dev 5y ago
    I'm a nerd and don't take it as an insult. Rather as "going full nerd on this" in the sense of giving a lot more importance and effort to a small detail in an already full life. And even if it was not, let's not get upset for every quip, I don't want to live in a world where bloggers have to ponder every word because they are afraid of offending someone. A bit of spice is ok, the dose makes the poison.
    • ekianjo 5y ago
      > I don't want to live in a world where bloggers have to ponder every word because they are afraid of offending someone. I don't find it offending, I find it stupid to write like that, that's not the same. So you write a long piece about how Apple is bad for privacy, just at the end to detract the available alternatives because you know, you don't like them for no particular reason? Who said that privacy was going to be easy anyway?
      • neop1x 5y ago
        As Stallmain said - people are trading privacy and freedom for convenience.
  • nohr 5y ago
    Nerd in 2021 is equivalent to "not casual/mainstream". If you consider sideloading OS's you are in fact a nerd.
    • ekianjo 5y ago
      It's not even remotely difficult to install GrapheneOS, so not sure how much of a "total nerd" you need to be to follow simple instructions. Has education dropped that low already?
• pers0n 5y ago
  Now if the government hates you they can claim they found this on your phone
• rambambram 5y ago
  So Apple is going to take care of positive matches with highly reliable and trained personnel? Just like their highly trained personnel who kept the App Store clean of shitty apps? :')
• raxxorrax 5y ago
  On the internet almost everyone wants to extract money from kids and their parents and they try to hook them up with different mechanisms. That is also true for Apple, although they appeal to protective instincts of their guardians. I get why a safe environment is appealing. Parents know that their kids get milked by virtual goods in games or social media and don't know how to protect them from that. I think states are indeed responsible to set sensible boundaries for the industry to protect minors. But this cannot lead to subject the whole net to it. Age verification is also not possible, so a protected environment is the way to go. The latter is difficult to advertise to developers because they also know about corporate ambitions to get their hands on market share. Google isn't even the worst actor, more aggressive corps like Amazon are far more destructive in this field, but there isn't a single corp that is guilty here, so legislation also needs to protect free spaces. While seemingly in contradiction, this is also extremely important for digital education of future generations, even more so than questionable content in my opinion. Most here might have been subjected to that as kids. Was it that bad as generally assumed? This is a threat that should not be overblown. Parents feeling guilty neglecting their kids are extremely vulnerable to this line of thinking, even if they don't neglect their kids at all. Many countries have rules against cartels, but there is a conflict of interest here. No country likes to split their most successful companies for nothing in an international market. So nobody does.
• arespredator 5y ago
  Hi, post author here. To anyone upset or offended by the Linux/nerd paragraph: please chill, and please forgive my tone. I am a nerd myself indeed, and what I wanted to convey by this not-as-funny-sa-expected paragraph was that "going full nerd" is not a solution. There are ways to protect your privacy that will not be available to less tech-savvy people, and it's a problem. HN crowd will use Thinkpads with Arch on them, and phones with Graphene or whatever, but most people won't. Yours, Absolute nerd and lover of desktop Linux since SuSE 6.0
  • fsflover 5y ago
    > HN crowd will use Thinkpads with Arch on them, and phones with Graphene or whatever, but most people won't. Non-nerds can just buy devices with preinstalled Linux and never care about the maintenance or support. I never had any problems with WiFi or suspend on my Librem 15. Same I expect from Librem 5 smartphone.
• ashneo76 5y ago
  We need more people using Linux on desktop and pouring money into the Linux phone to justify these. Voting with your wallet matters and works. It is why apple and Google still do so much marketing and hype about their phones and devices.
• citizenpaul 5y ago
  This system has 0 transparency. I cannot appeal. I cannot check my status. I cannot check this mysterious "counter" I cannot even check an image i have to see if it is flagged. I cannot know who is manually looking at my data no matter how private it its. I get a note in my fruit delivery with the name of the person and time they were at work packing my food. Yet I'm told that I cannot know anything about what is happening with at least a partially automated system that can potentially put me in jail for the next 20 years?