51 ms·

Security Threat Model Review of the Apple Child Safety Features [pdf]

325 points by sylens 5y ago

sylens 5y ago
I think this is the first time they have mentioned that you will be able to compare the hash of the database on your device with a hash published in their KB article. They also detailed that the database is only the intersection of hash lists from two child safety organizations under separate governmental jurisdictions. My immediate thought is that this could still be poisoned by Five Eyes participants, and that it does not preclude state actors forcing Apple to replicate this functionality for other purposes (which would leave the integrity of the CSAM database alone, thus not triggering the tripwire).
- notJim 5y ago
  > this could still be poisoned by Five Eyes participants, and that it does not preclude state actors forcing Apple to replicate this functionality for other purposes The thing is, if this is your threat model you're already screwed. Apple has said they comply with laws in jurisdictions where they operate. The state can pass whatever surveillance laws they want, and I do believe Apple has shown they'll fight them to an extent, but at the end of the day they're not going to shut down the company to protect you. This all seems orthogonal to the CSAM scanning. Additionally, as laid out in the report, the human review process means even if somehow there is a match that isn't CSAM, they don't report it until it has been verified.
- vmception 5y ago
  the opportunity being to add general functions in photo viewing apps that add a little entropy to every image (for this specific purpose), to rotate hashes, rendering the dual databases useless monetization I guess being to hope for subscribers on github, as this could likely just be a nested dependency that many apps import. a convenient app for this specific purpose might not last long in app stores.
  - initplus 5y ago
    Perceptual hashes are specifically designed to be resistant to minor visual alterations.
    - vmception 5y ago
      Let the cat and mouse race begin Any perceptual hash apps to test that theory on?
magicloop 5y ago
I think is this a good document and it also brings to the table the threshold images count (around 30) and the alternate neural hash they will keep private (to guard against adverserial images trying to create a false positive on-device). FWIW, I actually did an amateur threat model analysis in a comment in separate HN thread. I always thought this was called for because the initial document set was just the mathematics, not the people/process/implementation/policy risks and threat model that was the source of widespread concerns.
shuckles 5y ago
It’s pretty lonely over here in technical discussion land. Have we considered Reuters’s intern’s take on this?
- dang 5y ago
  Please don't post unsubstantive comments yourself. If other threads/comments aren't good, options include posting something better, or not posting. Degrading things even further is not a good choice (edit: particularly when the thread is new—threads are extremely sensitive to initial conditions, and a flamebait/unsubstantive comment early on can have a strongly degrading effect). https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=by%3Adang%20threads%20sensitive%20initial%20conditions&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
  - shuckles 5y ago
    Sincere question: why doesn’t this policy apply to the innumerable number of hot takes which are variations on the exact same, often misinformed, reaction? I’m thinking of comments like, “Apple will send me to prison for taking bath tub pics of my infant!” I have seen many on here.
    - dang 5y ago
      Two answers. First, it does apply. If you see a post that ought to have been moderated but hasn't been, the likeliest explanation is that we didn't see it. We don't come close to seeing everything that gets posted here—there is far too much. You can help by flagging it or emailing us at hn@ycombinator.com. Second, there are degrees of these things. It's definitely bad for comments to repeat the same shallow things over and over, but the GP was particularly information-free and snarky, and mucking up this thread while claiming to be speaking in favor of technical discussion is particularly counterproductive. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=likeliest%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
underseacables 5y ago
What’s not discussed in this Paper is how it won’t be used by third-party actors to falsely accuse someone. Does nobody remember the iCloud hack? What about instead of downloading Jennifer Lawrence’s photos, a hacker uploaded children to get someone falsely accused?
- dogma1138 5y ago
  Google, Microsoft and a bunch of other services already scan for CSAM materials, yet this hasn’t been an issue so far. In fact I can’t really find a single case when someone was framed like this, despite how easy it is to fill someone’s account with CSAM and call the cops/wait. I don’t like the privacy and property rights issues of this but the whole someone will use this to frame someone is quite BS.
  - staticassertion 5y ago
    As I recall someone's neighbor once tried to frame them by downloading CSAM on their network and then calling the police or something like that.
  - fuckcensorship 5y ago
    How would someone prove that they were framed by someone planting CSAM? Without evidence, how likely is it that they would be believed?
  - underseacables 5y ago
    But the services you mentioned don’t do it on the device. The blind trust in Apple that everything will be ok is quite BS, and very naïve .
  - Renaud 5y ago
    1) how would you know someone's being framed if it's not detected as such? in this case it just looks like the target had CSAM on their machine/account and they are found guilty, despite their objections, maybe because no-one really looked closely at the evidence, or it was planted well enough. 2) if it is detected as a plant. Would you know it? Would it fly above our radar? There is evidence this happens [1], some cases are quite high profile [2]. 3) This technique is already in use. Russia is allegedly using it to discredit opponents [3]. Do you think you would hear from it if it was done by secret services of any country to discredit or get rid of foreign targets or even some at home? 4) it's the ultimate weapon against foes and opponents. It's a dangerous one but I have to wonder how many estranged spouses get revenge planting evidence [4], or how many journalists or troublemakers have been the recipient of some carefully crafted planting. I mean, nearly anyone reading HN could do it sufficiently well to get someone in serious trouble, it's not really a technical issue so it's not hard to imagine it's being done. [1]: https://www.independent.co.uk/news/uk/crime/handyman-planted-child-porn-colleague-s-computer-2037439.html https://www.independent.co.uk/news/uk/crime/handyman-planted... [2]: https://www.miaminewtimes.com/news/shaq-hit-with-lawsuit-for-attempting-to-plant-child-porn-on-former-employees-computer-6530360 https://www.miaminewtimes.com/news/shaq-hit-with-lawsuit-for... [3]: https://www.nytimes.com/2016/12/09/world/europe/vladimir-putin-russia-fake-news-hacking-cybersecurity.html https://www.nytimes.com/2016/12/09/world/europe/vladimir-put... [4]: https://www.crimeonline.com/2019/07/04/wife-plants-child-porn-on-husbands-phone-falsely-accuses-him-of-raping-their-daughter-a-judge-gives-her-probation/ https://www.crimeonline.com/2019/07/04/wife-plants-child-por...
tehnub 5y ago
Their use of the phrase "This claim is subject to code inspection by security researchers like all other iOS device-side security claims" stood out to me. Could someone tell me how that inspection works? Are there researchers who are given the source code? (I posted this on another thread [0] earlier, but it's more relevant here) [0]: https://news.ycombinator.com/item?id=28175619 https://news.ycombinator.com/item?id=28175619
- skybrian 5y ago
  It’s not my field, but my understanding is that security researchers often disassemble and/or debug binaries without having access to the source code. For example, as soon as an OS update is released, people will take it apart. Having source might be nice but it’s not necessary.
  - deleted 5y ago
    [deleted]
- eivarv 5y ago
  Could someone tell me how that inspection works? Are there researchers who are given the source code? It doesn't (with the exception of an exclusive program[0]). Not only do they generally make security work on their devices difficult – they have a history of suing companies that facilitate it [1]. [0]: https://developer.apple.com/programs/security-research-device/ https://developer.apple.com/programs/security-research-devic... [1]: https://www.theverge.com/2021/8/11/22620014/apple-corellium-security-virtual-iphone-dmca-lawsuit-settled https://www.theverge.com/2021/8/11/22620014/apple-corellium-...
- kevinsundar 5y ago
  Apple actually gives special devices to Security Researchers that allow them further access into the device than a normal consumer device: https://developer.apple.com/programs/security-research-device/ https://developer.apple.com/programs/security-research-devic... In this way, third party security researchers can verify their claims. It actually works out pretty well for them since third party security researchers often find pretty severe vulnerabilities through this program.
almostdigital 5y ago
> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating system that supports the feature. Additionally, users will be able to inspect the root hash of the en- crypted database present on their device, and compare it to the expected root hash in the Knowledge Base article. This is just security theater, they already sign the operating system images where the database reside. And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. > This feature runs exclusively as part of the cloud storage pipeline for images being up- loaded to iCloud Photos and cannot act on any other image content on the device Until a 1-line code change happens that hooks it into UIImage.
- kemayo 5y ago
  > And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust your OS vendor.
  - majormajor 5y ago
    If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud). And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc. It's curious that this is the particular breaking point on the slope for people. The "on device" aspect just makes it more immediate feeling, I guess?
    - kemayo 5y ago
      Yeah, it's weird. Speaking purely personally, whether the scanning happens immediately-before-upload on my phone or immediately-after-upload in the cloud doesn't really make a difference to me. But this is clearly not a universal opinion. The most-optimistic take on this I can see is that this program could be the prelude to needing to trust less people. If Apple can turn on e2e encryption for photos, using this program as the PR shield from law enforcement to be able to do it, that'd leave us having to only trust the OS vendor.
rootsudo 5y ago
Truly, 1984. Siri now gives you suggestions if you have dangerous thoughts. "Anonymous helplines and guidance exist for adults with at-risk thoughts and behavior" - "Learn more and get help." Now a fake AI on your phone that knows your actions and behaviors can sleep and wait for specific triggers and give you feedback and alert.
TrumpRapedWomen 5y ago
Why don't they just scan photos service side? Are they trying to save on CPU costs?
jorgesborges 5y ago
I keep changing my mind on this. On the one hand I already operate on the assumption that uploading data to a cloud service renders that data non-private, or at least in great risk of becoming non-private in the future. This simply makes my operating assumption explicit. Also this particular implementation and its stated goals aren’t egregious. But then there’s the slippery slope we’ve all been discussing — and the gradual precedents set for it, this being a nail in the coffin. But I sympathize with Apple for making transparent what I assume happens behind closed doors anyway. My optimistic side thinks maybe this is a sign that extensions of this project will also be made explicit. Maybe this is the only way for them to take this fight if they’re being pressured. In any case now is the time to speak out. If they extend it to texts and other documents I’m out - even as it stands I’m thinking of alternatives. But the fact we’re all discussing it now is at least a point of hope.
- naasking 5y ago
  > But I sympathize with Apple for making transparent what I assume happens behind closed doors anyway. Using your devices' CPU and battery seems more egregious than doing it on their servers. If they want to help law enforcement, then they should pay for it. Of course they want to help law enforcement by forcing other people to pay the costs. Imagine if Ford came out with a cannabis sensor in their cars that automatically called the cops on you if it detected cannabis inside the car. The sensor is allegedly only active when you're renting a Ford vehicle, not if you purchase it outright. Not a perfect analogy, but how comfortable would you find this situation?
  - Spivak 5y ago
    Let's do a more realistic example. Ford introduces a "driver safety" mechanism where the car requires a clean breathalyzer reading in order to start. If it fails it pops up a message that reminds you that drunk driving is illegal but doesn't actually stop you from starting the engine. It then sends Ford the results in an encrypted payload along with 1/30th of the decryption key. After 30 reports someone at Ford opens the payloads, looks at the results and, decides whether to contact the police based on how damning the evidence is. Because all the tests are performed locally you're never giving up any of your privacy unless you actually are driving drunk or are the 1/billion and get 30 false positives. I feel like this is a much stronger than the "if you have nothing to hide" meme because local testing like this lets you can have everything to hide while still revealing specific bad behavior. Like if the TSA had scanners that somehow could only reveal banned items and nothing else I’m not sure I would even consider it a privacy violation.
PaulDavisThe1st 5y ago
> Apple generates the on-device perceptual CSAM hash database through an intersection of hashes provided by at least two child safety organizations operating in separate sovereign jurisdictions – that is, not under the control of the same government. Any perceptual hashes appearing in only one participating child safety organization’s database, or only in databases from multiple agencies in a single sovereign jurisdiction, are discarded by this process, and not included in the encrypted CSAM database that Apple includes in the operating system. Well, that's quite clever, isn't it. It's just the sort of thing that someone who had "never heard of" the Five Eyes, or extradition treaties or illegal rendition operations involving multiple nations, might come up with. Genius!
- cmsj 5y ago
  If your threat model includes pervasive spying by multiple nation states, and being grabbed in the night by black helicopters, it seems unlikely you'll be overly concerned about them precisely inserting at least 30 of your photos into multiple CSAM databases and also co-ercing Apple's manual review to get you reported to NCMEC.
  - sylens 5y ago
    I don't think people are worried about multiple nation states framing them with CSAM photos - they're worried about multiple nation states in an intelligence collaboration poisoning both sets of hash lists with non-CSAM material, so that there is an intersection that makes it onto the device. There is still that Apple human reviewer once the threshold has passed. What I would love to ask Apple is - what happens if/when their reviewers start noticing political material, religious material, etc. is being flagged on a consistent basis, thereby insinuating that the hash list has been poisoned. What's their play at that point?
    - jdlshore 5y ago
      The document states that incorrectly flagged items are forwarded to engineering for analysis. Given their target false-positive rate (1 in 3 trillion, was it?) it seems likely that engineering would very carefully analyze a rush of false positives.
lifty 5y ago
Is there a good argument for doing the scanning on the phone and not on the iCloud servers?
- kemayo 5y ago
  Well, theoretically this could be a step towards e2e encrypting the photos so that Apple can't see them on their servers. This entire "secure tickets that unlock a derivative of your photos so we can review them" system would, in this interpretation, be there as a way to neuter law enforcement objections[1] to no longer being able to serve warrants for the contents of your photo library. Now, will this actually happen? Hard to say. [1] There was a story last year claiming that Apple wanted to turn on e2e, but the FBI got them to drop it. https://www.popularmechanics.com/technology/security/a30631827/apple-fbi-encryption-whatsapp/ https://www.popularmechanics.com/technology/security/a306318...
- himaraya 5y ago
  No. Contrary to Craig's claims, Google & Microsoft et al. do similar hash matching on the cloud, instead of 'looking at images'.
  - notJim 5y ago
    But people consider that bad! With this system, Apple can never look at your photos in the cloud. They are a pipe for the bits, but they don't examine the contents, except on your device. I'm not saying I necessarily agree that this is better, all-told (mixed feelings), but I think it's worth acknowledging why a rational actor might consider this superior to Google doing whatever the fuck they want with anything you upload.
    - himaraya 5y ago
      I mean, Apple can, as long as iCloud remains unsecured by e2e, the roadmap for which remains MIA.
      - notJim 5y ago
xuki 5y ago
What is the process to make sure the next management will honor these promises?
- hypothesis 5y ago
  Heck, what is the process for current management to honor them?
  - xuki 5y ago
    Weirdly enough, I trust the current management to do the right thing. But that can be changed at a whim, and in my opinion, the greatest security threat to this whole thing.
    - himaraya 5y ago
      I mean, I trusted current management not to pull stunts like this to begin with.
- notJim 5y ago
  Apple management changing would be a massive threat regardless of these premises. Apple has gone to bat for encryption and privacy to a greater degree than Google, for example, and that's because of their current management. You should not in any way assume that the next group will do the same!
deleted 5y ago
[deleted]
cmsj 5y ago
This seems like a pretty helpful document, and really shows how carefully the CSAM system has been designed. Two things that stuck out to me: 1) The CSAM hash database is encrypted on-device 2) The device doesn't know the results of hash comparisons These suggest to me that the hash comparison is going to happen inside the Secure Enclave - give it the encrypted database and the NeuralHash of an image, it gives you back a Security Voucher to upload. I don't quite get how the hash database can stay secret though - the decryption key has to come from somewhere. AFAIK the hash database has never been made publicly available before, which probably makes it quite a high value target for security researchers and curious parties.
- shuckles 5y ago
  The hash database is blinded on the server before being embedded into iOS. You’d need to access Apple’s private key or break cryptography to get access to it from the client. For the less cryptographically savvy, the “Technical Summary” is a good document on their Child Safety page.
- FabHK 5y ago
  > These suggest to me that the hash comparison is going to happen inside the Secure Enclave - give it the encrypted database and the NeuralHash of an image, it gives you back a Security Voucher to upload. My understanding is that the hash comparison, or at any rate the counting of matches, is done in the cloud then. The way the crypto is set up is that the server has no access to the security envelope and its contents (and, after E2EE in the future, to any images) unless the number of matches exceeds the threshold.
- nullc 5y ago
  No secure enclave is needed, the system uses strong cryptography to protect Apple and their database providers from accountability. I gave a semi-technical description of how private set intersection works here: https://news.ycombinator.com/item?id=28124716 https://news.ycombinator.com/item?id=28124716 Assuming they implement it correctly and there are no apple-internal leaks we won't be discovering the content of the database.
- stjohnswarts 5y ago
ty___ler 5y ago
If apple can be coerced by governments to scan photos on the serverside then what is the operational purpose of a frontend scan, given what apple is publicly saying they are trying to do? This is the most confusing aspect for me.
- codeecan 5y ago
  Apple will turn it on eventually for non iCloud destined photos (and documents), you don't build a system like this not to use it. We know Apple concedes to China's demands, and with another Snowden situation, the US would not hesitate to add classified documents to the scan list and identify individuals. This system will catch no abusers, because they're not Airdropping photos to each other. And if they're smart enough to use a VPN, they're not storing that on a phone thats logged into the cloud. And if they're not using a VPN, they can be caught at download time.
concinds 5y ago
I strongly considered switching away from Apple products last weekend; but this document has convinced me otherwise. The threats people identify have minimal risk. If a total stranger offers you a bottle of water, you may worry about it being spiked, but him having offered the bottle doesn't make it more, or less, likely that he'll stab you after you accept it. They're separate events, no "slippery slope". It's very convincing that any alteration that would "scan" your whole phone would be caught eventually, and even if it's not, the announcement of this feature has no bearing on it. If Apple has evil intent, or is being coerced by NSLs, they would (be forced to) implement the dangerous mechanism whether this Child Safety feature existed or not. The concept of backdoors is hardly foreign to governments, and Apple didn't let any cats out of any bags. This document shows that Apple did go to great lengths to preserve privacy; including the necessity of hashes being verified in two juristictions, the fact that neither the phone nor Apple know if any images have been matched below the threshold; the lack of remote updates of this mechanism; the use of vouchers instead of sending the full image; the use of synthetic vouchers; and on and on. Furthermore, the risks of the risks of this mechanism are lower than the existing PhotoDNA used by practically every competing service. Those have no thresholds; the human review process is obscure; there is no claim that other pictures won't be looked at. The controversy, the fact that it uses an on-device component. But PhotoDNA-solutions fail many of Apple's design criterias, which require an on-device solution: - database update transparency - matching software correctness - matching software transparency - database and software universality - data access restrictions. What about the concerns that the hash databases could be altered to contain political images? PhotoDNA could be, too; but would be undetectable unlike with Apple's solutions. Worse: with serverside solutions, the server admin could use an altered hash DB only for specific target users, causing harm without arousing suspicion. Apple's design prevents this since the DB is verifiably universal to all users. A rational look at every argument I've seen against Apple's solution indicates that it is strictly superior to current solutions, and less of a threat to users. Cryptographically generating vouchers and comparing hashes is categorically not "scanning people's phones" or a "backdoor." I think the more people understand its technical underpinnings, the less they'll see similarities with sinister dystopian cyberpunk aesthetics.
- Syonyk 5y ago
drodgers 5y ago
> The second protection [against mis-inclusion of non CSAM hashes] is human review: there is no automated reporting in Apple’s system. All positive matches must be visually confirmed by Apple as containing CSAM before Apple will disable the account and file a report with the child safety organization. I don't understand this at all. As I understand it, part of the problem is that — in the US — Apple isn't legally allowed to review, transmit or do anything else with suspected CSAM images, so they can't have a manual review process (or even check that their neural hashing is working as expected on the real dataset). Does anyone else have any idea of what this is trying to describe? If Apple really are somehow reviewing flagged photos to confirm that they're CSAM and not maliciously flagged files before sending any reports, then that does make the system substantially more resilient to Five Eyes abuse (not that I wish that job on anyone). Edit: There's more context later in the document > First, as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possibility that the match threshold was exceeded due to non-CSAM images that were adversarially perturbed to cause false NeuralHash matches against the on-device encrypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation. This is more confusing: visually comparison using a second perceptual hash doesn't actually to provide any protection against mis-inclusion of non-CSAM images: it just double-checks that the image really was a match in the database (ie. protects against hash-collision errors), but it does't check that the database contained actual CSAM. Apple explicitly says that this process protects against mis-inclusion though, which doesn't make sense to me yet.
- shuckles 5y ago
  Your understanding is incorrect. Apple can, and is in fact required to, verify that they have actual CSAM before forwarding it to the Cyber Tip line. At that point, they must delete the information within 60 days.
  - drodgers 5y ago
    Interesting. In that case, do you know why they talk about reviews only seeing "visual derivatives" (from the second perceptual hash)? Either these 'derivatives' basically contain the original image (so reviewers can verify that it's actual CSAM) and there's no point in using derivatives at all, or they're more abstract (eg. 8x8 pixellated images) in which case the reviewer can't see the actual content (but could confirm a database match). Edit: I was able to find the answer, they suggest that the 'visual derivative' is something like a 'low-resolution version' of the original image, so the content should still be clearly visible.
chipotle_coyote 5y ago
In other HN comments on this subject I've (hopefully) made it clear that I'm not really in favor of this project of Apple's, and that there's a legitimate "slippery slope" argument to be made here. So I hope people will entertain a contrarian question without downvoting me into oblivion. :) Here's the thing I keep circling around: assume that bad actors, government or otherwise, want to target political dissidents using internet-enabled smartphones. The more we learn about the way Apple actually implemented this technology, the less likely it seems that it would make it radically easier for those bad actors to do so. For instance, the "it only scans photos uploaded to iCloud" element isn't just an arbitrary limitation that can be flipped with one line of code, as some folks seem to think; as Erik Neuenschwander, head of Privacy Engineering at Apple, explained in an interview on TechCrunch[1]: > Our system involves both an on-device component where the voucher is created, but nothing is learned, and a server-side component, which is where that voucher is sent along with data coming to Apple service and processed across the account to learn if there are collections of illegal CSAM. That means that it is a service feature. Will this stop those bad actors if they're determined? No, of course not, but there are so many ways they can do it already. They'll get cloud storage providers to give them access. If they can't, they'll get network providers to give them access. And if those bad actors are, as many people fear, government actors, then they have tools far more potent than code: they have laws. It was illegal to export "strong encryption" for many years, remember? I've seen multiple reports that European lawmakers are planning to require some kind of scanning for CSAM. If this goes into effect, technology isn't going to block those laws for you. Your Purism phone will either be forced to comply or be illegal. I wrote in a previous comment on this that one of Silicon Valley's original sins is that we tend to treat all problems as if they're engineering problems. Apple is treating CSAM as an engineering problem. Most of the discussion on HN about how horrible and wrong Apple is here still treats it as an engineering problem, though: well, you can get around this by just turning off iCloud Photos or never using Apple software or throwing your iPhone in the nearest lake and switching to Android, but only the right kind of Android, or maybe just never doing anything with computers again, which admittedly will probably be effective. Yet at the end of the day, this isn't an engineering problem. It's a policy problem. It's a governance problem. In the long run, we solve this, at least in liberal democracies, by voting people into office who understand technology, understand the value of personal encryption, and last but certainly not least, understand the value of, well, liberal democracy. I know that's easy to dismiss as Pollyannaism, but "we need to protect ourselves from our own government" has a pretty dismal track record historically. The entire point of having a liberal democracy is that we are the government, and we can pull it back from authoritarianism. The one thing that Apple is absolutely right about is that expanding what those hashes check for is a policy decision. Maybe where those hashes get checked isn't really what we need to be arguing about. [1]: https://techcrunch.com/2021/08/10/interview-apples-head-of-privacy-details-child-abuse-detection-and-messages-safety-features/ https://techcrunch.com/2021/08/10/interview-apples-head-of-p...
616c 5y ago
I have long been interested in what a professional-grade threat model from a large FAANG/SV organization is. Is this a representative model? Microsoft came up with DREAD and STRIDE and they suggest there threat models are more elaborate. Would love to see more representative examples!
- jdlshore 5y ago
  This feels more like a marketing document to me than a workaday threat model. It's fairly handwavey, and the goal seems to be to convince rather than to do a hardnosed analysis. Not that it's not useful—I found it convincing—but I doubt this is what a real threat model looks like. Not that I actually know. I'd be interested in seeing a real one too.
hendersoon 5y ago
I have no doubt the features work exactly the way Apple said they do. Seriously. I very much doubt that they will refuse to scan for whatever China asks. I very much doubt they'll risk the PRC shutting down Foxconn factories. I very much doubt Apple will ultimately be able to resist scanning for whatever the US Government asks in a national security letter attached to a gag order. They will take them to a secret court session which we'll never hear about, the court will rule in the government's favor, and Apple will be forced to comply and not allowed to tell anybody. This happened to Cloudflare and they were gagged while fighting it for 4 years. They didn't win in court, but the government eventually dropped the case as it was no longer needed. And this started under Obama, not Trump. Now, will Apple bother to fight this out in court? I think they probably will. But we won't know about that until they either definitively win or the government gives up. If the government wins, on the other hand, we'll NEVER hear about it.
- heavyset_go 5y ago
  > Now, will Apple bother to fight this out in court? I think they probably will. Why would they? Each year they give up customer data on over 150,000 users without a fight when the US government simply asks them for it[1]. [1] https://www.apple.com/legal/transparency/us.html https://www.apple.com/legal/transparency/us.html
throwawaymanbot 5y ago
Chinafy the population.
semerda 5y ago
“Human review and reporting” - fact checkers? Or are we to assume that their AI is 100% effective. This will not end well..
Jerry2 5y ago
What strikes me about this paper is that there are no names of the people who wrote it on the title page. Why were they afraid to put their name(s) on this?
- celeritascelery 5y ago
  Seeing HN over the last week, I can think of a few reasons…
  - stjohnswarts 5y ago
    Yeah seeing as its just a white paper there really isn't much need, and the possibility of retribution is immense on this one. I know a lot of pissed off people, let alone some of the nuts who this is angering and who could dox or do worse to the authors.
gandalfgeek 5y ago
Sounds like it still comes down to trust. Quoting from the paper: "Apple will refuse all requests to add non-CSAM images to the perceptual CSAM hash database; third party auditors can confirm this through the process outlined before. Apple will also refuse all requests to instruct human reviewers to file reports for anything other than CSAM materials for accounts that exceed the match threshold."
- celeritascelery 5y ago
  To me, the fact that they are making such a strong statement is a big deal. If they had plans or thought that they would be forced to add other content they would not be so direct in this statement. I hope I am not wrong. As you said: trust.
- 0xy 5y ago
  It's already a lie, considering NCMEC's database already has non-CSAM images. It won't ever be true, since the hundreds and possibly thousands of entities with direct and indirect access to the database can upload SunnyMeadow.jpg labeled as CSAM content, and it's blindly accepted. If Apple is receiving a hash labeled as "CSAM", how can they possibly make this guarantee? They do not know it's CSAM. It's unverifiable and comes down to "trust us". Remember, NCMEC is an unaccountable organisation inextricably linked with the FBI and the U.S. government. It is not subject to FOIA requests. Its database has never been audited, and it is known to be a mess already.
  - simondotau 5y ago
    > "hundreds and possibly thousands of entities with direct and indirect access to the database can upload SunnyMeadow.jpg labeled as CSAM content, and it's blindly accepted." According to whom? > "it is known to be a mess already." According to whom? You've posted a number of claims about the CSAM database administration process on HN and the one instance where you've cited a source, it's been a junk source.
makecheck 5y ago
I don’t like the idea of stuff running on my device, consuming my battery and data, when the only point is to see if I am doing something wrong? An analogy I can come up with is: the government hires people to visit your house every day, and while they’re there they need your resources (say, food, water, and electricity). In other words, they use up some of the stuff you would otherwise be able to use only for yourself — constantly — and the only reason they’re there is to see if you’re doing anything wrong. Why would you put up with this?
- shuckles 5y ago
  The point of it is to make sure iCloud Photos remains a viable service in light of real and perceived regulatory threats, and possibly leave the door open to end to end encryption in the future.
  - gpm 5y ago
    There is no regulatory threat within the US that could require this happen, if this was demanded by the government it would be a blatant violation of the 4th amendment. Apple should have stood their ground if this was in response to perceived government pressure.
    - dwaite 5y ago
      IMHO, the relevant regulations are pretty carefully worded to make 4th amendment defenses harder. In this case, Apple has some liability for criminal activity on their platform whether it is was E2E encrypted or not.
    - matwood 5y ago
      There are two gov. issues. One is if the FBI comes knocking and the other is new laws. The government could absolutely write a law banning e2ee so that when the FBI does knock with a warrant they can get access. In the past, Apple has done what they can to stand their ground against the first, but they (like any other company) will have to comply with any laws passed. Whether the 'if a warrant is obtained the gov. should be granted access' law is a violation of the 4th amendment remains to be seen.
1f60c 5y ago
Having read the document, I am even more confident in Apple's approach than before. I commend Apple for its transparency.
bnj 5y ago
A curious outcome of the scrutiny that the system design is receiving is that it will make it less effective at the task of catching actual collections of CSAM. Page eleven promises something I haven’t seen before: that Apple will publish the match threshold for each version. Meaning, even if after all this, there are abusers still using iCloud photos to manage their illegal collection of CSAM, they can check back to know what number of images is safe to keep. A twisted version of the Streisand effect.
- somebodythere 5y ago
  If you knew about the existence of this program and were determined to keep a collection of CSAM on your phone I don't understand why someone would choose to dance around specific thresholds rather than just disable iCloud sync.
m3kw9 5y ago
Setting arguments aside, if ever there is a CSAM scanner I’d let loose in my account is Apples.
deleted 5y ago
[deleted]
swiley 5y ago
Burn it all down. If you can't understand the theory and reimplement it then you shouldn't even look at it. The internet is full of pathological wizards and witches. Come here for obscure books but otherwise stay far away.
unityByFreedom 5y ago
> The system is designed so that a user need not trust Apple, any other single entity, or even any set of possibly-colluding entities from the same sovereign jurisdiction (that is, under the control of the same government) Who's to say authoritarian governments don't team up? We just saw this happen over the last decade with the rise of nationalism everywhere. We already know governments can put significant pressure on private companies. Don't capitulate, Apple. Blow the whistle. You're getting screwed and you have our support if you just stand up for what you said 5 years ago.
zepto 5y ago
A key point that needs to be mentioned: we strongly dislike being distrusted. It might well be a genetic heritage. Being trusted in a tribe is crucial to survival, and so is likely wired deep into our social psychology. Apple is making a mistake by ignoring that. This isn’t about people not trusting Apple. It’s about people not feeling trusted by Apple. Because of this, it doesn’t matter how trustworthy the system is or what they do to make it less abusable. It will still represent distrust of the end user, and people will still feel that in their bones. People argue about the App Store and not being trusted to install their own Apps etc. That isn’t the same. We all know we are fallible and a lot of people like the protection of the store, and having someone to ‘look after’ them. This is different and deeper than that. Nobody want to be a suspect for something they know they aren’t doing. It feels dirty.
- shuckles 5y ago
  But the catch is: all the incumbents already treated your data as if you were guilty until proven innocent. Apple’s transparency about that change may have led people to internalize that, but it’s been the de facto terms of most cloud relationships. What I personally don’t understand is why Apple didn’t come out with a different message: we’ve made your iPhone so secure that we’ll let it vouch for your behalf when it sends us data to store. We don’t want to see the data, and we won’t see any of it unless we find that lots of the photos you send us are fishy. Android could never contemplate this because they can’t trust the OS to send vouchers for the same photos it uploads, so instead they snoop on everything you send them. It seems like a much more win-win framing that emphasizes their strengths.
  - zepto 5y ago
    I agree this is vastly better than what anyone else is doing, and you know I understand the technology. However, I don’t think any framing would have improved things. I think it was always going to feel wrong. I would prefer they don’t do this because it feels bad to be a suspect even in this abstract and in-practice harmless way. Having said that, having heard from people who have investigated how bad pedophile activity actually is, I can imagine being easily persuaded back in the other direction. I think thins is about the logic of evolutionary psychology, not the logic of cryptography. My guess is that between now and the October iPhone release we are going to see more media about the extent of the problem they are trying to solve. That is how Apple wins this.
simondotau 5y ago
Here's what I don't get: who is importing CSAM into their camera roll in the first place? I for one have never felt the urge to import regular, legal porn into my camera roll. Who the hell is going to be doing that with stuff they know will land them in prison? Who the hell co-mingles their deepest darkest dirtiest secret amongst pictures of their family and last night’s dinner? I can believe that some people might be so technically illiterate that they'd upload CSAM to a private Facebook group thinking it was secure. But as far as I'm aware it's not possible to password protect photo albums using the built-in Photos app.
- simondotau 5y ago
  Actually it occurs to me that perhaps they're noticing a lot of CSAM is being produced with smartphone cameras and are hoping to snag the phone which produced the originals. If they can get new content into their database before the photographer deletes them, they might find the phone which took the originals—and then find the child victim. Beyond implausible, but then most law enforcement tends to rely on someone eventually doing something stupid.
  - phnofive 5y ago
    It is only supposed to detect CSAM already known to NCMEC, not identify new images.
    - simondotau 5y ago
      That's my point. If the original photographer doesn't delete them by the time they're known to NCMEC and the hash database is updated, they would match. As I said, I recognise this is beyond implausible.
    - 0xy 5y ago
      NCMEC's database does not only contain CSAM. It has never been audited. It's full of false positives. They are immune from FOIA requests. They are accountable to nobody. They work so closely with the FBI that there are FBI agents working on the database directly. NCMEC is essentially an unaccountable, unauditable department of the FBI that also happens to be incompetent (the amount of non-CSAM in the database is large).
fortran77 5y ago
Why don't they scan files as they're downloaded, the way we scan for malware and viruses on PCs? This way, we can warn people _before_ a law is broken!
akomtu 5y ago
This is Apple's trying to shift the focus onto technicalities. Remotely monitored CCTV cams in your house is never a good idea, no matter what the treat model is.
lucasyvas 5y ago
If they continue down this road, they will not only have effectively created the smartphone category, but also coincidentally destroyed it. If our devices are designed to spy on us, we're frankly not even going to use them anymore. I wonder if they forgot that using this thing is optional? You'll see a resurgence of single purpose dumb devices.
istingray 5y ago
I just found the perfect video clip to represent Apple's response to launching this feature: https://youtu.be/-7XchZICFQU?t=17 https://youtu.be/-7XchZICFQU?t=17
makerofthings 5y ago
My phone is an extension of my brain, it is my most trusted companion. It holds my passwords, my mail, my messages, my photos, my plans and notes, it holds the keys to my bank accounts and my investments. I sleep with it by my bed and I carry it around every day. It is my partner in crime. Now apple are telling me that my trusted companion is scanning my photos as they are uploaded to iCloud, looking for evidence that apple can pass to the authorities? They made my phone a snitch? This isn't about security or privacy, I don't care about encryption or hashes here, this is about trust. If I can't trust my phone, I can't use it.
- zabatuvajdka 5y ago
  Technically they could have been scanning the photos already to power some AI algorithms or whatever else. I think this is a nudge for folks to wake up and see the reality of what it means to use the cloud. We are leasing storage space from Apple in this case. Technically, it’s no different than a landlord checking up on their tenants to make sure “everything is okay.” And technically, if you do not like iCloud, don’t use it and roll your own custom cloud storage! After all, it’s redundancy and access the cloud provides. And Apple provides APIs to build them. Hell, with the new Files app on iOS i just use a Samba share with my NAS server (just a custom built desktop with RAID 5 and Ubuntu.)
  - a012 5y ago
    > Technically, it’s no different than a landlord checking up on their tenants to make sure “everything is okay.” By this logic, you don't own your phone but rent it from Apple?
    - zabatuvajdka 5y ago
      Technically I own my iPhone but the software is licensed to me (not a lawyer but that’s my understanding). We probably also sign something that allows Apple to do what they will with our images—albeit we retain the copyrights to them. Just to reiterate: the landlord metaphor is referring to software which we lease from Apple—not the device itself. This is yet another case where I want to emphasize the importance of OPEN hardware and software designs. If we truly want ownership, we have to take ownership into our hands (which means a lot of hard work). Most commercial software is licensed so no we don’t own anything in whole that runs commercial software.
nokya 5y ago
A threat model without a discussion on threats. Nice.
rapdangsfamily 5y ago
Look at you being a little fascist cunt once again. Fuck you nigger and your nigger rules.
rapdangsfamily 5y ago
You don't care about the truth cunt. Just your rich friends nigger

325 points by sylens 5y ago

I think this is the first time they have mentioned that you will be able to compare the hash of the database on your device with a hash published in their KB article. They also detailed that the database is only the intersection of hash lists from two child safety organizations under separate governmental jurisdictions. My immediate thought is that this could still be poisoned by Five Eyes participants, and that it does not preclude state actors forcing Apple to replicate this functionality for other purposes (which would leave the integrity of the CSAM database alone, thus not triggering the tripwire).

notJim 5y ago
> this could still be poisoned by Five Eyes participants, and that it does not preclude state actors forcing Apple to replicate this functionality for other purposes The thing is, if this is your threat model you're already screwed. Apple has said they comply with laws in jurisdictions where they operate. The state can pass whatever surveillance laws they want, and I do believe Apple has shown they'll fight them to an extent, but at the end of the day they're not going to shut down the company to protect you. This all seems orthogonal to the CSAM scanning. Additionally, as laid out in the report, the human review process means even if somehow there is a match that isn't CSAM, they don't report it until it has been verified.
vmception 5y ago
the opportunity being to add general functions in photo viewing apps that add a little entropy to every image (for this specific purpose), to rotate hashes, rendering the dual databases useless monetization I guess being to hope for subscribers on github, as this could likely just be a nested dependency that many apps import. a convenient app for this specific purpose might not last long in app stores.
- initplus 5y ago
  Perceptual hashes are specifically designed to be resistant to minor visual alterations.
  - vmception 5y ago
    Let the cat and mouse race begin Any perceptual hash apps to test that theory on?

magicloop 5y ago

I think is this a good document and it also brings to the table the threshold images count (around 30) and the alternate neural hash they will keep private (to guard against adverserial images trying to create a false positive on-device). FWIW, I actually did an amateur threat model analysis in a comment in separate HN thread. I always thought this was called for because the initial document set was just the mathematics, not the people/process/implementation/policy risks and threat model that was the source of widespread concerns.

shuckles 5y ago

It’s pretty lonely over here in technical discussion land. Have we considered Reuters’s intern’s take on this?

dang 5y ago
Please don't post unsubstantive comments yourself. If other threads/comments aren't good, options include posting something better, or not posting. Degrading things even further is not a good choice (edit: particularly when the thread is new—threads are extremely sensitive to initial conditions, and a flamebait/unsubstantive comment early on can have a strongly degrading effect). https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=by%3Adang%20threads%20sensitive%20initial%20conditions&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
- shuckles 5y ago
  Sincere question: why doesn’t this policy apply to the innumerable number of hot takes which are variations on the exact same, often misinformed, reaction? I’m thinking of comments like, “Apple will send me to prison for taking bath tub pics of my infant!” I have seen many on here.
  - dang 5y ago
    Two answers. First, it does apply. If you see a post that ought to have been moderated but hasn't been, the likeliest explanation is that we didn't see it. We don't come close to seeing everything that gets posted here—there is far too much. You can help by flagging it or emailing us at hn@ycombinator.com. Second, there are degrees of these things. It's definitely bad for comments to repeat the same shallow things over and over, but the GP was particularly information-free and snarky, and mucking up this thread while claiming to be speaking in favor of technical discussion is particularly counterproductive. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=likeliest%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...

underseacables 5y ago

What’s not discussed in this Paper is how it won’t be used by third-party actors to falsely accuse someone. Does nobody remember the iCloud hack? What about instead of downloading Jennifer Lawrence’s photos, a hacker uploaded children to get someone falsely accused?

dogma1138 5y ago
Google, Microsoft and a bunch of other services already scan for CSAM materials, yet this hasn’t been an issue so far. In fact I can’t really find a single case when someone was framed like this, despite how easy it is to fill someone’s account with CSAM and call the cops/wait. I don’t like the privacy and property rights issues of this but the whole someone will use this to frame someone is quite BS.
- staticassertion 5y ago
  As I recall someone's neighbor once tried to frame them by downloading CSAM on their network and then calling the police or something like that.
- fuckcensorship 5y ago
  How would someone prove that they were framed by someone planting CSAM? Without evidence, how likely is it that they would be believed?
- underseacables 5y ago
  But the services you mentioned don’t do it on the device. The blind trust in Apple that everything will be ok is quite BS, and very naïve .
- Renaud 5y ago
  1) how would you know someone's being framed if it's not detected as such? in this case it just looks like the target had CSAM on their machine/account and they are found guilty, despite their objections, maybe because no-one really looked closely at the evidence, or it was planted well enough. 2) if it is detected as a plant. Would you know it? Would it fly above our radar? There is evidence this happens [1], some cases are quite high profile [2]. 3) This technique is already in use. Russia is allegedly using it to discredit opponents [3]. Do you think you would hear from it if it was done by secret services of any country to discredit or get rid of foreign targets or even some at home? 4) it's the ultimate weapon against foes and opponents. It's a dangerous one but I have to wonder how many estranged spouses get revenge planting evidence [4], or how many journalists or troublemakers have been the recipient of some carefully crafted planting. I mean, nearly anyone reading HN could do it sufficiently well to get someone in serious trouble, it's not really a technical issue so it's not hard to imagine it's being done. [1]: https://www.independent.co.uk/news/uk/crime/handyman-planted-child-porn-colleague-s-computer-2037439.html https://www.independent.co.uk/news/uk/crime/handyman-planted... [2]: https://www.miaminewtimes.com/news/shaq-hit-with-lawsuit-for-attempting-to-plant-child-porn-on-former-employees-computer-6530360 https://www.miaminewtimes.com/news/shaq-hit-with-lawsuit-for... [3]: https://www.nytimes.com/2016/12/09/world/europe/vladimir-putin-russia-fake-news-hacking-cybersecurity.html https://www.nytimes.com/2016/12/09/world/europe/vladimir-put... [4]: https://www.crimeonline.com/2019/07/04/wife-plants-child-porn-on-husbands-phone-falsely-accuses-him-of-raping-their-daughter-a-judge-gives-her-probation/ https://www.crimeonline.com/2019/07/04/wife-plants-child-por...

tehnub 5y ago

Their use of the phrase "This claim is subject to code inspection by security researchers like all other iOS device-side security claims" stood out to me. Could someone tell me how that inspection works? Are there researchers who are given the source code? (I posted this on another thread [0] earlier, but it's more relevant here) [0]: https://news.ycombinator.com/item?id=28175619 https://news.ycombinator.com/item?id=28175619

skybrian 5y ago
It’s not my field, but my understanding is that security researchers often disassemble and/or debug binaries without having access to the source code. For example, as soon as an OS update is released, people will take it apart. Having source might be nice but it’s not necessary.
- deleted 5y ago
  [deleted]
eivarv 5y ago
Could someone tell me how that inspection works? Are there researchers who are given the source code? It doesn't (with the exception of an exclusive program[0]). Not only do they generally make security work on their devices difficult – they have a history of suing companies that facilitate it [1]. [0]: https://developer.apple.com/programs/security-research-device/ https://developer.apple.com/programs/security-research-devic... [1]: https://www.theverge.com/2021/8/11/22620014/apple-corellium-security-virtual-iphone-dmca-lawsuit-settled https://www.theverge.com/2021/8/11/22620014/apple-corellium-...
kevinsundar 5y ago
Apple actually gives special devices to Security Researchers that allow them further access into the device than a normal consumer device: https://developer.apple.com/programs/security-research-device/ https://developer.apple.com/programs/security-research-devic... In this way, third party security researchers can verify their claims. It actually works out pretty well for them since third party security researchers often find pretty severe vulnerabilities through this program.

almostdigital 5y ago

> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating system that supports the feature. Additionally, users will be able to inspect the root hash of the en- crypted database present on their device, and compare it to the expected root hash in the Knowledge Base article. This is just security theater, they already sign the operating system images where the database reside. And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. > This feature runs exclusively as part of the cloud storage pipeline for images being up- loaded to iCloud Photos and cannot act on any other image content on the device Until a 1-line code change happens that hooks it into UIImage.

kemayo 5y ago
> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust your OS vendor.
- majormajor 5y ago
  If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud). And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc. It's curious that this is the particular breaking point on the slope for people. The "on device" aspect just makes it more immediate feeling, I guess?
  - kemayo 5y ago
    Yeah, it's weird. Speaking purely personally, whether the scanning happens immediately-before-upload on my phone or immediately-after-upload in the cloud doesn't really make a difference to me. But this is clearly not a universal opinion. The most-optimistic take on this I can see is that this program could be the prelude to needing to trust less people. If Apple can turn on e2e encryption for photos, using this program as the PR shield from law enforcement to be able to do it, that'd leave us having to only trust the OS vendor.

rootsudo 5y ago

Truly, 1984. Siri now gives you suggestions if you have dangerous thoughts. "Anonymous helplines and guidance exist for adults with at-risk thoughts and behavior" - "Learn more and get help." Now a fake AI on your phone that knows your actions and behaviors can sleep and wait for specific triggers and give you feedback and alert.

TrumpRapedWomen 5y ago

Why don't they just scan photos service side? Are they trying to save on CPU costs?

jorgesborges 5y ago

I keep changing my mind on this. On the one hand I already operate on the assumption that uploading data to a cloud service renders that data non-private, or at least in great risk of becoming non-private in the future. This simply makes my operating assumption explicit. Also this particular implementation and its stated goals aren’t egregious. But then there’s the slippery slope we’ve all been discussing — and the gradual precedents set for it, this being a nail in the coffin. But I sympathize with Apple for making transparent what I assume happens behind closed doors anyway. My optimistic side thinks maybe this is a sign that extensions of this project will also be made explicit. Maybe this is the only way for them to take this fight if they’re being pressured. In any case now is the time to speak out. If they extend it to texts and other documents I’m out - even as it stands I’m thinking of alternatives. But the fact we’re all discussing it now is at least a point of hope.

naasking 5y ago
> But I sympathize with Apple for making transparent what I assume happens behind closed doors anyway. Using your devices' CPU and battery seems more egregious than doing it on their servers. If they want to help law enforcement, then they should pay for it. Of course they want to help law enforcement by forcing other people to pay the costs. Imagine if Ford came out with a cannabis sensor in their cars that automatically called the cops on you if it detected cannabis inside the car. The sensor is allegedly only active when you're renting a Ford vehicle, not if you purchase it outright. Not a perfect analogy, but how comfortable would you find this situation?
- Spivak 5y ago
  Let's do a more realistic example. Ford introduces a "driver safety" mechanism where the car requires a clean breathalyzer reading in order to start. If it fails it pops up a message that reminds you that drunk driving is illegal but doesn't actually stop you from starting the engine. It then sends Ford the results in an encrypted payload along with 1/30th of the decryption key. After 30 reports someone at Ford opens the payloads, looks at the results and, decides whether to contact the police based on how damning the evidence is. Because all the tests are performed locally you're never giving up any of your privacy unless you actually are driving drunk or are the 1/billion and get 30 false positives. I feel like this is a much stronger than the "if you have nothing to hide" meme because local testing like this lets you can have everything to hide while still revealing specific bad behavior. Like if the TSA had scanners that somehow could only reveal banned items and nothing else I’m not sure I would even consider it a privacy violation.

PaulDavisThe1st 5y ago

> Apple generates the on-device perceptual CSAM hash database through an intersection of hashes provided by at least two child safety organizations operating in separate sovereign jurisdictions – that is, not under the control of the same government. Any perceptual hashes appearing in only one participating child safety organization’s database, or only in databases from multiple agencies in a single sovereign jurisdiction, are discarded by this process, and not included in the encrypted CSAM database that Apple includes in the operating system. Well, that's quite clever, isn't it. It's just the sort of thing that someone who had "never heard of" the Five Eyes, or extradition treaties or illegal rendition operations involving multiple nations, might come up with. Genius!

cmsj 5y ago
If your threat model includes pervasive spying by multiple nation states, and being grabbed in the night by black helicopters, it seems unlikely you'll be overly concerned about them precisely inserting at least 30 of your photos into multiple CSAM databases and also co-ercing Apple's manual review to get you reported to NCMEC.
- sylens 5y ago
  I don't think people are worried about multiple nation states framing them with CSAM photos - they're worried about multiple nation states in an intelligence collaboration poisoning both sets of hash lists with non-CSAM material, so that there is an intersection that makes it onto the device. There is still that Apple human reviewer once the threshold has passed. What I would love to ask Apple is - what happens if/when their reviewers start noticing political material, religious material, etc. is being flagged on a consistent basis, thereby insinuating that the hash list has been poisoned. What's their play at that point?
  - jdlshore 5y ago
    The document states that incorrectly flagged items are forwarded to engineering for analysis. Given their target false-positive rate (1 in 3 trillion, was it?) it seems likely that engineering would very carefully analyze a rush of false positives.

lifty 5y ago

Is there a good argument for doing the scanning on the phone and not on the iCloud servers?

kemayo 5y ago
Well, theoretically this could be a step towards e2e encrypting the photos so that Apple can't see them on their servers. This entire "secure tickets that unlock a derivative of your photos so we can review them" system would, in this interpretation, be there as a way to neuter law enforcement objections[1] to no longer being able to serve warrants for the contents of your photo library. Now, will this actually happen? Hard to say. [1] There was a story last year claiming that Apple wanted to turn on e2e, but the FBI got them to drop it. https://www.popularmechanics.com/technology/security/a30631827/apple-fbi-encryption-whatsapp/ https://www.popularmechanics.com/technology/security/a306318...
himaraya 5y ago
No. Contrary to Craig's claims, Google & Microsoft et al. do similar hash matching on the cloud, instead of 'looking at images'.
- notJim 5y ago
  But people consider that bad! With this system, Apple can never look at your photos in the cloud. They are a pipe for the bits, but they don't examine the contents, except on your device. I'm not saying I necessarily agree that this is better, all-told (mixed feelings), but I think it's worth acknowledging why a rational actor might consider this superior to Google doing whatever the fuck they want with anything you upload.
  - himaraya 5y ago
    I mean, Apple can, as long as iCloud remains unsecured by e2e, the roadmap for which remains MIA.
    - notJim 5y ago

xuki 5y ago

What is the process to make sure the next management will honor these promises?

hypothesis 5y ago
Heck, what is the process for current management to honor them?
- xuki 5y ago
  Weirdly enough, I trust the current management to do the right thing. But that can be changed at a whim, and in my opinion, the greatest security threat to this whole thing.
  - himaraya 5y ago
    I mean, I trusted current management not to pull stunts like this to begin with.
notJim 5y ago
Apple management changing would be a massive threat regardless of these premises. Apple has gone to bat for encryption and privacy to a greater degree than Google, for example, and that's because of their current management. You should not in any way assume that the next group will do the same!

deleted 5y ago

[deleted]

cmsj 5y ago

This seems like a pretty helpful document, and really shows how carefully the CSAM system has been designed. Two things that stuck out to me: 1) The CSAM hash database is encrypted on-device 2) The device doesn't know the results of hash comparisons These suggest to me that the hash comparison is going to happen inside the Secure Enclave - give it the encrypted database and the NeuralHash of an image, it gives you back a Security Voucher to upload. I don't quite get how the hash database can stay secret though - the decryption key has to come from somewhere. AFAIK the hash database has never been made publicly available before, which probably makes it quite a high value target for security researchers and curious parties.

shuckles 5y ago
The hash database is blinded on the server before being embedded into iOS. You’d need to access Apple’s private key or break cryptography to get access to it from the client. For the less cryptographically savvy, the “Technical Summary” is a good document on their Child Safety page.
FabHK 5y ago
> These suggest to me that the hash comparison is going to happen inside the Secure Enclave - give it the encrypted database and the NeuralHash of an image, it gives you back a Security Voucher to upload. My understanding is that the hash comparison, or at any rate the counting of matches, is done in the cloud then. The way the crypto is set up is that the server has no access to the security envelope and its contents (and, after E2EE in the future, to any images) unless the number of matches exceeds the threshold.
nullc 5y ago
No secure enclave is needed, the system uses strong cryptography to protect Apple and their database providers from accountability. I gave a semi-technical description of how private set intersection works here: https://news.ycombinator.com/item?id=28124716 https://news.ycombinator.com/item?id=28124716 Assuming they implement it correctly and there are no apple-internal leaks we won't be discovering the content of the database.
stjohnswarts 5y ago

ty___ler 5y ago

If apple can be coerced by governments to scan photos on the serverside then what is the operational purpose of a frontend scan, given what apple is publicly saying they are trying to do? This is the most confusing aspect for me.

codeecan 5y ago
Apple will turn it on eventually for non iCloud destined photos (and documents), you don't build a system like this not to use it. We know Apple concedes to China's demands, and with another Snowden situation, the US would not hesitate to add classified documents to the scan list and identify individuals. This system will catch no abusers, because they're not Airdropping photos to each other. And if they're smart enough to use a VPN, they're not storing that on a phone thats logged into the cloud. And if they're not using a VPN, they can be caught at download time.

concinds 5y ago

I strongly considered switching away from Apple products last weekend; but this document has convinced me otherwise. The threats people identify have minimal risk. If a total stranger offers you a bottle of water, you may worry about it being spiked, but him having offered the bottle doesn't make it more, or less, likely that he'll stab you after you accept it. They're separate events, no "slippery slope". It's very convincing that any alteration that would "scan" your whole phone would be caught eventually, and even if it's not, the announcement of this feature has no bearing on it. If Apple has evil intent, or is being coerced by NSLs, they would (be forced to) implement the dangerous mechanism whether this Child Safety feature existed or not. The concept of backdoors is hardly foreign to governments, and Apple didn't let any cats out of any bags. This document shows that Apple did go to great lengths to preserve privacy; including the necessity of hashes being verified in two juristictions, the fact that neither the phone nor Apple know if any images have been matched below the threshold; the lack of remote updates of this mechanism; the use of vouchers instead of sending the full image; the use of synthetic vouchers; and on and on. Furthermore, the risks of the risks of this mechanism are lower than the existing PhotoDNA used by practically every competing service. Those have no thresholds; the human review process is obscure; there is no claim that other pictures won't be looked at. The controversy, the fact that it uses an on-device component. But PhotoDNA-solutions fail many of Apple's design criterias, which require an on-device solution: - database update transparency - matching software correctness - matching software transparency - database and software universality - data access restrictions. What about the concerns that the hash databases could be altered to contain political images? PhotoDNA could be, too; but would be undetectable unlike with Apple's solutions. Worse: with serverside solutions, the server admin could use an altered hash DB only for specific target users, causing harm without arousing suspicion. Apple's design prevents this since the DB is verifiably universal to all users. A rational look at every argument I've seen against Apple's solution indicates that it is strictly superior to current solutions, and less of a threat to users. Cryptographically generating vouchers and comparing hashes is categorically not "scanning people's phones" or a "backdoor." I think the more people understand its technical underpinnings, the less they'll see similarities with sinister dystopian cyberpunk aesthetics.

Syonyk 5y ago

drodgers 5y ago

> The second protection [against mis-inclusion of non CSAM hashes] is human review: there is no automated reporting in Apple’s system. All positive matches must be visually confirmed by Apple as containing CSAM before Apple will disable the account and file a report with the child safety organization. I don't understand this at all. As I understand it, part of the problem is that — in the US — Apple isn't legally allowed to review, transmit or do anything else with suspected CSAM images, so they can't have a manual review process (or even check that their neural hashing is working as expected on the real dataset). Does anyone else have any idea of what this is trying to describe? If Apple really are somehow reviewing flagged photos to confirm that they're CSAM and not maliciously flagged files before sending any reports, then that does make the system substantially more resilient to Five Eyes abuse (not that I wish that job on anyone). Edit: There's more context later in the document > First, as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possibility that the match threshold was exceeded due to non-CSAM images that were adversarially perturbed to cause false NeuralHash matches against the on-device encrypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation. This is more confusing: visually comparison using a second perceptual hash doesn't actually to provide any protection against mis-inclusion of non-CSAM images: it just double-checks that the image really was a match in the database (ie. protects against hash-collision errors), but it does't check that the database contained actual CSAM. Apple explicitly says that this process protects against mis-inclusion though, which doesn't make sense to me yet.

shuckles 5y ago
Your understanding is incorrect. Apple can, and is in fact required to, verify that they have actual CSAM before forwarding it to the Cyber Tip line. At that point, they must delete the information within 60 days.
- drodgers 5y ago
  Interesting. In that case, do you know why they talk about reviews only seeing "visual derivatives" (from the second perceptual hash)? Either these 'derivatives' basically contain the original image (so reviewers can verify that it's actual CSAM) and there's no point in using derivatives at all, or they're more abstract (eg. 8x8 pixellated images) in which case the reviewer can't see the actual content (but could confirm a database match). Edit: I was able to find the answer, they suggest that the 'visual derivative' is something like a 'low-resolution version' of the original image, so the content should still be clearly visible.

chipotle_coyote 5y ago

In other HN comments on this subject I've (hopefully) made it clear that I'm not really in favor of this project of Apple's, and that there's a legitimate "slippery slope" argument to be made here. So I hope people will entertain a contrarian question without downvoting me into oblivion. :) Here's the thing I keep circling around: assume that bad actors, government or otherwise, want to target political dissidents using internet-enabled smartphones. The more we learn about the way Apple actually implemented this technology, the less likely it seems that it would make it radically easier for those bad actors to do so. For instance, the "it only scans photos uploaded to iCloud" element isn't just an arbitrary limitation that can be flipped with one line of code, as some folks seem to think; as Erik Neuenschwander, head of Privacy Engineering at Apple, explained in an interview on TechCrunch[1]: > Our system involves both an on-device component where the voucher is created, but nothing is learned, and a server-side component, which is where that voucher is sent along with data coming to Apple service and processed across the account to learn if there are collections of illegal CSAM. That means that it is a service feature. Will this stop those bad actors if they're determined? No, of course not, but there are so many ways they can do it already. They'll get cloud storage providers to give them access. If they can't, they'll get network providers to give them access. And if those bad actors are, as many people fear, government actors, then they have tools far more potent than code: they have laws. It was illegal to export "strong encryption" for many years, remember? I've seen multiple reports that European lawmakers are planning to require some kind of scanning for CSAM. If this goes into effect, technology isn't going to block those laws for you. Your Purism phone will either be forced to comply or be illegal. I wrote in a previous comment on this that one of Silicon Valley's original sins is that we tend to treat all problems as if they're engineering problems. Apple is treating CSAM as an engineering problem. Most of the discussion on HN about how horrible and wrong Apple is here still treats it as an engineering problem, though: well, you can get around this by just turning off iCloud Photos or never using Apple software or throwing your iPhone in the nearest lake and switching to Android, but only the right kind of Android, or maybe just never doing anything with computers again, which admittedly will probably be effective. Yet at the end of the day, this isn't an engineering problem. It's a policy problem. It's a governance problem. In the long run, we solve this, at least in liberal democracies, by voting people into office who understand technology, understand the value of personal encryption, and last but certainly not least, understand the value of, well, liberal democracy. I know that's easy to dismiss as Pollyannaism, but "we need to protect ourselves from our own government" has a pretty dismal track record historically. The entire point of having a liberal democracy is that we are the government, and we can pull it back from authoritarianism. The one thing that Apple is absolutely right about is that expanding what those hashes check for is a policy decision. Maybe where those hashes get checked isn't really what we need to be arguing about. [1]: https://techcrunch.com/2021/08/10/interview-apples-head-of-privacy-details-child-abuse-detection-and-messages-safety-features/ https://techcrunch.com/2021/08/10/interview-apples-head-of-p...

616c 5y ago

I have long been interested in what a professional-grade threat model from a large FAANG/SV organization is. Is this a representative model? Microsoft came up with DREAD and STRIDE and they suggest there threat models are more elaborate. Would love to see more representative examples!

jdlshore 5y ago
This feels more like a marketing document to me than a workaday threat model. It's fairly handwavey, and the goal seems to be to convince rather than to do a hardnosed analysis. Not that it's not useful—I found it convincing—but I doubt this is what a real threat model looks like. Not that I actually know. I'd be interested in seeing a real one too.

hendersoon 5y ago

I have no doubt the features work exactly the way Apple said they do. Seriously. I very much doubt that they will refuse to scan for whatever China asks. I very much doubt they'll risk the PRC shutting down Foxconn factories. I very much doubt Apple will ultimately be able to resist scanning for whatever the US Government asks in a national security letter attached to a gag order. They will take them to a secret court session which we'll never hear about, the court will rule in the government's favor, and Apple will be forced to comply and not allowed to tell anybody. This happened to Cloudflare and they were gagged while fighting it for 4 years. They didn't win in court, but the government eventually dropped the case as it was no longer needed. And this started under Obama, not Trump. Now, will Apple bother to fight this out in court? I think they probably will. But we won't know about that until they either definitively win or the government gives up. If the government wins, on the other hand, we'll NEVER hear about it.

heavyset_go 5y ago
> Now, will Apple bother to fight this out in court? I think they probably will. Why would they? Each year they give up customer data on over 150,000 users without a fight when the US government simply asks them for it[1]. [1] https://www.apple.com/legal/transparency/us.html https://www.apple.com/legal/transparency/us.html

throwawaymanbot 5y ago

Chinafy the population.

semerda 5y ago

“Human review and reporting” - fact checkers? Or are we to assume that their AI is 100% effective. This will not end well..

Jerry2 5y ago

What strikes me about this paper is that there are no names of the people who wrote it on the title page. Why were they afraid to put their name(s) on this?

celeritascelery 5y ago
Seeing HN over the last week, I can think of a few reasons…
- stjohnswarts 5y ago
  Yeah seeing as its just a white paper there really isn't much need, and the possibility of retribution is immense on this one. I know a lot of pissed off people, let alone some of the nuts who this is angering and who could dox or do worse to the authors.

gandalfgeek 5y ago

Sounds like it still comes down to trust. Quoting from the paper: "Apple will refuse all requests to add non-CSAM images to the perceptual CSAM hash database; third party auditors can confirm this through the process outlined before. Apple will also refuse all requests to instruct human reviewers to file reports for anything other than CSAM materials for accounts that exceed the match threshold."

celeritascelery 5y ago
To me, the fact that they are making such a strong statement is a big deal. If they had plans or thought that they would be forced to add other content they would not be so direct in this statement. I hope I am not wrong. As you said: trust.
0xy 5y ago
It's already a lie, considering NCMEC's database already has non-CSAM images. It won't ever be true, since the hundreds and possibly thousands of entities with direct and indirect access to the database can upload SunnyMeadow.jpg labeled as CSAM content, and it's blindly accepted. If Apple is receiving a hash labeled as "CSAM", how can they possibly make this guarantee? They do not know it's CSAM. It's unverifiable and comes down to "trust us". Remember, NCMEC is an unaccountable organisation inextricably linked with the FBI and the U.S. government. It is not subject to FOIA requests. Its database has never been audited, and it is known to be a mess already.
- simondotau 5y ago
  > "hundreds and possibly thousands of entities with direct and indirect access to the database can upload SunnyMeadow.jpg labeled as CSAM content, and it's blindly accepted." According to whom? > "it is known to be a mess already." According to whom? You've posted a number of claims about the CSAM database administration process on HN and the one instance where you've cited a source, it's been a junk source.

makecheck 5y ago

I don’t like the idea of stuff running on my device, consuming my battery and data, when the only point is to see if I am doing something wrong? An analogy I can come up with is: the government hires people to visit your house every day, and while they’re there they need your resources (say, food, water, and electricity). In other words, they use up some of the stuff you would otherwise be able to use only for yourself — constantly — and the only reason they’re there is to see if you’re doing anything wrong. Why would you put up with this?

shuckles 5y ago
The point of it is to make sure iCloud Photos remains a viable service in light of real and perceived regulatory threats, and possibly leave the door open to end to end encryption in the future.
- gpm 5y ago
  There is no regulatory threat within the US that could require this happen, if this was demanded by the government it would be a blatant violation of the 4th amendment. Apple should have stood their ground if this was in response to perceived government pressure.
  - dwaite 5y ago
    IMHO, the relevant regulations are pretty carefully worded to make 4th amendment defenses harder. In this case, Apple has some liability for criminal activity on their platform whether it is was E2E encrypted or not.
  - matwood 5y ago
    There are two gov. issues. One is if the FBI comes knocking and the other is new laws. The government could absolutely write a law banning e2ee so that when the FBI does knock with a warrant they can get access. In the past, Apple has done what they can to stand their ground against the first, but they (like any other company) will have to comply with any laws passed. Whether the 'if a warrant is obtained the gov. should be granted access' law is a violation of the 4th amendment remains to be seen.

1f60c 5y ago

Having read the document, I am even more confident in Apple's approach than before. I commend Apple for its transparency.

bnj 5y ago

A curious outcome of the scrutiny that the system design is receiving is that it will make it less effective at the task of catching actual collections of CSAM. Page eleven promises something I haven’t seen before: that Apple will publish the match threshold for each version. Meaning, even if after all this, there are abusers still using iCloud photos to manage their illegal collection of CSAM, they can check back to know what number of images is safe to keep. A twisted version of the Streisand effect.

somebodythere 5y ago
If you knew about the existence of this program and were determined to keep a collection of CSAM on your phone I don't understand why someone would choose to dance around specific thresholds rather than just disable iCloud sync.

m3kw9 5y ago

Setting arguments aside, if ever there is a CSAM scanner I’d let loose in my account is Apples.

deleted 5y ago

[deleted]

swiley 5y ago

Burn it all down. If you can't understand the theory and reimplement it then you shouldn't even look at it. The internet is full of pathological wizards and witches. Come here for obscure books but otherwise stay far away.

unityByFreedom 5y ago

> The system is designed so that a user need not trust Apple, any other single entity, or even any set of possibly-colluding entities from the same sovereign jurisdiction (that is, under the control of the same government) Who's to say authoritarian governments don't team up? We just saw this happen over the last decade with the rise of nationalism everywhere. We already know governments can put significant pressure on private companies. Don't capitulate, Apple. Blow the whistle. You're getting screwed and you have our support if you just stand up for what you said 5 years ago.

zepto 5y ago

A key point that needs to be mentioned: we strongly dislike being distrusted. It might well be a genetic heritage. Being trusted in a tribe is crucial to survival, and so is likely wired deep into our social psychology. Apple is making a mistake by ignoring that. This isn’t about people not trusting Apple. It’s about people not feeling trusted by Apple. Because of this, it doesn’t matter how trustworthy the system is or what they do to make it less abusable. It will still represent distrust of the end user, and people will still feel that in their bones. People argue about the App Store and not being trusted to install their own Apps etc. That isn’t the same. We all know we are fallible and a lot of people like the protection of the store, and having someone to ‘look after’ them. This is different and deeper than that. Nobody want to be a suspect for something they know they aren’t doing. It feels dirty.

shuckles 5y ago
But the catch is: all the incumbents already treated your data as if you were guilty until proven innocent. Apple’s transparency about that change may have led people to internalize that, but it’s been the de facto terms of most cloud relationships. What I personally don’t understand is why Apple didn’t come out with a different message: we’ve made your iPhone so secure that we’ll let it vouch for your behalf when it sends us data to store. We don’t want to see the data, and we won’t see any of it unless we find that lots of the photos you send us are fishy. Android could never contemplate this because they can’t trust the OS to send vouchers for the same photos it uploads, so instead they snoop on everything you send them. It seems like a much more win-win framing that emphasizes their strengths.
- zepto 5y ago
  I agree this is vastly better than what anyone else is doing, and you know I understand the technology. However, I don’t think any framing would have improved things. I think it was always going to feel wrong. I would prefer they don’t do this because it feels bad to be a suspect even in this abstract and in-practice harmless way. Having said that, having heard from people who have investigated how bad pedophile activity actually is, I can imagine being easily persuaded back in the other direction. I think thins is about the logic of evolutionary psychology, not the logic of cryptography. My guess is that between now and the October iPhone release we are going to see more media about the extent of the problem they are trying to solve. That is how Apple wins this.

simondotau 5y ago

Here's what I don't get: who is importing CSAM into their camera roll in the first place? I for one have never felt the urge to import regular, legal porn into my camera roll. Who the hell is going to be doing that with stuff they know will land them in prison? Who the hell co-mingles their deepest darkest dirtiest secret amongst pictures of their family and last night’s dinner? I can believe that some people might be so technically illiterate that they'd upload CSAM to a private Facebook group thinking it was secure. But as far as I'm aware it's not possible to password protect photo albums using the built-in Photos app.

simondotau 5y ago
Actually it occurs to me that perhaps they're noticing a lot of CSAM is being produced with smartphone cameras and are hoping to snag the phone which produced the originals. If they can get new content into their database before the photographer deletes them, they might find the phone which took the originals—and then find the child victim. Beyond implausible, but then most law enforcement tends to rely on someone eventually doing something stupid.
- phnofive 5y ago
  It is only supposed to detect CSAM already known to NCMEC, not identify new images.
  - simondotau 5y ago
    That's my point. If the original photographer doesn't delete them by the time they're known to NCMEC and the hash database is updated, they would match. As I said, I recognise this is beyond implausible.
  - 0xy 5y ago
    NCMEC's database does not only contain CSAM. It has never been audited. It's full of false positives. They are immune from FOIA requests. They are accountable to nobody. They work so closely with the FBI that there are FBI agents working on the database directly. NCMEC is essentially an unaccountable, unauditable department of the FBI that also happens to be incompetent (the amount of non-CSAM in the database is large).

fortran77 5y ago

Why don't they scan files as they're downloaded, the way we scan for malware and viruses on PCs? This way, we can warn people _before_ a law is broken!

akomtu 5y ago

This is Apple's trying to shift the focus onto technicalities. Remotely monitored CCTV cams in your house is never a good idea, no matter what the treat model is.

lucasyvas 5y ago

If they continue down this road, they will not only have effectively created the smartphone category, but also coincidentally destroyed it. If our devices are designed to spy on us, we're frankly not even going to use them anymore. I wonder if they forgot that using this thing is optional? You'll see a resurgence of single purpose dumb devices.

istingray 5y ago

I just found the perfect video clip to represent Apple's response to launching this feature: https://youtu.be/-7XchZICFQU?t=17 https://youtu.be/-7XchZICFQU?t=17

makerofthings 5y ago

My phone is an extension of my brain, it is my most trusted companion. It holds my passwords, my mail, my messages, my photos, my plans and notes, it holds the keys to my bank accounts and my investments. I sleep with it by my bed and I carry it around every day. It is my partner in crime. Now apple are telling me that my trusted companion is scanning my photos as they are uploaded to iCloud, looking for evidence that apple can pass to the authorities? They made my phone a snitch? This isn't about security or privacy, I don't care about encryption or hashes here, this is about trust. If I can't trust my phone, I can't use it.

zabatuvajdka 5y ago
Technically they could have been scanning the photos already to power some AI algorithms or whatever else. I think this is a nudge for folks to wake up and see the reality of what it means to use the cloud. We are leasing storage space from Apple in this case. Technically, it’s no different than a landlord checking up on their tenants to make sure “everything is okay.” And technically, if you do not like iCloud, don’t use it and roll your own custom cloud storage! After all, it’s redundancy and access the cloud provides. And Apple provides APIs to build them. Hell, with the new Files app on iOS i just use a Samba share with my NAS server (just a custom built desktop with RAID 5 and Ubuntu.)
- a012 5y ago
  > Technically, it’s no different than a landlord checking up on their tenants to make sure “everything is okay.” By this logic, you don't own your phone but rent it from Apple?
  - zabatuvajdka 5y ago
    Technically I own my iPhone but the software is licensed to me (not a lawyer but that’s my understanding). We probably also sign something that allows Apple to do what they will with our images—albeit we retain the copyrights to them. Just to reiterate: the landlord metaphor is referring to software which we lease from Apple—not the device itself. This is yet another case where I want to emphasize the importance of OPEN hardware and software designs. If we truly want ownership, we have to take ownership into our hands (which means a lot of hard work). Most commercial software is licensed so no we don’t own anything in whole that runs commercial software.

nokya 5y ago

A threat model without a discussion on threats. Nice.

rapdangsfamily 5y ago

Look at you being a little fascist cunt once again. Fuck you nigger and your nigger rules.

rapdangsfamily 5y ago

You don't care about the truth cunt. Just your rich friends nigger