8 ms·

> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating sys

> Apple will publish a Knowledge Base article containing a root hash of the encrypted CSAM hash database included with each version of every Apple operating system that supports the feature. Additionally, users will be able to inspect the root hash of the en- crypted database present on their device, and compare it to the expected root hash in the Knowledge Base article. This is just security theater, they already sign the operating system images where the database reside. And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. > This feature runs exclusively as part of the cloud storage pipeline for images being up- loaded to iCloud Photos and cannot act on any other image content on the device Until a 1-line code change happens that hooks it into UIImage.

kemayo 5y ago
> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust your OS vendor.
- majormajor 5y ago
  If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud). And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc. It's curious that this is the particular breaking point on the slope for people. The "on device" aspect just makes it more immediate feeling, I guess?
  - kemayo 5y ago
    Yeah, it's weird. Speaking purely personally, whether the scanning happens immediately-before-upload on my phone or immediately-after-upload in the cloud doesn't really make a difference to me. But this is clearly not a universal opinion. The most-optimistic take on this I can see is that this program could be the prelude to needing to trust less people. If Apple can turn on e2e encryption for photos, using this program as the PR shield from law enforcement to be able to do it, that'd leave us having to only trust the OS vendor.
    - notJim 5y ago
      > Speaking purely personally, whether the scanning happens immediately-before-upload on my phone or immediately-after-upload in the cloud doesn't really make a difference to me. What I find interesting is that so many people find it worse to do it on device, because of the risk that they do it to photos you don't intend to upload. This is clearly where Apple got caught off-guard, because to them, on-device = private. It seems like the issue is really the mixing of on-device and off. People seem to be fine with on-device data that stays on-device, and relatively fine with the idea that Apple gets your content if you upload it to them. But when they analyze the data on-device, and then upload the results to the cloud, that really gets people.
      - shuckles 5y ago
        This seems like a necessary discussion to have in preparation for widespread, default end to end encryption.
      - rustymonday 5y ago
        Is this really surprising to you? I'm not trying to be rude, but this is an enormous distinction. In today's world, smartphones are basically an appendage of your body. They should not work to potentially incriminate its owner.
        alwillis 5y ago
        They should not work to potentially incriminate its owner. But that ship has long sailed, right? Every packet that leaves a device potentially incriminates its owner. Every access point and router is a potential capture point.
  - userbinator 5y ago
    Yes, you had to trust Apple, but the huge difference with this new thing is that hiding behind CSAM gives them far more (legally obligated, in fact --- because showing you the images those hashes came from would be illegal) plausible deniability and difficulty of verifying their claims. In other words, extracting the code and analysing it to determine that it does do what you expect is, although not easy, still legal. But the source, the CSAM itself, is illegal to possess, so you can't do that verification much less publish the results. It is this effective legal moat around those questioning the ultimate targets of this system which people are worried about.
    - majormajor 5y ago
      Surely they could do their image matching against all photos in iCloud without telling you in advance, and then you'd be in exactly the same boat? Google was doing this for email as early as 2014, for instance, with the same concerns about its extensibility raised by the ACLU: https://www.theguardian.com/technology/2014/aug/04/google-child-abuse-ncmec-internet-watch-gmail https://www.theguardian.com/technology/2014/aug/04/google-ch... So in a world where Apple pushes you to set up icloud photos by default, and can do whatever they want there, and other platforms have been doing this sort of of thing for years, it's a bit startling that "on device before you upload" vs "on uploaded content" triggers far more discontent? Maybe it's that Apple announced it at all, vs doing it relatively silently like the others? Apple has always had access to every photo on your device, after all.
      - pseudalopex 5y ago
        It isn't startling people trust they can opt out of iCloud photos.
        kemayo 5y ago
        If you trust that you can opt out of iCloud Photos to avoid server-side scanning, trusting that this on-device scanning only happens as part of the iCloud Photos upload process (with the only way it submits the reports being as metadata attached to the photo-upload, as far as I can tell) seems equivalent. There's certainly a slippery-slope argument, where some future update might change that scanning behavior. But the system-as-currently-presented seems similarly trustable.
  - himaraya 5y ago
    No, the threat model differs entirely. Local scanning introduces a whole host of single points of failure, including the 'independent auditor' & involuntary scans, that risk the privacy & security of all local files on a device. Cloud scanning largely precludes these potential vulnerabilities.
    - majormajor 5y ago
      Your phone threat model should already include "the OS author has full access to do whatever they want to whatever data is on my phone, and can change what they do any time they push out an update." I don't think anyone's necessarily being too upset or paranoid about THIS, but maybe everyone should also be a little less trusting of every closed OS - macOS, Windows, Android as provided by Google - that has root access too.
      - himaraya 5y ago
        Sure, but that doesn't change the fact that the vulnerabilities with local scanning remain a significant superset of cloud scanning's. Apple has built iOS off user trust & goodwill, unlike most other OSes.
        shuckles 5y ago
        Cloud Scanning vulnerability: no transparency over data use. On the phone, you can always confirm the contents of what’s added to the safety voucher’s associated data. On the cloud, anything about your photos is fair game. Where does that fit in your set intersection?
        himaraya 5y ago
        > On the phone, you can always confirm the contents of what’s added to the safety voucher’s associated data. ...except you can't? Not sure where these assumptions come from.
        5y ago
- almostdigital 5y ago
  Yeah that's true, although to do some sort of mass scanning stealthily they would need a system exactly like what they built with this, if they tried to upload everything for scanning the data use would be enormous and give it away. I guess it comes down to that I don't trust an OS vendor that ships an A.I. based snitch program that they promise will be dormant.
  - kemayo 5y ago
    Speaking cynically, I think that them having announced this program like they did makes it less likely that they have any sort of nefarious plans for it. There's a lot of attention being paid to it now, and it's on everyone's radar going forwards. If they actually wanted to be sneaky, we wouldn't have known about this for ages.
    - theonlybutlet 5y ago
      They'd have to be transparent about it as someone would easily figure it out.You have no way of verifying the contents of that hash database. once the infrastructure is in place (i.e. on your phone) it's a lot easier to expand on it. People have short memories and are easily desensitized, after a year or two of this, everyone will forget and we'll be in uproar about it expanding to include this or that...
    - smichel17 5y ago
      You're making the mistake of anthropomorphizing a corporation. Past a certain size, corporations start behaving less like people and more like computers, or maybe profit-maximizing sociopaths. The intent doesn't matter, because 5 or 10 years down the line, it'll likely be a totally different set of people making the decision. If you want to predict a corporation's behavior, you need to look at the constants (or at least, slower-changing things), like incentives, legal/technical limitations, and internal culture/structure of decision-making (e.g. How much agency do individual humans have?).
      - kemayo 5y ago
        I feel that I was stating the incentives, though. This being an area people are paying attention to makes it less likely they'll do unpopular things involving it, from a pure "we like good PR and profits" standpoint. They might sneak these things in elsewhere, but this specific on-device-scanning program has been shown to be a risk even at its current anodyne level.
- slg 5y ago
  Which is why I am confused by a lot of this backlash. Apple already controls the hardware, software, and services. I don't see why it really matters where in that chain the scanning is done when they control the entire system. If Apple can't be trusted with this control today, why did people trust them with this control a week ago?
  - zwily 5y ago
    That’s where I’m at. They could have just started doing this without even saying anything at all.
    - jimworm 5y ago
      But in that case they would eventually be caught red-handed and won't get to do the "for the children" spiel and get it swept under the rug like it's about to be.
      - bredren 5y ago
        The goal is not for it to be swept under the rug. The goal is for it to deflect concerns over the coming Privacy Relay service.
        threatofrain 5y ago
        Their private relay service appears orthogonal to CSAM… it won’t make criminals and child abusers easier or harder to catch, and it doesn’t affect how people use their iCloud Photos storage.
        bredren 5y ago
        These people are commonly prosecuted using evidence that includes server logs that showing their static IP address. Read evidence from past trials it is obvious. See also successful and failed attempts to subpoena this info from VPN services. Only people with iCloud will be using the relay. It is true on the surface the photos is disconnected from the use. However, Apple only needs a solid answer that handles the bad optics of what you can do with the Tor-like anonymity of iCloud Privacy Relay. However, if you look more closely, the CSAM service and its implementation are crafted exactly around the introduction of the relay.
- g42gregory 5y ago
  > At some point you have to trust your OS vendor. Yes, and we were trusting Apple. And now this trust is going away.
  - rootusrootus 5y ago
    > now this trust is going away Is it really? There are some very loud voices making their discontent felt. But what does the Venn diagram look like between 'people who are loudly condemning Apple for this' and 'people who were vehemently anti-Apple to begin with'? My trust was shaken a bit, but the more I hear about the technology they've implemented, the more comfortable I am with it. And frankly, I'm far more worried about gov't policy than I am about the technical details. We can't fix policy with tech.
    - matwood 5y ago
      > I'm far more worried about gov't policy than I am about the technical details. We can't fix policy with tech. Yeah. I don't really understand the tech utopia feeling that Apple could simply turn on e2ee and ignore any future legislation to ban e2ee. The policy winds are clearly blowing towards limiting encryption in some fashion. Maybe this whole event will get people to pay more attention to policy...maybe.
- Veserv 5y ago
  Yes, they can technically already do so, but that is not the question. The question is what can they legally do and justify with high confidence in the event of a legal challenge. Changes to binding contractual terms that allow broad readings and provide legal justification for future overreach are dangerous. If they really are serious that they are going to use these new features in a highly limited way then they can put their money where their mouth is and add legally binding contractual terms that limit what they can do with serious consequences if they are found to be in breach. Non-binding marketing PR assurances that they will not abuse their contractually justified powers are no substitute for the iron fist of legal penalty clause.
- m4rtink 5y ago
  What about trust-but-verify ? If the OS was open source and supported reproducible builds, you would not have to trust them, you could verify what it actually does & make sure the signed binaries they ship you actually correspond to the source code. Once kinda wonders what they want to hide if they talks so much about user privacy yet don't provide any means for users to verify their claims.
- ianmiers 5y ago
  What happens if someone tries to coerce Apple into writing backdoor code? Engineers at Apple could resist, resign, slow roll the design and engineering process. They could leak it and it would get killed. Things would have to get very very bad for that kind of pressure to work. On the other hand, once Apple has written a backdoor enthusiastically themselves, it's a lot easier to force someone to change how it can be used. The changes are small and compliance can be immediately verified and refusal punished. To take it to its logical extreme: you cannot really fire or execute people who delay something (especially if you lake the expertise to tell how long it should take). But you can fire or execute people who refuse to flip a switch. This technology deeply erodes Apple and its engineers' ability to resist future pressure. And the important bit here is there adversary isn't all powerful. It can coerce you to do things in secret, but its power isn't unlimited. See what happened with yahoo.[0] https://www.reuters.com/article/us-yahoo-nsa-exclusive/exclusive-yahoo-secretly-scanned-customer-emails-for-u-s-intelligence-sources-idUSKCN1241YT https://www.reuters.com/article/us-yahoo-nsa-exclusive/exclu...
jolux 5y ago
> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. They describe a process for third parties to audit that the database was produced correctly.
- LexGray 5y ago
  Do we have any idea how the NCMEC database is curated? Are there cartoons from Hustler depicting underage girls in distress? Green text stories stating they are true about illegal sexual acts? CGI images of pre-pubescent looking mythical creatures? Manga/Anime images which are sold on the Apple Store? Legitimate artistic images from books currently sold? Images of Winnie the Pooh the government has declared pornographic? From the amount of material the Feds claim is being generated every year I would have to guess all of this is included. The multi-government clause is completely pointless with the five-eyes cooperation. The story here is that there is a black box of pictures. Apple will then use their own black box of undeclared rules to pass things along to the feds which they have not shared what would be considered offending in any way shape or form other than "we will know it when we see it". Part of the issue here is that Apple is taking the role of a moral authority. Traditionally Apple has been incredibly anti-pornography and I suspect that anything that managed to get into the database will be something Apple will just pass along.
  - jolux 5y ago
    Apple is manually reviewing every case to ensure it’s CSAM. You do have to trust them on that. But if your problem is with NCMEC, you’ve got a problem with Facebook and Google who are already doing this too. And you can’t go to jail for possessing adult pornography. So even if you assume adult porn images are in the database, and Apple’s reviewers decide to forward them to NCMEC, you would still not be able to be prosecuted, at least in the US. Ditto for pictures of Winnie the Pooh. But for the rest of what you describe, simulated child pornography is already legally dicey as far as I know, so you can’t really blame Apple or NCMEC for that.
    - LexGray 5y ago
shuckles 5y ago
Apple shipped iCloud Private Relay which is a “1-line code change that hooks into CFNetwork” away from MITMing all your network connections, by this standard.
- almostdigital 5y ago
  For me the standard is that I don't want any 1-line code change between me and near-perfect Orwellian surveillance.
  - shuckles 5y ago
    Since your one-liners seem to be immensely dense with functional changes, I can’t understand how you trust any software.
- ec109685 5y ago
  Any connection worth its salt should be TLS protected.
  - shuckles 5y ago
    Also in CFNetwork. Probably a one line change to replace all session keys with an Apple generated symmetric key.
notJim 5y ago
> Until a 1-line code change happens that hooks it into UIImage. I really don't understand this view. You are using proprietary software, you are always an N-line change away from someone doing something you don't like. This situation doesn't change this. If you only use open source software and advocate for others to do the same, I would understand it more.
- almostdigital 5y ago
  > I really don't understand this view. You are using proprietary software, you are always an N-line change away from someone doing something you don't like. This situation doesn't change this. And I don't understand why it has to be black and white, I think the N is very important in this formula and if it is low that is a cause for concern. Like an enemy building a missile silo on an island just off your coast but promising it's just for defense. All arguments I see is along the lines of "Apple can technically do anything they want anyways so this doesn't matter". But maybe you're right and moving to FOSS is the only solution long-term, that's what I'm doing if Apple goes through with this.
  - notJim 5y ago
    The size of N doesn't really matter. I'm sure Apple ships large PRs in every release, as any software company does.
    - almostdigital 5y ago
      Maybe not if you assume Apple is evil but for the case of Apple being good intentioned but having its hand forced, they will have a much harder time resisting a 1 line change than a mandate to spend years to develop a surveillance system
  - fay59 5y ago
    I’d leave this one out to the lawyers. I’m not one but I don’t think that the court will evaluate the number of lines of code required for help.
- 5y ago

kemayo 5y ago

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that. At some point you have to trust your OS vendor.

majormajor 5y ago
If you're uploading to the cloud, you have to trust a lot more than just your OS vendor (well, in the default case, your OS vendor often == your cloud vendor, but the access is a lot greater once the data is on the cloud). And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc. It's curious that this is the particular breaking point on the slope for people. The "on device" aspect just makes it more immediate feeling, I guess?
- kemayo 5y ago
  Yeah, it's weird. Speaking purely personally, whether the scanning happens immediately-before-upload on my phone or immediately-after-upload in the cloud doesn't really make a difference to me. But this is clearly not a universal opinion. The most-optimistic take on this I can see is that this program could be the prelude to needing to trust less people. If Apple can turn on e2e encryption for photos, using this program as the PR shield from law enforcement to be able to do it, that'd leave us having to only trust the OS vendor.
  - notJim 5y ago
    > Speaking purely personally, whether the scanning happens immediately-before-upload on my phone or immediately-after-upload in the cloud doesn't really make a difference to me. What I find interesting is that so many people find it worse to do it on device, because of the risk that they do it to photos you don't intend to upload. This is clearly where Apple got caught off-guard, because to them, on-device = private. It seems like the issue is really the mixing of on-device and off. People seem to be fine with on-device data that stays on-device, and relatively fine with the idea that Apple gets your content if you upload it to them. But when they analyze the data on-device, and then upload the results to the cloud, that really gets people.
    - shuckles 5y ago
      This seems like a necessary discussion to have in preparation for widespread, default end to end encryption.
    - rustymonday 5y ago
      Is this really surprising to you? I'm not trying to be rude, but this is an enormous distinction. In today's world, smartphones are basically an appendage of your body. They should not work to potentially incriminate its owner.
      - alwillis 5y ago
        They should not work to potentially incriminate its owner. But that ship has long sailed, right? Every packet that leaves a device potentially incriminates its owner. Every access point and router is a potential capture point.
- userbinator 5y ago
  Yes, you had to trust Apple, but the huge difference with this new thing is that hiding behind CSAM gives them far more (legally obligated, in fact --- because showing you the images those hashes came from would be illegal) plausible deniability and difficulty of verifying their claims. In other words, extracting the code and analysing it to determine that it does do what you expect is, although not easy, still legal. But the source, the CSAM itself, is illegal to possess, so you can't do that verification much less publish the results. It is this effective legal moat around those questioning the ultimate targets of this system which people are worried about.
  - majormajor 5y ago
    Surely they could do their image matching against all photos in iCloud without telling you in advance, and then you'd be in exactly the same boat? Google was doing this for email as early as 2014, for instance, with the same concerns about its extensibility raised by the ACLU: https://www.theguardian.com/technology/2014/aug/04/google-child-abuse-ncmec-internet-watch-gmail https://www.theguardian.com/technology/2014/aug/04/google-ch... So in a world where Apple pushes you to set up icloud photos by default, and can do whatever they want there, and other platforms have been doing this sort of of thing for years, it's a bit startling that "on device before you upload" vs "on uploaded content" triggers far more discontent? Maybe it's that Apple announced it at all, vs doing it relatively silently like the others? Apple has always had access to every photo on your device, after all.
    - pseudalopex 5y ago
      It isn't startling people trust they can opt out of iCloud photos.
      - kemayo 5y ago
        If you trust that you can opt out of iCloud Photos to avoid server-side scanning, trusting that this on-device scanning only happens as part of the iCloud Photos upload process (with the only way it submits the reports being as metadata attached to the photo-upload, as far as I can tell) seems equivalent. There's certainly a slippery-slope argument, where some future update might change that scanning behavior. But the system-as-currently-presented seems similarly trustable.
- himaraya 5y ago
  No, the threat model differs entirely. Local scanning introduces a whole host of single points of failure, including the 'independent auditor' & involuntary scans, that risk the privacy & security of all local files on a device. Cloud scanning largely precludes these potential vulnerabilities.
  - majormajor 5y ago
    Your phone threat model should already include "the OS author has full access to do whatever they want to whatever data is on my phone, and can change what they do any time they push out an update." I don't think anyone's necessarily being too upset or paranoid about THIS, but maybe everyone should also be a little less trusting of every closed OS - macOS, Windows, Android as provided by Google - that has root access too.
    - himaraya 5y ago
      Sure, but that doesn't change the fact that the vulnerabilities with local scanning remain a significant superset of cloud scanning's. Apple has built iOS off user trust & goodwill, unlike most other OSes.
      - shuckles 5y ago
        Cloud Scanning vulnerability: no transparency over data use. On the phone, you can always confirm the contents of what’s added to the safety voucher’s associated data. On the cloud, anything about your photos is fair game. Where does that fit in your set intersection?
        himaraya 5y ago
        > On the phone, you can always confirm the contents of what’s added to the safety voucher’s associated data. ...except you can't? Not sure where these assumptions come from.
        5y ago
almostdigital 5y ago
Yeah that's true, although to do some sort of mass scanning stealthily they would need a system exactly like what they built with this, if they tried to upload everything for scanning the data use would be enormous and give it away. I guess it comes down to that I don't trust an OS vendor that ships an A.I. based snitch program that they promise will be dormant.
- kemayo 5y ago
  Speaking cynically, I think that them having announced this program like they did makes it less likely that they have any sort of nefarious plans for it. There's a lot of attention being paid to it now, and it's on everyone's radar going forwards. If they actually wanted to be sneaky, we wouldn't have known about this for ages.
  - theonlybutlet 5y ago
    They'd have to be transparent about it as someone would easily figure it out.You have no way of verifying the contents of that hash database. once the infrastructure is in place (i.e. on your phone) it's a lot easier to expand on it. People have short memories and are easily desensitized, after a year or two of this, everyone will forget and we'll be in uproar about it expanding to include this or that...
  - smichel17 5y ago
    You're making the mistake of anthropomorphizing a corporation. Past a certain size, corporations start behaving less like people and more like computers, or maybe profit-maximizing sociopaths. The intent doesn't matter, because 5 or 10 years down the line, it'll likely be a totally different set of people making the decision. If you want to predict a corporation's behavior, you need to look at the constants (or at least, slower-changing things), like incentives, legal/technical limitations, and internal culture/structure of decision-making (e.g. How much agency do individual humans have?).
    - kemayo 5y ago
      I feel that I was stating the incentives, though. This being an area people are paying attention to makes it less likely they'll do unpopular things involving it, from a pure "we like good PR and profits" standpoint. They might sneak these things in elsewhere, but this specific on-device-scanning program has been shown to be a risk even at its current anodyne level.
slg 5y ago
Which is why I am confused by a lot of this backlash. Apple already controls the hardware, software, and services. I don't see why it really matters where in that chain the scanning is done when they control the entire system. If Apple can't be trusted with this control today, why did people trust them with this control a week ago?
- zwily 5y ago
  That’s where I’m at. They could have just started doing this without even saying anything at all.
  - jimworm 5y ago
    But in that case they would eventually be caught red-handed and won't get to do the "for the children" spiel and get it swept under the rug like it's about to be.
    - bredren 5y ago
      The goal is not for it to be swept under the rug. The goal is for it to deflect concerns over the coming Privacy Relay service.
      - threatofrain 5y ago
        Their private relay service appears orthogonal to CSAM… it won’t make criminals and child abusers easier or harder to catch, and it doesn’t affect how people use their iCloud Photos storage.
        bredren 5y ago
        These people are commonly prosecuted using evidence that includes server logs that showing their static IP address. Read evidence from past trials it is obvious. See also successful and failed attempts to subpoena this info from VPN services. Only people with iCloud will be using the relay. It is true on the surface the photos is disconnected from the use. However, Apple only needs a solid answer that handles the bad optics of what you can do with the Tor-like anonymity of iCloud Privacy Relay. However, if you look more closely, the CSAM service and its implementation are crafted exactly around the introduction of the relay.
g42gregory 5y ago
> At some point you have to trust your OS vendor. Yes, and we were trusting Apple. And now this trust is going away.
- rootusrootus 5y ago
  > now this trust is going away Is it really? There are some very loud voices making their discontent felt. But what does the Venn diagram look like between 'people who are loudly condemning Apple for this' and 'people who were vehemently anti-Apple to begin with'? My trust was shaken a bit, but the more I hear about the technology they've implemented, the more comfortable I am with it. And frankly, I'm far more worried about gov't policy than I am about the technical details. We can't fix policy with tech.
  - matwood 5y ago
    > I'm far more worried about gov't policy than I am about the technical details. We can't fix policy with tech. Yeah. I don't really understand the tech utopia feeling that Apple could simply turn on e2ee and ignore any future legislation to ban e2ee. The policy winds are clearly blowing towards limiting encryption in some fashion. Maybe this whole event will get people to pay more attention to policy...maybe.
Veserv 5y ago
Yes, they can technically already do so, but that is not the question. The question is what can they legally do and justify with high confidence in the event of a legal challenge. Changes to binding contractual terms that allow broad readings and provide legal justification for future overreach are dangerous. If they really are serious that they are going to use these new features in a highly limited way then they can put their money where their mouth is and add legally binding contractual terms that limit what they can do with serious consequences if they are found to be in breach. Non-binding marketing PR assurances that they will not abuse their contractually justified powers are no substitute for the iron fist of legal penalty clause.
m4rtink 5y ago
What about trust-but-verify ? If the OS was open source and supported reproducible builds, you would not have to trust them, you could verify what it actually does & make sure the signed binaries they ship you actually correspond to the source code. Once kinda wonders what they want to hide if they talks so much about user privacy yet don't provide any means for users to verify their claims.
ianmiers 5y ago
What happens if someone tries to coerce Apple into writing backdoor code? Engineers at Apple could resist, resign, slow roll the design and engineering process. They could leak it and it would get killed. Things would have to get very very bad for that kind of pressure to work. On the other hand, once Apple has written a backdoor enthusiastically themselves, it's a lot easier to force someone to change how it can be used. The changes are small and compliance can be immediately verified and refusal punished. To take it to its logical extreme: you cannot really fire or execute people who delay something (especially if you lake the expertise to tell how long it should take). But you can fire or execute people who refuse to flip a switch. This technology deeply erodes Apple and its engineers' ability to resist future pressure. And the important bit here is there adversary isn't all powerful. It can coerce you to do things in secret, but its power isn't unlimited. See what happened with yahoo.[0] https://www.reuters.com/article/us-yahoo-nsa-exclusive/exclusive-yahoo-secretly-scanned-customer-emails-for-u-s-intelligence-sources-idUSKCN1241YT https://www.reuters.com/article/us-yahoo-nsa-exclusive/exclu...

jolux 5y ago

> And there is no way to audit that the database is what they claim it is, doesn't contain multiple databases that can be activated under certain conditions, etc. They describe a process for third parties to audit that the database was produced correctly.

LexGray 5y ago
Do we have any idea how the NCMEC database is curated? Are there cartoons from Hustler depicting underage girls in distress? Green text stories stating they are true about illegal sexual acts? CGI images of pre-pubescent looking mythical creatures? Manga/Anime images which are sold on the Apple Store? Legitimate artistic images from books currently sold? Images of Winnie the Pooh the government has declared pornographic? From the amount of material the Feds claim is being generated every year I would have to guess all of this is included. The multi-government clause is completely pointless with the five-eyes cooperation. The story here is that there is a black box of pictures. Apple will then use their own black box of undeclared rules to pass things along to the feds which they have not shared what would be considered offending in any way shape or form other than "we will know it when we see it". Part of the issue here is that Apple is taking the role of a moral authority. Traditionally Apple has been incredibly anti-pornography and I suspect that anything that managed to get into the database will be something Apple will just pass along.
- jolux 5y ago
  Apple is manually reviewing every case to ensure it’s CSAM. You do have to trust them on that. But if your problem is with NCMEC, you’ve got a problem with Facebook and Google who are already doing this too. And you can’t go to jail for possessing adult pornography. So even if you assume adult porn images are in the database, and Apple’s reviewers decide to forward them to NCMEC, you would still not be able to be prosecuted, at least in the US. Ditto for pictures of Winnie the Pooh. But for the rest of what you describe, simulated child pornography is already legally dicey as far as I know, so you can’t really blame Apple or NCMEC for that.
  - LexGray 5y ago

shuckles 5y ago

Apple shipped iCloud Private Relay which is a “1-line code change that hooks into CFNetwork” away from MITMing all your network connections, by this standard.

almostdigital 5y ago
For me the standard is that I don't want any 1-line code change between me and near-perfect Orwellian surveillance.
- shuckles 5y ago
  Since your one-liners seem to be immensely dense with functional changes, I can’t understand how you trust any software.
ec109685 5y ago
Any connection worth its salt should be TLS protected.
- shuckles 5y ago
  Also in CFNetwork. Probably a one line change to replace all session keys with an Apple generated symmetric key.

notJim 5y ago

> Until a 1-line code change happens that hooks it into UIImage. I really don't understand this view. You are using proprietary software, you are always an N-line change away from someone doing something you don't like. This situation doesn't change this. If you only use open source software and advocate for others to do the same, I would understand it more.

almostdigital 5y ago
> I really don't understand this view. You are using proprietary software, you are always an N-line change away from someone doing something you don't like. This situation doesn't change this. And I don't understand why it has to be black and white, I think the N is very important in this formula and if it is low that is a cause for concern. Like an enemy building a missile silo on an island just off your coast but promising it's just for defense. All arguments I see is along the lines of "Apple can technically do anything they want anyways so this doesn't matter". But maybe you're right and moving to FOSS is the only solution long-term, that's what I'm doing if Apple goes through with this.
- notJim 5y ago
  The size of N doesn't really matter. I'm sure Apple ships large PRs in every release, as any software company does.
  - almostdigital 5y ago
    Maybe not if you assume Apple is evil but for the case of Apple being good intentioned but having its hand forced, they will have a much harder time resisting a 1 line change than a mandate to spend years to develop a surveillance system
- fay59 5y ago
  I’d leave this one out to the lawyers. I’m not one but I don’t think that the court will evaluate the number of lines of code required for help.
5y ago