3 ms·

I think many at Apple really are surprised by this, and I put it down to a failure of elite consensus. I think that a number of folks at high levels in the SV

I think many at Apple really are surprised by this, and I put it down to a failure of elite consensus. I think that a number of folks at high levels in the SV executive suite have accepted a manufactured consensus along with their peers in Washington, and that consensus is something like: "people don't care about the privacy of what's on their device and they'll put up with anything to stop CSAM, even if it means scanning personal backups and local files (as opposed to shared files.)" This seems like a reasonable thing to believe, since server-side scanning of (mostly shared) files has been going on for years and nobody has pushed back very hard on it. But what I think the consensus missed is that the reason for this lack-of-pushback is that nobody in the wider world had really been asked to weigh in on it before. It was something that a few elite tech busybodies were aware of, and most people accepted the idea that providers needed to check out photos that lived (unencrypted) on their servers. Apple accepted this logic and extended it unthinkingly beyond shared photos to unshared private photo libraries on the user's personal device (even if they are staged for backup as part of the iCloud Photos synchronization service, which is just a policy choice.) This was a second mistake because it assumed that because users mostly ignored the scanning of shared server-hosted files, they had somehow given consent to having their private files searched on their device. I don't think they had. Overall, this announcement is the first time anyone has attempted to have an actual public debate to see how real users feel about this kind of surveillance, particularly automated surveillance of private photos (and an automated system with potential flaws.) Apple's mistake here was to assume that their user base had already given consent -- when they'd just never been asked. It's a very human mistake to make, frankly. The question is whether Apple will listen to their users or if they'll double down and push this through against their users' pushback. I can forgive Apple for misunderstanding their users once, but continuing down this path will be a lot harder to understand. ETA: To illustrate how much more pervasive Apple's surveillance is than the standard (ignoring the PSI protocols), consider this quote from an EU Parliament briefing: "Others, such as Dropbox, Google and Microsoft perform scans for illegal images, but 'only when someone shares them, not when they are uploaded'." (I can only trust that this is factually true.) In this sense, Apple's move to scan all photos in your library is a significant functional escalation.) https://www.europarl.europa.eu/RegData/etudes/BRIE/2020/659360/EPRS_BRI(2020)659360_EN.pdf https://www.europarl.europa.eu/RegData/etudes/BRIE/2020/6593...

shuckles 5y ago
This is a bad take, given that questioning the “elite consensus” is most likely how Apple arrived at this solution. Their proposed system doesn’t take for granted a whole host of assumptions made by every incumbent CSAM detection framework. Next, it claims that consent is ambiguous when in reality the system doesn’t exist and users who don’t consent have a means and plenty of time to opt out. Finally, there are people who argue the exact opposite: Alex Stamos has claimed one issue with Apple is that they didn’t engage the industry and academics like him enough when developing this solution.
- matthewdgreen 5y ago
  I think generally saying "this is a bad take" isn't very helpful. You should elaborate on how in terms of practical functionality Apple's system is really that different than other systems out there.
  - shuckles 5y ago
    You argued that somehow Apple blindly followed “elite consensus” to end up with the system they announced. I pointed out that the system they announced questions every aspect of “elite consensus” about how to scan for CSAM in a cloud service. Now you’re waving your hands and saying they’re practically the same when there is a 14 page security threat model on Apple’s website which lists the many ways in which it’s not. How many of the incumbent systems make their perceptual hash available or let clients inspect the data used for manual review in an associated data packet? Where does Dropbox publish its root hash for the known CSAM database? I could go on. Next, arriving at “practically the same solution” is a necessary but not sufficient condition for having the same assumptions. Arguing about practical functionality doesn’t even prove your point. Finally, I explained why it’s a bad take so I’m not sure what you find unhelpful.
    - matthewdgreen 5y ago
      I'm a cryptographer, so I do understand the system fairly well. What Apple is doing is applying an automated hash-based image matching system to all photos in your library, and triggering an alert (plus a low-resolution photo) to a human being when 30 photos match. Except for the use of cryptography and the "30" threshold, this is functionally identical to what existing providers do on the server side. As an expert in this field I'm thrilled to see the use of fancy cryptography, but the use of a 2PC protocol does not change the underlying functionality at all. What is different about the Apple system is that unlike many companies (as of 2019) [0] they are not simply scanning shared photos intended for distribution. They are scanning all photos in your library, even unshared ones. This to me is much more significant than the use of PSI or adding a threshold of 30 reports. You can agree or disagree but it's much more helpful to argue this based on the merits than to be rude about it. [0] page 8: https://www.europarl.europa.eu/RegData/etudes/BRIE/2020/659360/EPRS_BRI(2020)659360_EN.pdf https://www.europarl.europa.eu/RegData/etudes/BRIE/2020/6593...

shuckles 5y ago

This is a bad take, given that questioning the “elite consensus” is most likely how Apple arrived at this solution. Their proposed system doesn’t take for granted a whole host of assumptions made by every incumbent CSAM detection framework. Next, it claims that consent is ambiguous when in reality the system doesn’t exist and users who don’t consent have a means and plenty of time to opt out. Finally, there are people who argue the exact opposite: Alex Stamos has claimed one issue with Apple is that they didn’t engage the industry and academics like him enough when developing this solution.

matthewdgreen 5y ago
I think generally saying "this is a bad take" isn't very helpful. You should elaborate on how in terms of practical functionality Apple's system is really that different than other systems out there.
- shuckles 5y ago
  You argued that somehow Apple blindly followed “elite consensus” to end up with the system they announced. I pointed out that the system they announced questions every aspect of “elite consensus” about how to scan for CSAM in a cloud service. Now you’re waving your hands and saying they’re practically the same when there is a 14 page security threat model on Apple’s website which lists the many ways in which it’s not. How many of the incumbent systems make their perceptual hash available or let clients inspect the data used for manual review in an associated data packet? Where does Dropbox publish its root hash for the known CSAM database? I could go on. Next, arriving at “practically the same solution” is a necessary but not sufficient condition for having the same assumptions. Arguing about practical functionality doesn’t even prove your point. Finally, I explained why it’s a bad take so I’m not sure what you find unhelpful.
  - matthewdgreen 5y ago
    I'm a cryptographer, so I do understand the system fairly well. What Apple is doing is applying an automated hash-based image matching system to all photos in your library, and triggering an alert (plus a low-resolution photo) to a human being when 30 photos match. Except for the use of cryptography and the "30" threshold, this is functionally identical to what existing providers do on the server side. As an expert in this field I'm thrilled to see the use of fancy cryptography, but the use of a 2PC protocol does not change the underlying functionality at all. What is different about the Apple system is that unlike many companies (as of 2019) [0] they are not simply scanning shared photos intended for distribution. They are scanning all photos in your library, even unshared ones. This to me is much more significant than the use of PSI or adding a threshold of 30 reports. You can agree or disagree but it's much more helpful to argue this based on the merits than to be rude about it. [0] page 8: https://www.europarl.europa.eu/RegData/etudes/BRIE/2020/659360/EPRS_BRI(2020)659360_EN.pdf https://www.europarl.europa.eu/RegData/etudes/BRIE/2020/6593...