5 ms·

> This problem is the capability > The only way to prevent all this is for the system not to exist. I've been having trouble following this argument over the

> This problem is the capability > The only way to prevent all this is for the system not to exist. I've been having trouble following this argument over the last week. Isn't it clear that the capability already exists? Whether or not Apple goes through with its CSAM plan, the capability is evidently there. In other words, since Apple is a closed system, the capability was there before they announced the CSAM plans. Their announcement has changed nothing about capability other than reminding people that Apple has privileged access to the data on iPhones. I guess the question is, if Apple does the CSAM program does that make them more likely to cave to government pressure to search for other things? And to do so without telling users?

n8cpdx 5y ago
There shouldn’t be a single line of code, or single binary blob, on my device that can compute these photo DNA hashes. Apple could always go ahead and add that functionality in an update, but then there would be a big backlash and the opportunity to not update or switch providers.
- k2enemy 5y ago
  Wouldn't users have the ability to not update or switch providers if Apple expands the scope of its search beyond CSAM? I guess I don't see the huge difference between the surveillance code existing on the phone but not used for objectionable purposes versus the line of code sitting in a different branch in git and not deployed on actual phones (yet). I'm completely against this move by the way -- I'm not trying to defend it. But I want to be able to argue effectively against it.
  - n8cpdx 5y ago
    No, because they’re not scanning for CSAM, they’re scanning for hashes. That hash database needs to be updateable for it to be useful at all. A significant part of the policy protections are implemented off device, so those can’t be trusted.
  - ajsnigrutin 5y ago
    This is like renting a house and the owner saying "we installed a camera in your house, but we'll only look at the video, if someone screams 'help'" Then someone says "what if they change it so it sends the video if someone says 'do you want some pot?'" And your argument would be "wouldn't you just move if they did that?" - this argument is pointless, because it's not the "help" part that bothersome, it's the always recording camera in a black box, for which you never know what's recording and what's it sending to apple. Let's say hypothetically they do this, and modify the database to trigger on the "tank man" photo, and give that photo such weight, that it triggers manual review by the CCP observer (remember, apple data for chinese users is hosted in china) - you'll never know the image got flagged, you'll never even know that it was sent anywhere, but your social score will go down and you might "vanish" during the night. Imagine some new wikileaks happening... 15 journalists around the world get the data, NSA/CIA/FBI has hacked one of them and found some photos of USA doing something bad.... add those photos to the database, have apple scan all the phones, you find the other 14 journalists, and possible even the leaker who took the original photo. If this is expended to macbooks (which it probably will), even worse, because people keep more sensitive stuff on ther computers than their phones. And all this for what? Child molesters and rapists (the worst of the pedos) create new photos and videos, that are in none of the databases, so you never catch them. Their direct sponsors (paying customers) get the photos first, and again, before they're in the database. So all you've caught is someone browsing 4chan.
    - n8cpdx 5y ago
      > And all this for what? Child molesters and rapists (the worst of the pedos) create new photos and videos, that are in none of the databases, so you never catch them. Their direct sponsors (paying customers) get the photos first, and again, before they're in the database. I hadn’t considered that. This would seem to create a market incentive to abuse more children and create a greater variety of new abusive content. Has that been considered at all?
- tshaddox 5y ago
  > Apple could always go ahead and add that functionality in an update, but then there would be a big backlash and the opportunity to not update or switch providers. Isn't that exactly what is happening now?
  - n8cpdx 5y ago
    It is exactly. And I have my opportunity to start migrating to Android before the big update.
ipv6ipv4 5y ago
There is a qualitative difference between tweaking an existing code base (expanding the scope of this detector) and coercing a company to dedicate teams of tens to hundreds of engineers over years to create a brand new code base from scratch (writing this detector in the first place.)
tshaddox 5y ago
It's a very ill-considered argument, because Apple obviously already has the capability of pushing literally any conceivable software update to iPhones. Apple (and thus iPhones) already have the capability to do anything you can conceive of, regardless of how terrible. For instance, iPhones have the capability of activating the cameras at all times, detecting when they capture compromising or embarrassing video, and uploading those videos to a public website along with your name and contact information. The only thing preventing Apple from doing that is their own policies. If you think it's bad that Apple can push software updates to iPhones, then by all means make that argument. But "the only way to prevent bad changes to this system down the road is for this first version of the system to never exist" is a very poor argument, given that the only "system" that needs to exist is the ability to push software updates, and that system existed long ago.
- babesh 5y ago
  That isn't the argument at all. What Apple can't have happen without major reputational damage is get caught doing it without telling its customers. Thus this is seen as a marketing backdoor to the actual backdoor. This is seen as a ploy that later gives Apple the plausible deniability argument that the government is making them do it (other forms of content) even though they created the means (on device scanning) for the government to exploit. Why limit it to content that will be uploaded to iCloud? Why limit iMessage scanning to kids? Why not also let the government know? Why limit it to child porn since the system works for any image? Why limit it to 30 hits?
  - babesh 5y ago
    Imagine the reputational damage to Apple if say in 2 years it was discovered that they had been scanning all photos on your device and that the matches had been going to the government and that they didn't tell you. Their privacy marketing angle would be completely toast. Imagine if they did this to phones outside the US. Any sane marketing person would try to find a way to sugarcoat it first.
- pseudalopex 5y ago
  > But "the only way to prevent bad changes to this system down the road is for this first version of the system to never exist" is a very poor argument, given that the only "system" that needs to exist is the ability to push software updates, and that system existed long ago. Apple relied on that argument 5 years ago successfully.[1] [1] https://en.wikipedia.org/wiki/FBI–Apple_encryption_dispute https://en.wikipedia.org/wiki/FBI–Apple_encryption_dispute
zarzavat 5y ago
Once a certain piece of technology exists, courts can order that it be used a certain way other than the way it was intended. Neither courts nor legislative bodies are capable of explicitly requiring the invention of technology that doesn't yet exist, although in some cases they could make it an implicit requirement of a law. At least in the US most lawmakers are varying shades of technologically illiterate and don't have a grasp of what is possible or not possible. If technology doesn't exist yet, you can say to them "this is impossible", and with enough of a bribe, they will believe it. Apple by building this technology, has put it on the map and announced that this technology both possible and is ready.

n8cpdx 5y ago

There shouldn’t be a single line of code, or single binary blob, on my device that can compute these photo DNA hashes. Apple could always go ahead and add that functionality in an update, but then there would be a big backlash and the opportunity to not update or switch providers.

k2enemy 5y ago
Wouldn't users have the ability to not update or switch providers if Apple expands the scope of its search beyond CSAM? I guess I don't see the huge difference between the surveillance code existing on the phone but not used for objectionable purposes versus the line of code sitting in a different branch in git and not deployed on actual phones (yet). I'm completely against this move by the way -- I'm not trying to defend it. But I want to be able to argue effectively against it.
- n8cpdx 5y ago
  No, because they’re not scanning for CSAM, they’re scanning for hashes. That hash database needs to be updateable for it to be useful at all. A significant part of the policy protections are implemented off device, so those can’t be trusted.
- ajsnigrutin 5y ago
  This is like renting a house and the owner saying "we installed a camera in your house, but we'll only look at the video, if someone screams 'help'" Then someone says "what if they change it so it sends the video if someone says 'do you want some pot?'" And your argument would be "wouldn't you just move if they did that?" - this argument is pointless, because it's not the "help" part that bothersome, it's the always recording camera in a black box, for which you never know what's recording and what's it sending to apple. Let's say hypothetically they do this, and modify the database to trigger on the "tank man" photo, and give that photo such weight, that it triggers manual review by the CCP observer (remember, apple data for chinese users is hosted in china) - you'll never know the image got flagged, you'll never even know that it was sent anywhere, but your social score will go down and you might "vanish" during the night. Imagine some new wikileaks happening... 15 journalists around the world get the data, NSA/CIA/FBI has hacked one of them and found some photos of USA doing something bad.... add those photos to the database, have apple scan all the phones, you find the other 14 journalists, and possible even the leaker who took the original photo. If this is expended to macbooks (which it probably will), even worse, because people keep more sensitive stuff on ther computers than their phones. And all this for what? Child molesters and rapists (the worst of the pedos) create new photos and videos, that are in none of the databases, so you never catch them. Their direct sponsors (paying customers) get the photos first, and again, before they're in the database. So all you've caught is someone browsing 4chan.
  - n8cpdx 5y ago
    > And all this for what? Child molesters and rapists (the worst of the pedos) create new photos and videos, that are in none of the databases, so you never catch them. Their direct sponsors (paying customers) get the photos first, and again, before they're in the database. I hadn’t considered that. This would seem to create a market incentive to abuse more children and create a greater variety of new abusive content. Has that been considered at all?
tshaddox 5y ago
> Apple could always go ahead and add that functionality in an update, but then there would be a big backlash and the opportunity to not update or switch providers. Isn't that exactly what is happening now?
- n8cpdx 5y ago
  It is exactly. And I have my opportunity to start migrating to Android before the big update.

ipv6ipv4 5y ago

There is a qualitative difference between tweaking an existing code base (expanding the scope of this detector) and coercing a company to dedicate teams of tens to hundreds of engineers over years to create a brand new code base from scratch (writing this detector in the first place.)

tshaddox 5y ago

It's a very ill-considered argument, because Apple obviously already has the capability of pushing literally any conceivable software update to iPhones. Apple (and thus iPhones) already have the capability to do anything you can conceive of, regardless of how terrible. For instance, iPhones have the capability of activating the cameras at all times, detecting when they capture compromising or embarrassing video, and uploading those videos to a public website along with your name and contact information. The only thing preventing Apple from doing that is their own policies. If you think it's bad that Apple can push software updates to iPhones, then by all means make that argument. But "the only way to prevent bad changes to this system down the road is for this first version of the system to never exist" is a very poor argument, given that the only "system" that needs to exist is the ability to push software updates, and that system existed long ago.

babesh 5y ago
That isn't the argument at all. What Apple can't have happen without major reputational damage is get caught doing it without telling its customers. Thus this is seen as a marketing backdoor to the actual backdoor. This is seen as a ploy that later gives Apple the plausible deniability argument that the government is making them do it (other forms of content) even though they created the means (on device scanning) for the government to exploit. Why limit it to content that will be uploaded to iCloud? Why limit iMessage scanning to kids? Why not also let the government know? Why limit it to child porn since the system works for any image? Why limit it to 30 hits?
- babesh 5y ago
  Imagine the reputational damage to Apple if say in 2 years it was discovered that they had been scanning all photos on your device and that the matches had been going to the government and that they didn't tell you. Their privacy marketing angle would be completely toast. Imagine if they did this to phones outside the US. Any sane marketing person would try to find a way to sugarcoat it first.
pseudalopex 5y ago
> But "the only way to prevent bad changes to this system down the road is for this first version of the system to never exist" is a very poor argument, given that the only "system" that needs to exist is the ability to push software updates, and that system existed long ago. Apple relied on that argument 5 years ago successfully.[1] [1] https://en.wikipedia.org/wiki/FBI–Apple_encryption_dispute https://en.wikipedia.org/wiki/FBI–Apple_encryption_dispute

zarzavat 5y ago

Once a certain piece of technology exists, courts can order that it be used a certain way other than the way it was intended. Neither courts nor legislative bodies are capable of explicitly requiring the invention of technology that doesn't yet exist, although in some cases they could make it an implicit requirement of a law. At least in the US most lawmakers are varying shades of technologically illiterate and don't have a grasp of what is possible or not possible. If technology doesn't exist yet, you can say to them "this is impossible", and with enough of a bribe, they will believe it. Apple by building this technology, has put it on the map and announced that this technology both possible and is ready.