6 ms·

Here's an interesting thought experiment for anyone who disagrees that the proposed system is problematic. Let's say the police installs a new device in your h

by noptd 5y ago

Here's an interesting thought experiment for anyone who disagrees that the proposed system is problematic. Let's say the police installs a new device in your house. This device could be capable of seeing or hearing anything that happens in your entire home. However, the police claim that the device is only watching a specific door in your house and forwarding encrypted low resolution photos to their servers "just in case" some criteria is met and they need to take a look. Nevermind that it would only take a policy change and an OTA update that you have no visibility into, or chance of blocking, before it's watching your entire house in real-time and sending practically anything to their servers. After all, they did release a carefully worded statement regarding what the device supposedly does. But hey, you have several other doors to enter or exit your house from so what's the big deal right? You could seal that door and never be subjected to the system _as advertised_. Would you trust the police in that scenario, regardless of whether you have anything to hide?

Joeri 5y ago
It’s not just your house, it’s everyone’s house. Our smartphones and computers are our digital homes. When something is installed in those homes which monitors everyone for potential crimes, that is what I would label as an unreasonable search and a violation of privacy. It doesn’t matter what they’re looking for, our personal devices should be inviolate unless there is a suspicion of crime, and even then a court order should be involved. What apple doesn’t understand is that scanning on the device is worse for privacy than scanning on the server, not better.
- hypothesis 5y ago
  > What apple doesn’t understand is that scanning on the device is worse for privacy than scanning on the server, not better. We don’t have to excuse one of the most valuable corp around. They do understand, but are willing to cash-in some of the “privacy budget”.
- chuckee 5y ago
  > our personal devices should be inviolate unless there is a suspicion of crime, and even then a court order should be involved. Even with a court order, your devices should not cooperate with a search, unless instructed to do so by you, their owner.
spullara 5y ago
This is already the case with the iPhone. They could theoretically update it with software that does whatever they want at any time.
- noptd 5y ago
  True, but that's tangential to the issue at hand. To continue the physical home analogy, let's say you're a renter and your landlord has the ability and access to install something in your home at any time without you knowing. Maybe they include it as part of the terms for your upcoming lease renewal, maybe they don't. Does that mean you should just accept it when they decide to install one of those devices in your home?
  - thinkloop 5y ago
    > should you just accept it when they decide to install one of those devices in your home? The device they installed is invisible and you will never have any way of knowing they were in there, it's irrelevant what you accept.
    - feanaro 5y ago
      That's a further problem. That's the reason why you shouldn't accept closed platforms and closed-source software. But it also doesn't lead to the conclusion that what Apple is trying to do here is good or even neutral.
- simion314 5y ago
  There is a giant difference,If you are a developer you might understand this better, it is much easier to have an application behavior setup from a config file or from a database then from code. So you have a config that looks like scan_everythin = false; and an update would flip this to true for everyone or for special users. Also with the hashes , the check is not done locally so the government will ask Apple "hey check those hashes you have stored against this secret database too and if you find even 1 match send it to us, here is a judge order so shut up and do it". So after this the governments around the world job is easy, they throw hashes to Apple(and fortunately for Apple they can say they had no idea what original photos content was) where before the government would have needed to ask Apple to write new software and Apple could have said that it can't do it without starting a shit storem.
onethought 5y ago
That is nothing like this… you’ve just made an argument against having smart phones at all. Not apples improvement to protect the privacy of your photos.
- yunohn 5y ago
  > Not apples improvement to protect the privacy of your photos. This is quite the twist. What’s decreasing the privacy of my personal photos, before the neuralhash scanner is deployed?
  - onethought 5y ago
    Because the alternative is to scan all the images on the server. With the hash scanner only hash matches are scanned. All the other images are kept private.
    - ComputerGuru 5y ago
      Stop being disingenuous. They’re all scanned, regardless of whether it’s on the server or the phone.
      - onethought 5y ago
        Apple specifically said that isn’t the case. I’m being disingenuous?
- feanaro 5y ago
  Interesting choice of words. What exactly did they improve with this and how?
  - onethought 5y ago
    The ability to avoid decrypting and mining your entire photo library in order to comply with us law. Instead performing a hash match on the device that gives their compliance team a partial key to decrypt only the images in question to confirm it wasn’t a false positive. Mean while on a google pixel, google will just scan your entire library.
robertoandred 5y ago
Hashing on device instead of on the server is a policy change too.
beezischillin 5y ago
I don't trust anyone even though I have nothing to hide. If Apple was serious about privacy maybe they'd put some of their lobbying / peer pressure efforts towards that instead of trendy social fads at least this once.
vbezhenar 5y ago
Now change that a little bit. And imagine that you're renting this place. And that's not a police requirement, but home owner requirement who wants to prevent undesirable behaviour to protect everyone (or so he says). That would be a better analogy. You don't own your iPhone at all. You own few dozens of grams of steel, glass and silicone. You don't even own parts of it, they're sealed, DRMed and non-functional without proprietary firmware. And of course you hardly own the entire device, as you can't control software which runs on it. Don't pretend that you own a house.
oreilles 5y ago
> This device could be capable of seeing or hearing anything that happens in your entire home This device is your smartphone. It can already watch and listen everything you do. It can already read and write everything that's inside because that's how an operating system works. The fact that it can now scan for CP makes zero difference privacy wise. If you don't trust your smartphone to do what they advertise, you don't use one. If you believe Apple can make the iPhone send data to them or the governments without your consent and without even telling you, you shouldn't use Apple software in the first place because you'd have no reason to believe that such a feature hasn't already been implemented. Your device scanning its content is not a breach of privacy until that information is send without your consent. Wether it is for malware or for child porn. Claiming that this is a slippery slope is like claiming that building a bridge is a slippery slope as the worker could place a bomb inside anytime.
matwood 5y ago
The analogy is missing a key part of the system as designed today - putting something on someone else's property. The house analogies are terrible, but closer to what's happening is as follows. I'm your neighbor and I want you to store a box for me. It's tapped up so you don't really want to open it, but would you store it for me without knowing what's in it? You could be like other cloud providers, rip the tape off and rummage through looking for whatever. Apple has designed a method for me to scan the box at my house only for CSAM (agreed upon by the intersection of multiple databases), then hand it to you with a note that says 'this is not CSAM'. Now you can store the box with some confidence in knowing you're not storing CSAM. It's also more private because you don't need to go rummaging through my box - even though you currently can if you want. Of course we're starting from the basis that you have decided you don't want to store any CSAM. If you don't care about what you store it doesn't matter. And you're right that Apple could make a policy change to scan my whole house and upload that somewhere, but guess what? iOS users (smart phone users in general) have always been one policy change away from Apple/Google doing something like uploading/sharing all the face ML data (which IMO is way more valuable than some hash matches). So nothing has really changed.
- frzj 5y ago
  Analogies are always misleading. A cloud provider is no neighbor. They facilitate a specific service, and it is clear that the package is owned by the client. The cloud provider shouldn't have any liability for the data he stores, especially if it's encrypted and not shared to the public, nobody should ever care what's inside the package. If the authorities have a good reason to think that you have drugs or CP in your package, then they can force you to open it. IMHO, no combination of ones and zeros should ever be illegal. The act of distributing them to others or creating them in the real world should be. The energy stored on a flash drive doesn't harm anybody, human action does.
  - askafriend 5y ago
    iCloud servers are not your property, they are Apple’s property. Even the neighbor terminology is accurate since there’s is an implied trust higher than a random stranger. OP’s analogy is spot on.
nedt 5y ago
But in reality it‘s you installing this device by turning on cloud sync. And the law that needs photos to be scanned that are in the cloud. Which is done by all cloud providers. Because the photos are unencrypted. How are we going to change that?

by noptd 5y ago

Joeri 5y ago

It’s not just your house, it’s everyone’s house. Our smartphones and computers are our digital homes. When something is installed in those homes which monitors everyone for potential crimes, that is what I would label as an unreasonable search and a violation of privacy. It doesn’t matter what they’re looking for, our personal devices should be inviolate unless there is a suspicion of crime, and even then a court order should be involved. What apple doesn’t understand is that scanning on the device is worse for privacy than scanning on the server, not better.

hypothesis 5y ago
> What apple doesn’t understand is that scanning on the device is worse for privacy than scanning on the server, not better. We don’t have to excuse one of the most valuable corp around. They do understand, but are willing to cash-in some of the “privacy budget”.
chuckee 5y ago
> our personal devices should be inviolate unless there is a suspicion of crime, and even then a court order should be involved. Even with a court order, your devices should not cooperate with a search, unless instructed to do so by you, their owner.

spullara 5y ago

This is already the case with the iPhone. They could theoretically update it with software that does whatever they want at any time.

noptd 5y ago
True, but that's tangential to the issue at hand. To continue the physical home analogy, let's say you're a renter and your landlord has the ability and access to install something in your home at any time without you knowing. Maybe they include it as part of the terms for your upcoming lease renewal, maybe they don't. Does that mean you should just accept it when they decide to install one of those devices in your home?
- thinkloop 5y ago
  > should you just accept it when they decide to install one of those devices in your home? The device they installed is invisible and you will never have any way of knowing they were in there, it's irrelevant what you accept.
  - feanaro 5y ago
    That's a further problem. That's the reason why you shouldn't accept closed platforms and closed-source software. But it also doesn't lead to the conclusion that what Apple is trying to do here is good or even neutral.
simion314 5y ago
There is a giant difference,If you are a developer you might understand this better, it is much easier to have an application behavior setup from a config file or from a database then from code. So you have a config that looks like scan_everythin = false; and an update would flip this to true for everyone or for special users. Also with the hashes , the check is not done locally so the government will ask Apple "hey check those hashes you have stored against this secret database too and if you find even 1 match send it to us, here is a judge order so shut up and do it". So after this the governments around the world job is easy, they throw hashes to Apple(and fortunately for Apple they can say they had no idea what original photos content was) where before the government would have needed to ask Apple to write new software and Apple could have said that it can't do it without starting a shit storem.

onethought 5y ago

That is nothing like this… you’ve just made an argument against having smart phones at all. Not apples improvement to protect the privacy of your photos.

yunohn 5y ago
> Not apples improvement to protect the privacy of your photos. This is quite the twist. What’s decreasing the privacy of my personal photos, before the neuralhash scanner is deployed?
- onethought 5y ago
  Because the alternative is to scan all the images on the server. With the hash scanner only hash matches are scanned. All the other images are kept private.
  - ComputerGuru 5y ago
    Stop being disingenuous. They’re all scanned, regardless of whether it’s on the server or the phone.
    - onethought 5y ago
      Apple specifically said that isn’t the case. I’m being disingenuous?
feanaro 5y ago
Interesting choice of words. What exactly did they improve with this and how?
- onethought 5y ago
  The ability to avoid decrypting and mining your entire photo library in order to comply with us law. Instead performing a hash match on the device that gives their compliance team a partial key to decrypt only the images in question to confirm it wasn’t a false positive. Mean while on a google pixel, google will just scan your entire library.

robertoandred 5y ago

Hashing on device instead of on the server is a policy change too.

beezischillin 5y ago

I don't trust anyone even though I have nothing to hide. If Apple was serious about privacy maybe they'd put some of their lobbying / peer pressure efforts towards that instead of trendy social fads at least this once.

vbezhenar 5y ago

Now change that a little bit. And imagine that you're renting this place. And that's not a police requirement, but home owner requirement who wants to prevent undesirable behaviour to protect everyone (or so he says). That would be a better analogy. You don't own your iPhone at all. You own few dozens of grams of steel, glass and silicone. You don't even own parts of it, they're sealed, DRMed and non-functional without proprietary firmware. And of course you hardly own the entire device, as you can't control software which runs on it. Don't pretend that you own a house.

oreilles 5y ago

> This device could be capable of seeing or hearing anything that happens in your entire home This device is your smartphone. It can already watch and listen everything you do. It can already read and write everything that's inside because that's how an operating system works. The fact that it can now scan for CP makes zero difference privacy wise. If you don't trust your smartphone to do what they advertise, you don't use one. If you believe Apple can make the iPhone send data to them or the governments without your consent and without even telling you, you shouldn't use Apple software in the first place because you'd have no reason to believe that such a feature hasn't already been implemented. Your device scanning its content is not a breach of privacy until that information is send without your consent. Wether it is for malware or for child porn. Claiming that this is a slippery slope is like claiming that building a bridge is a slippery slope as the worker could place a bomb inside anytime.

matwood 5y ago

The analogy is missing a key part of the system as designed today - putting something on someone else's property. The house analogies are terrible, but closer to what's happening is as follows. I'm your neighbor and I want you to store a box for me. It's tapped up so you don't really want to open it, but would you store it for me without knowing what's in it? You could be like other cloud providers, rip the tape off and rummage through looking for whatever. Apple has designed a method for me to scan the box at my house only for CSAM (agreed upon by the intersection of multiple databases), then hand it to you with a note that says 'this is not CSAM'. Now you can store the box with some confidence in knowing you're not storing CSAM. It's also more private because you don't need to go rummaging through my box - even though you currently can if you want. Of course we're starting from the basis that you have decided you don't want to store any CSAM. If you don't care about what you store it doesn't matter. And you're right that Apple could make a policy change to scan my whole house and upload that somewhere, but guess what? iOS users (smart phone users in general) have always been one policy change away from Apple/Google doing something like uploading/sharing all the face ML data (which IMO is way more valuable than some hash matches). So nothing has really changed.

frzj 5y ago
Analogies are always misleading. A cloud provider is no neighbor. They facilitate a specific service, and it is clear that the package is owned by the client. The cloud provider shouldn't have any liability for the data he stores, especially if it's encrypted and not shared to the public, nobody should ever care what's inside the package. If the authorities have a good reason to think that you have drugs or CP in your package, then they can force you to open it. IMHO, no combination of ones and zeros should ever be illegal. The act of distributing them to others or creating them in the real world should be. The energy stored on a flash drive doesn't harm anybody, human action does.
- askafriend 5y ago
  iCloud servers are not your property, they are Apple’s property. Even the neighbor terminology is accurate since there’s is an implied trust higher than a random stranger. OP’s analogy is spot on.

nedt 5y ago

But in reality it‘s you installing this device by turning on cloud sync. And the law that needs photos to be scanned that are in the cloud. Which is done by all cloud providers. Because the photos are unencrypted. How are we going to change that?