6 ms·

Irrespective of whether or not NeuralHash is flawed, should Apple scan user data or should they not? If not, what is going to convince them to stop at this poi

Irrespective of whether or not NeuralHash is flawed, should Apple scan user data or should they not? If not, what is going to convince them to stop at this point? I believe that they should scan user data in some capacity, because this is about data that causes harm to children. However, I believe that they should not run the scan on the device, because that carries significant drawbacks for personal privacy.

bequanna 5y ago
My friend, this was never about “data that causes harm to children”. This is a carefully chosen stated reason for Apple to add this functionality to avoid user pushback. The actual uses will vary from country to country, but on the whole will not result in the protection of anyone other than the government or ruling party. In countries with less civil freedom this will be exploited and used as a tool of oppression against minority political groups.
- sharken 5y ago
  I think this has been established already, but nevertheless it's best to repeat it. The on-device scanning capabilities will be expanded in the future, but it can be stopped if Apple care about their users. Right now it seems they don't care. Forgot to mention that I'll not be recommending or purchasing any Apple devices until they halt this scanning idea.
  - bequanna 5y ago
    > but it can be stopped if Apple care about their users. Apple cares about selling devices, maximizing revenue. Users are just something required to authorize transactions. If Apple has to kowtow to the most oppressive regimes in the world to keep doing the above, you bet they will and probably already are. And since it is in the interest of both Apple + Gov't to do shady stuff in the dark, there is a good chance they can hide it the public indefinitely.
djbebs 5y ago
Your devices should not betray you, period no exceptions. I dont care if it will save billions of lives, your devices should not betray you.
totony 5y ago
I don't believe they should scan user data. User privacy is of utmost important (even more than slim risk of doing whatever). Do you want companies to have free reign scanning anything you use for "unauthorized" activities?
least 5y ago
No, they shouldn't scan data. Apple's role is not the police and they don't have the responsibility or moral imperative to scan for illegal imagery on their customers' phones. In fact it's the opposite. They should be in service to their customers and people's devices especially should be in service to its owner. On-device scanning violates this premise. There are things you can use technology for to help combat CSAM, but this is certainly not an acceptable way to go about it. Creating tools to help report abuse and illegal imagery in their software, for example, could be useful. This is not.
- slg 5y ago
  >Apple's role is not the police and they don't have the responsibility or moral imperative to scan for illegal imagery on their customers' phones What if the police come to Apple with a warrant? Should Apple have the responsibility to reveal the user's content to authorities? Part of the motivation for this type of thing is that E2EE effectively makes the old system of warrants obsolete and hides this data permanently from authorities. Lots of people see that as a huge win. Lots of people see that as a huge loss.
  - least 5y ago
    > What if the police come to Apple with a warrant? Should Apple have the responsibility to reveal the user's content to authorities? If they lack the capability to do so then they don't need to worry about it. > Part of the motivation for this type of thing is that E2EE effectively makes the old system of warrants obsolete and hides this data permanently from authorities. Lots of people see that as a huge win. Lots of people see that as a huge loss. The speculation that this enables E2EE is just that; speculation. It's also not a strong argument because Apple could implement E2EE without snooping people's photographs and they'd be in the same position of being unable to assist law enforcement. If you have a system in place that intercepts all files before they're encrypted and uploaded to the cloud, then E2EE doesn't serve its purpose of ensuring privacy of people's data. It completely defeats the point.
    - slg 5y ago
varispeed 5y ago
The children are used as a manipulation tool - if you talk about CSAM, your videos are being demonetised and reach is being reduced and people don't want to be associated with it in any way or being attacked for opposing the idea of protecting children. If they said from the outset that they are going to scan phones for any illegal content, the push back would be much bigger. So now they have a wedge that they can gradually use to expand what is going to be scanned and when. This is a classic power move straight from 48 laws of power.
- nullc 5y ago
  Even searching for case law and public policy on this subject causes a chilling "WARNING CHILD ABUSE IMAGERY IS ILLEGAL" box to display on google. Speaking out enough on this gets you called a pedo. I will not go as far to claim that the people for this are intentionally using children to shield themselves from criticism push out a universal surveillance mechanism. ... but if you wanted to do exactly that, hiding behind the spectre of child abuse is exactly that. There are some hard questions we should be asking. Facebook claims to have reported some 20 million insances of child porn last year, but there were only a couple prosecutions. Are the matches fake? Is the goal really to just build a huge database of kompromat? If these images aren't a concern why are only an infinitesimal proportion of cases prosecuted?
dvt 5y ago
> I believe that they should scan user data in some capacity, because this is about data that causes harm to children. I think this is actually a pretty weak argument. Children may be vulnerable, but so are many other segments of the population, including those with disabilities, the elderly, and so on. I guess my question is if we're trying to protect children, then why not go all the way? The answer is because going all the way is dangerous: where does it stop? Why here but not there? I think a better argument would be: we should scan phones for X because X is illegal, and we've deemed, as a society, that X is wrong/bad/punishable/etc. But the problem with that argument is that we already have ways of checking phones/user data for X, and, thankfully, most have to go through a legal process of probable cause, obtaining warrants, and so on. Apple's proposal completely circumvents the legal system. This is a complicated question that I'm sure will be litigated in court in the next few years. SCOTUS has ruled that drug sniffing dogs (e.g. in airports) are constitutional, whereas using infrared imaging (e.g. for finding drug grow houses) is not. Are hashing algorithms more like drug sniffing dogs or like infrared imaging?
- nonbirithm 5y ago
  I'm not understanding how it completely circumvents the legal system if the law enforcement agencies are still contacted after the flag is triggered. But if the lack of a warrant is the problem then the main issue would be the third-party doctrine and having no reasonable expectation of privacy when entrusting your data with any company, not just Apple.
  - xboxnolifes 5y ago
    > I'm not understanding how it completely circumvents the legal system if the law enforcement agencies are still contacted after the flag is triggered. Because, if I were to compare to what OP claims here, constant device scanning and then contacting law enforcement is like having a company constantly infrared scan your house for drug growing, and then contact the authorities about it. This is circumventing the authorities' legal limitations by moving the burden of responsibility to a company.
ak391 5y ago
Here’s a web demo[0] where you can try out any two images and see the resulting hashes, and whether there’s a collision. You can also try your own transformations (rotation, adding a filter, etc) on the image. Demo was built using Gradio[1]. [0]: https://huggingface.co/spaces/akhaliq/AppleNeuralHash2ONNX https://huggingface.co/spaces/akhaliq/AppleNeuralHash2ONNX [1]: https://gradio.dev https://gradio.dev
tw04 5y ago
> I believe that they should scan user data in some capacity, because this is about data that causes harm to children. Should GM use onstar to record every conversation you have in your car and use AI to flag conversations that relate to child abuse? After all it involves children. And if down the road law enforcement decides to expand what gets flagged, you shouldn’t care if you aren’t breaking any laws, right? This is a horrible idea and it’s a matter of if, not when it is abused.
mfer 5y ago
> I believe that they should scan user data in some capacity, because this is about data that causes harm to children. Google, Facebook, and others are already scanning the images when uploaded to them. We should not be surprised that the images we upload are being scanned. Sometimes for CSAM, sometimes to train ML, and sometimes for other things. Apple doing scanning images that were uploaded to their servers would just be another company doing the same thing. If I understand things correctly, these companies even have some legal obligations to do some of this. > However, I believe that they should not run the scan on the device, because that carries significant drawbacks for personal privacy. This is about where Apple is doing the scanning. Doing it in icloud is on them. Doing it on our phones is an invasion of privacy. And, it builds a system that can be used for other things. It opens a door, as many others have pointed out. I wonder if Apple did all of this to scale the implementation. So it wouldn't have to run on their servers. They wouldn't have to maintain and power them. Poor implementation if that's the case.
yrral 5y ago
> If not, what is going to convince them to stop at this point? Maybe if masses of people go into apple stores and save photos on demo phones that collide with the files they're looking for. Do those phones get scanned? Will apple ban their own app store accounts?
craftinator 5y ago
I wonder when they'll add this feature to MacOS?
DavideNL 5y ago
> should Apple scan user data or should they not? I guess it depends on whether the system is 100% safe and cannot be exploited; Since only iCloud photos are scanned, and other cloud storage providers already do this and have been scanning their users photos for a long time. I haven’t heard anyone complain when Google started scanning their cloud photos, which is kind of… interesting.
lisper 5y ago
> this is about data that causes harm to children. The data is not what causes the harm. It is the method used to produce (some of) the data that causes the harm. And the problem is that looking for the data in order to find the people causing the harm runs afoul of the converse fallacy. It's the same fallacy that causes people to think that because most terrorists (that they know of) are Muslims, that most Muslims must be terrorists. Most child abusers have photos of naked children on their phones, therefore most people who have photos of naked children on their phones must be child abusers. It's false in both cases. A friend of mine, for example, has photos of her naked child on her phone that she sent to the doctor to help diagnose a rash.
throwaway81523 5y ago
> I believe that they should scan user data in some capacity, because this is about data that causes harm to children. The legal system already provides a means for invading people's privacy given certain criteria, whether by scanning their phones, physically searching their residences and offices, or whatever. The criterion is "probable cause" and the search is launched by a judge signing a warrant that the probable cause exists. That is how the Framers wrote it into the US Constitution (see Amendment 4), because they had dealt with too many bogus searches done by the Colonial governments and wanted to put a stop to that. Things have not really changed since then. "Think of the children!!!" is a traditional thought-stopper, but really, if there is substantive reason to search someone's phone, just convince a judge to sign a warrant and everything will be fine. Searching people's stuff when they are not under any concrete suspicion is part of why we had a revolution in this country in the first place. I.e., why does Apple hate America, as the memes like to put it?

bequanna 5y ago

My friend, this was never about “data that causes harm to children”. This is a carefully chosen stated reason for Apple to add this functionality to avoid user pushback. The actual uses will vary from country to country, but on the whole will not result in the protection of anyone other than the government or ruling party. In countries with less civil freedom this will be exploited and used as a tool of oppression against minority political groups.

sharken 5y ago
I think this has been established already, but nevertheless it's best to repeat it. The on-device scanning capabilities will be expanded in the future, but it can be stopped if Apple care about their users. Right now it seems they don't care. Forgot to mention that I'll not be recommending or purchasing any Apple devices until they halt this scanning idea.
- bequanna 5y ago
  > but it can be stopped if Apple care about their users. Apple cares about selling devices, maximizing revenue. Users are just something required to authorize transactions. If Apple has to kowtow to the most oppressive regimes in the world to keep doing the above, you bet they will and probably already are. And since it is in the interest of both Apple + Gov't to do shady stuff in the dark, there is a good chance they can hide it the public indefinitely.

djbebs 5y ago

Your devices should not betray you, period no exceptions. I dont care if it will save billions of lives, your devices should not betray you.

totony 5y ago

I don't believe they should scan user data. User privacy is of utmost important (even more than slim risk of doing whatever). Do you want companies to have free reign scanning anything you use for "unauthorized" activities?

least 5y ago

No, they shouldn't scan data. Apple's role is not the police and they don't have the responsibility or moral imperative to scan for illegal imagery on their customers' phones. In fact it's the opposite. They should be in service to their customers and people's devices especially should be in service to its owner. On-device scanning violates this premise. There are things you can use technology for to help combat CSAM, but this is certainly not an acceptable way to go about it. Creating tools to help report abuse and illegal imagery in their software, for example, could be useful. This is not.

slg 5y ago
>Apple's role is not the police and they don't have the responsibility or moral imperative to scan for illegal imagery on their customers' phones What if the police come to Apple with a warrant? Should Apple have the responsibility to reveal the user's content to authorities? Part of the motivation for this type of thing is that E2EE effectively makes the old system of warrants obsolete and hides this data permanently from authorities. Lots of people see that as a huge win. Lots of people see that as a huge loss.
- least 5y ago
  > What if the police come to Apple with a warrant? Should Apple have the responsibility to reveal the user's content to authorities? If they lack the capability to do so then they don't need to worry about it. > Part of the motivation for this type of thing is that E2EE effectively makes the old system of warrants obsolete and hides this data permanently from authorities. Lots of people see that as a huge win. Lots of people see that as a huge loss. The speculation that this enables E2EE is just that; speculation. It's also not a strong argument because Apple could implement E2EE without snooping people's photographs and they'd be in the same position of being unable to assist law enforcement. If you have a system in place that intercepts all files before they're encrypted and uploaded to the cloud, then E2EE doesn't serve its purpose of ensuring privacy of people's data. It completely defeats the point.
  - slg 5y ago

varispeed 5y ago

The children are used as a manipulation tool - if you talk about CSAM, your videos are being demonetised and reach is being reduced and people don't want to be associated with it in any way or being attacked for opposing the idea of protecting children. If they said from the outset that they are going to scan phones for any illegal content, the push back would be much bigger. So now they have a wedge that they can gradually use to expand what is going to be scanned and when. This is a classic power move straight from 48 laws of power.

nullc 5y ago
Even searching for case law and public policy on this subject causes a chilling "WARNING CHILD ABUSE IMAGERY IS ILLEGAL" box to display on google. Speaking out enough on this gets you called a pedo. I will not go as far to claim that the people for this are intentionally using children to shield themselves from criticism push out a universal surveillance mechanism. ... but if you wanted to do exactly that, hiding behind the spectre of child abuse is exactly that. There are some hard questions we should be asking. Facebook claims to have reported some 20 million insances of child porn last year, but there were only a couple prosecutions. Are the matches fake? Is the goal really to just build a huge database of kompromat? If these images aren't a concern why are only an infinitesimal proportion of cases prosecuted?

dvt 5y ago

> I believe that they should scan user data in some capacity, because this is about data that causes harm to children. I think this is actually a pretty weak argument. Children may be vulnerable, but so are many other segments of the population, including those with disabilities, the elderly, and so on. I guess my question is if we're trying to protect children, then why not go all the way? The answer is because going all the way is dangerous: where does it stop? Why here but not there? I think a better argument would be: we should scan phones for X because X is illegal, and we've deemed, as a society, that X is wrong/bad/punishable/etc. But the problem with that argument is that we already have ways of checking phones/user data for X, and, thankfully, most have to go through a legal process of probable cause, obtaining warrants, and so on. Apple's proposal completely circumvents the legal system. This is a complicated question that I'm sure will be litigated in court in the next few years. SCOTUS has ruled that drug sniffing dogs (e.g. in airports) are constitutional, whereas using infrared imaging (e.g. for finding drug grow houses) is not. Are hashing algorithms more like drug sniffing dogs or like infrared imaging?

nonbirithm 5y ago
I'm not understanding how it completely circumvents the legal system if the law enforcement agencies are still contacted after the flag is triggered. But if the lack of a warrant is the problem then the main issue would be the third-party doctrine and having no reasonable expectation of privacy when entrusting your data with any company, not just Apple.
- xboxnolifes 5y ago
  > I'm not understanding how it completely circumvents the legal system if the law enforcement agencies are still contacted after the flag is triggered. Because, if I were to compare to what OP claims here, constant device scanning and then contacting law enforcement is like having a company constantly infrared scan your house for drug growing, and then contact the authorities about it. This is circumventing the authorities' legal limitations by moving the burden of responsibility to a company.

ak391 5y ago

Here’s a web demo[0] where you can try out any two images and see the resulting hashes, and whether there’s a collision. You can also try your own transformations (rotation, adding a filter, etc) on the image. Demo was built using Gradio[1]. [0]: https://huggingface.co/spaces/akhaliq/AppleNeuralHash2ONNX https://huggingface.co/spaces/akhaliq/AppleNeuralHash2ONNX [1]: https://gradio.dev https://gradio.dev

tw04 5y ago

> I believe that they should scan user data in some capacity, because this is about data that causes harm to children. Should GM use onstar to record every conversation you have in your car and use AI to flag conversations that relate to child abuse? After all it involves children. And if down the road law enforcement decides to expand what gets flagged, you shouldn’t care if you aren’t breaking any laws, right? This is a horrible idea and it’s a matter of if, not when it is abused.

mfer 5y ago

> I believe that they should scan user data in some capacity, because this is about data that causes harm to children. Google, Facebook, and others are already scanning the images when uploaded to them. We should not be surprised that the images we upload are being scanned. Sometimes for CSAM, sometimes to train ML, and sometimes for other things. Apple doing scanning images that were uploaded to their servers would just be another company doing the same thing. If I understand things correctly, these companies even have some legal obligations to do some of this. > However, I believe that they should not run the scan on the device, because that carries significant drawbacks for personal privacy. This is about where Apple is doing the scanning. Doing it in icloud is on them. Doing it on our phones is an invasion of privacy. And, it builds a system that can be used for other things. It opens a door, as many others have pointed out. I wonder if Apple did all of this to scale the implementation. So it wouldn't have to run on their servers. They wouldn't have to maintain and power them. Poor implementation if that's the case.

yrral 5y ago

> If not, what is going to convince them to stop at this point? Maybe if masses of people go into apple stores and save photos on demo phones that collide with the files they're looking for. Do those phones get scanned? Will apple ban their own app store accounts?

craftinator 5y ago

I wonder when they'll add this feature to MacOS?

DavideNL 5y ago

> should Apple scan user data or should they not? I guess it depends on whether the system is 100% safe and cannot be exploited; Since only iCloud photos are scanned, and other cloud storage providers already do this and have been scanning their users photos for a long time. I haven’t heard anyone complain when Google started scanning their cloud photos, which is kind of… interesting.

lisper 5y ago

> this is about data that causes harm to children. The data is not what causes the harm. It is the method used to produce (some of) the data that causes the harm. And the problem is that looking for the data in order to find the people causing the harm runs afoul of the converse fallacy. It's the same fallacy that causes people to think that because most terrorists (that they know of) are Muslims, that most Muslims must be terrorists. Most child abusers have photos of naked children on their phones, therefore most people who have photos of naked children on their phones must be child abusers. It's false in both cases. A friend of mine, for example, has photos of her naked child on her phone that she sent to the doctor to help diagnose a rash.

throwaway81523 5y ago

> I believe that they should scan user data in some capacity, because this is about data that causes harm to children. The legal system already provides a means for invading people's privacy given certain criteria, whether by scanning their phones, physically searching their residences and offices, or whatever. The criterion is "probable cause" and the search is launched by a judge signing a warrant that the probable cause exists. That is how the Framers wrote it into the US Constitution (see Amendment 4), because they had dealt with too many bogus searches done by the Colonial governments and wanted to put a stop to that. Things have not really changed since then. "Think of the children!!!" is a traditional thought-stopper, but really, if there is substantive reason to search someone's phone, just convince a judge to sign a warrant and everything will be fine. Searching people's stuff when they are not under any concrete suspicion is part of why we had a revolution in this country in the first place. I.e., why does Apple hate America, as the memes like to put it?