7 ms·

> This invasive capability on the device level is a massive intrusion on everyone's privacy I am a user uploading a photo to the cloud. Google's servers scan

> This invasive capability on the device level is a massive intrusion on everyone's privacy I am a user uploading a photo to the cloud. Google's servers scan photos after they are uploaded to the cloud, and have done so for the past decade. >a man [was] arrested on child pornography charges, after Google tipped off authorities about illegal images found in the Houston suspect's Gmail account https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-led-to-a-mans-arrest-for-child-porn-was-not-a-privacy-violation/ https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le... Apple wants the device to scan the photo before it is uploaded to the cloud. Both are equally invasive and both can only be avoided by refusing to use the cloud.

hef19898 5y ago
But only Apple can, at least for now, scan content or your phone. Google cannot. Yet. Small, bit crucial difference.
- GeekyBear 5y ago
  Frankly, it's more of a problem for my data to be flagged after Google has it than it would be for it to never leave my device at all. People have certainly run afoul of the law when the data Google has about them is misused. >Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime https://www.dailymail.co.uk/news/article-7897319/Police-arrested-innocent-man-murder-using-Google-location-data.html https://www.dailymail.co.uk/news/article-7897319/Police-arre... We are only talking about what happens when you try to send the data to the cloud, after all. Not about all the data on the device.
  - hef19898 5y ago
    You are so close to understand the issue. When Google, or anyone else, is flagging it and sharing the result with, e.g., law enforcement you can have a problem. Apple will no just simply refuse to upload stuff that would be flagged in the cloud, it is flagging it already before upload on your device. And share the results with authorities. That for now they claim to only do so with stuff about to be uploaded is of minor concern, the tech is there to scan all the pictures on an iPhone now. No reason to believe it will stop with intended uploads, or pictures.
    - GeekyBear 5y ago
      > Apple will no just simply refuse to upload stuff that would be flagged in the cloud, it is flagging it already before upload on your device. And share the results with authorities. None of this is true. A sibling comment describes the process: >1. Only if you're uploading files are the files matched. 2. Only if the matches are very close are they considered matches. 3. Only if you have multiple very close matches is Apple able to decrypt the low-res versions of the images themselves. 4. Only if a human reviewer discovers any of the decrypted low-res images to be illegal content is any of your information shared with anyone else. https://news.ycombinator.com/item?id=28120598 https://news.ycombinator.com/item?id=28120598
- aj3 5y ago
  On the other hand, Google can run arbitrary queries over your data (published to their cloud) while Apple uses the same algorithm and hash database for everyone (which probably will get reverse engineered and audited by next BlackHat).
  - michaelmrose 5y ago
    You can't realistically reverse engineer a hash to source image.
    - aj3 5y ago
      1. Previous submission (http://www.hackerfactor.com/blog/ http://www.hackerfactor.com/blog/) claims that PhotoDNA is reversible to 26x26 grayscale 2. What’s more important is you can reverse engineer algorithm that’s used locally to compare these hashes. Unlike with cloud-based searches other providers run, this puts significant limitations to what Apple or LEA can do on your phone. E.g. they can’t extract strings from PDF or grep through image metadata or request all images of certain users without knowing what these images would depict in advance. The frequency of hash database updates even if opaque will still provide more insight into what is being searched for compared to current status quo where public has zero insights because all search / aggregation queries are run on the server.
  - jdiez17 5y ago
    What makes you think that Apple will use the same database for everyone?
- duped 5y ago
  I think so too but from a different angle. It sounds like the federal government is strong arming Apple to spy on their users. Apple would rather use on-device hashing to preserve some amount of privacy rather than give unfettered access to the feds to their cloud.
- dev_tty01 5y ago
  Really? Apple and Google have been able to scan content on your phone as long as they have sold phones. They control the OS, therefore they have access to the content. The ability has always been there. The only difference is that Apple has now openly announced what they are going to do and how. If governments wanted to pressure Apple (or Google) to scan content they could have done that anytime in the last few years. If Apple and Google resisted that before, why is now different?
  - stickfigure 5y ago
    From a legal perspective, there is a significant chasm between "you are hereby ordered to develop a whole new surveillance feature for devices you build" and "you are hereby ordered to add these new (potentially unrelated to CSAM) signatures to your existing surveillance network".
    - GeekyBear 5y ago
      Under what legal basis are certain corporations immune to government demands while others are not? Google does reporting on how many National Security Letters requesting information about it's users that it responds to annually. Why doesn't it just ignore them?
      - stickfigure 5y ago
        I don't understand your question. Google, Apple, Facebook et al comply with their legal obligations. Apple does not have a legal obligation to install spyware on your phone (at least not in the US, not yet).
      - heavyset_go 5y ago
        > Google does reporting on how many National Security Letters requesting information about it's users that it responds to annually. So does Apple[1]. Apple gave data on over 31,000 users/accounts based on National Security Letters in the first half of 2020 alone[1]. > Why doesn't it just ignore them? In the first half of 2020, Apple provided data to the US government (non-FISA or NSL) about 9,000 times, and responded to requests for data with the data about ~85% of the time, and 92% in cases of "emergencies"[1]. [1] https://www.apple.com/legal/transparency/us.html https://www.apple.com/legal/transparency/us.html
- artursapek 5y ago
  Apple can already find photos of my cat, of ice cream, and of the sky directly on my phone using AI
- veratwiger 5y ago
  My understanding is they only scan for content if it's being uploaded to iCloud. Scanning on the phone, yes, but only if the image is to be sent over the network.
  - handrous 5y ago
    And they seem to be doing that because they both 1) want to enable e2e encryption for iCloud images, soonish, and 2) don't want to become the world's most popular child porn storage and sharing platform. I get the arguments for why the fix for #2 is too dangerous to be worth it, but also get why they'd be inclined to choose it.
meterplech 5y ago
There are many photos one could take without uploading or saving them in the cloud. The question is can I safely store things locally on my device without a multi-national corporation being able to set the policy of what's allowed there. The article cites examples of banned memes/images by the Chinese government. There is also the risk of algorithm mistakes. Without the trust + safety team or any cloud verification, does this just get immediately forward to the FBI? What is the process to be made non-guilty if the Apple algorithm thinks you have something illegal and you don't if there is no human in the loop involved before it's forwarded to authorities? At least if you upload to a service someone on the trust + safety team can verify the algo worked properly. I would be very nervous building this kind of system of the risk of false positives! Maybe even so nervous to introduce many more false negatives - which is in itself terrible in the case of something like CSAM.
- GeekyBear 5y ago
  > The question is can I safely store things locally on my device without a multi-national corporation being able to set the policy of what's allowed there. Didn't Microsoft just start removing people's torrent clients from their computer without permission and announce that you wouldn't be allowed to use your own PC running Windows 11 Home without using an online Microsoft account to log in?
- danudey 5y ago
  Here's how it works, in short: 1. Images which are going to be uploaded to iCloud are matched against known hashes. A match has to be very close in order to actually be flagged as a match. The results are uploaded to iCloud Photos along with your image. 2. If your account hits a sufficient threshold of matches (i.e. you have to have multiple/many images matched) then Apple is alerted and they are able to decrypt low-resolution versions of the relevant stored images. 3. Apple's team performs a manual review of the images to ensure that they are in fact inappropriate images and not false positives. 4. If any of the matched images are actually illegal, then Apple reports you to the authorities. In other words, there are several safeguards in this process: 1. Only if you're uploading files are the files matched. 2. Only if the matches are very close are they considered matches. 3. Only if you have multiple very close matches is Apple able to decrypt the low-res versions of the images themselves. 4. Only if a human reviewer discovers any of the decrypted low-res images to be illegal content is any of your information shared with anyone else. In my opinion, this is a vastly better system than literally any other cloud provider at the moment; it allows Apple to find and report even more CSAM with effectively no risk of reporting false-positives to law enforcement, while also preserving your privacy through encryption in transit and at rest.
Clubber 5y ago
>Both are equally invasive and both can only be avoided by refusing to use the cloud. Technically Apple can do a scan regardless of wether or not you posted it to iCloud if they so desired to, or were ordered to.
- GeekyBear 5y ago
  Technically, Google can sell the entire trove of data they have collected about you to anyone willing to write them a check. Do you think it's likely they will do so?
  - Clubber 5y ago
    The difference is you don't put your data in Google's cloud, they can't sell it. With this Apple change on your local device, you don't have any recourse other than get a different phone. Also, Google sells your data already don't they? Probably not a great counterpoint. https://www.tampabay.com/news/2021/05/07/google-selling-users-personal-data-despite-promise-federal-court-lawsuit-claims/ https://www.tampabay.com/news/2021/05/07/google-selling-user...
    - GeekyBear 5y ago
      Again, the only thing different here is that when you upload a photo to the cloud, Apple scans it on device before the file is uploaded, while Google scans it on their server after it is uploaded. Nothing is happening to files you don't attempt to upload to the cloud. Also, if Google is selling everyone's data to third party data brokers, it's news to me.
      - Clubber 5y ago
        >Nothing is happening to files you don't attempt to upload to the cloud. I am aware, but because the scanner now exists on the device, they have the capability of scanning images on your device regardless of whether or not you upload it to the cloud if they so choose to OR ARE ORDERED TO. Do you not see the difference in capability?
gigel82 5y ago
A webpage does authentication by downloading a list of usernames and passwords and comparing your inputs in JavaScript in your browser. Another sends a hash out to an endpoint in the cloud. To the user - using your line of reasoning - they're both the same, right? This isn't supposed to be a technical comparison, I just want to point out that the implementation matters as much (or more) than the customers' experience.
BiteCode_dev 5y ago
I don't use the cloud. I don't even sign in an email address on my android phone. That makes all the difference to me.

hef19898 5y ago

But only Apple can, at least for now, scan content or your phone. Google cannot. Yet. Small, bit crucial difference.

GeekyBear 5y ago
Frankly, it's more of a problem for my data to be flagged after Google has it than it would be for it to never leave my device at all. People have certainly run afoul of the law when the data Google has about them is misused. >Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime https://www.dailymail.co.uk/news/article-7897319/Police-arrested-innocent-man-murder-using-Google-location-data.html https://www.dailymail.co.uk/news/article-7897319/Police-arre... We are only talking about what happens when you try to send the data to the cloud, after all. Not about all the data on the device.
- hef19898 5y ago
  You are so close to understand the issue. When Google, or anyone else, is flagging it and sharing the result with, e.g., law enforcement you can have a problem. Apple will no just simply refuse to upload stuff that would be flagged in the cloud, it is flagging it already before upload on your device. And share the results with authorities. That for now they claim to only do so with stuff about to be uploaded is of minor concern, the tech is there to scan all the pictures on an iPhone now. No reason to believe it will stop with intended uploads, or pictures.
  - GeekyBear 5y ago
    > Apple will no just simply refuse to upload stuff that would be flagged in the cloud, it is flagging it already before upload on your device. And share the results with authorities. None of this is true. A sibling comment describes the process: >1. Only if you're uploading files are the files matched. 2. Only if the matches are very close are they considered matches. 3. Only if you have multiple very close matches is Apple able to decrypt the low-res versions of the images themselves. 4. Only if a human reviewer discovers any of the decrypted low-res images to be illegal content is any of your information shared with anyone else. https://news.ycombinator.com/item?id=28120598 https://news.ycombinator.com/item?id=28120598
aj3 5y ago
On the other hand, Google can run arbitrary queries over your data (published to their cloud) while Apple uses the same algorithm and hash database for everyone (which probably will get reverse engineered and audited by next BlackHat).
- michaelmrose 5y ago
  You can't realistically reverse engineer a hash to source image.
  - aj3 5y ago
    1. Previous submission (http://www.hackerfactor.com/blog/ http://www.hackerfactor.com/blog/) claims that PhotoDNA is reversible to 26x26 grayscale 2. What’s more important is you can reverse engineer algorithm that’s used locally to compare these hashes. Unlike with cloud-based searches other providers run, this puts significant limitations to what Apple or LEA can do on your phone. E.g. they can’t extract strings from PDF or grep through image metadata or request all images of certain users without knowing what these images would depict in advance. The frequency of hash database updates even if opaque will still provide more insight into what is being searched for compared to current status quo where public has zero insights because all search / aggregation queries are run on the server.
- jdiez17 5y ago
  What makes you think that Apple will use the same database for everyone?
duped 5y ago
I think so too but from a different angle. It sounds like the federal government is strong arming Apple to spy on their users. Apple would rather use on-device hashing to preserve some amount of privacy rather than give unfettered access to the feds to their cloud.
dev_tty01 5y ago
Really? Apple and Google have been able to scan content on your phone as long as they have sold phones. They control the OS, therefore they have access to the content. The ability has always been there. The only difference is that Apple has now openly announced what they are going to do and how. If governments wanted to pressure Apple (or Google) to scan content they could have done that anytime in the last few years. If Apple and Google resisted that before, why is now different?
- stickfigure 5y ago
  From a legal perspective, there is a significant chasm between "you are hereby ordered to develop a whole new surveillance feature for devices you build" and "you are hereby ordered to add these new (potentially unrelated to CSAM) signatures to your existing surveillance network".
  - GeekyBear 5y ago
    Under what legal basis are certain corporations immune to government demands while others are not? Google does reporting on how many National Security Letters requesting information about it's users that it responds to annually. Why doesn't it just ignore them?
    - stickfigure 5y ago
      I don't understand your question. Google, Apple, Facebook et al comply with their legal obligations. Apple does not have a legal obligation to install spyware on your phone (at least not in the US, not yet).
    - heavyset_go 5y ago
      > Google does reporting on how many National Security Letters requesting information about it's users that it responds to annually. So does Apple[1]. Apple gave data on over 31,000 users/accounts based on National Security Letters in the first half of 2020 alone[1]. > Why doesn't it just ignore them? In the first half of 2020, Apple provided data to the US government (non-FISA or NSL) about 9,000 times, and responded to requests for data with the data about ~85% of the time, and 92% in cases of "emergencies"[1]. [1] https://www.apple.com/legal/transparency/us.html https://www.apple.com/legal/transparency/us.html
artursapek 5y ago
Apple can already find photos of my cat, of ice cream, and of the sky directly on my phone using AI
veratwiger 5y ago
My understanding is they only scan for content if it's being uploaded to iCloud. Scanning on the phone, yes, but only if the image is to be sent over the network.
- handrous 5y ago
  And they seem to be doing that because they both 1) want to enable e2e encryption for iCloud images, soonish, and 2) don't want to become the world's most popular child porn storage and sharing platform. I get the arguments for why the fix for #2 is too dangerous to be worth it, but also get why they'd be inclined to choose it.

meterplech 5y ago

There are many photos one could take without uploading or saving them in the cloud. The question is can I safely store things locally on my device without a multi-national corporation being able to set the policy of what's allowed there. The article cites examples of banned memes/images by the Chinese government. There is also the risk of algorithm mistakes. Without the trust + safety team or any cloud verification, does this just get immediately forward to the FBI? What is the process to be made non-guilty if the Apple algorithm thinks you have something illegal and you don't if there is no human in the loop involved before it's forwarded to authorities? At least if you upload to a service someone on the trust + safety team can verify the algo worked properly. I would be very nervous building this kind of system of the risk of false positives! Maybe even so nervous to introduce many more false negatives - which is in itself terrible in the case of something like CSAM.

GeekyBear 5y ago
> The question is can I safely store things locally on my device without a multi-national corporation being able to set the policy of what's allowed there. Didn't Microsoft just start removing people's torrent clients from their computer without permission and announce that you wouldn't be allowed to use your own PC running Windows 11 Home without using an online Microsoft account to log in?
danudey 5y ago
Here's how it works, in short: 1. Images which are going to be uploaded to iCloud are matched against known hashes. A match has to be very close in order to actually be flagged as a match. The results are uploaded to iCloud Photos along with your image. 2. If your account hits a sufficient threshold of matches (i.e. you have to have multiple/many images matched) then Apple is alerted and they are able to decrypt low-resolution versions of the relevant stored images. 3. Apple's team performs a manual review of the images to ensure that they are in fact inappropriate images and not false positives. 4. If any of the matched images are actually illegal, then Apple reports you to the authorities. In other words, there are several safeguards in this process: 1. Only if you're uploading files are the files matched. 2. Only if the matches are very close are they considered matches. 3. Only if you have multiple very close matches is Apple able to decrypt the low-res versions of the images themselves. 4. Only if a human reviewer discovers any of the decrypted low-res images to be illegal content is any of your information shared with anyone else. In my opinion, this is a vastly better system than literally any other cloud provider at the moment; it allows Apple to find and report even more CSAM with effectively no risk of reporting false-positives to law enforcement, while also preserving your privacy through encryption in transit and at rest.

Clubber 5y ago

>Both are equally invasive and both can only be avoided by refusing to use the cloud. Technically Apple can do a scan regardless of wether or not you posted it to iCloud if they so desired to, or were ordered to.

GeekyBear 5y ago
Technically, Google can sell the entire trove of data they have collected about you to anyone willing to write them a check. Do you think it's likely they will do so?
- Clubber 5y ago
  The difference is you don't put your data in Google's cloud, they can't sell it. With this Apple change on your local device, you don't have any recourse other than get a different phone. Also, Google sells your data already don't they? Probably not a great counterpoint. https://www.tampabay.com/news/2021/05/07/google-selling-users-personal-data-despite-promise-federal-court-lawsuit-claims/ https://www.tampabay.com/news/2021/05/07/google-selling-user...
  - GeekyBear 5y ago
    Again, the only thing different here is that when you upload a photo to the cloud, Apple scans it on device before the file is uploaded, while Google scans it on their server after it is uploaded. Nothing is happening to files you don't attempt to upload to the cloud. Also, if Google is selling everyone's data to third party data brokers, it's news to me.
    - Clubber 5y ago
      >Nothing is happening to files you don't attempt to upload to the cloud. I am aware, but because the scanner now exists on the device, they have the capability of scanning images on your device regardless of whether or not you upload it to the cloud if they so choose to OR ARE ORDERED TO. Do you not see the difference in capability?

gigel82 5y ago

A webpage does authentication by downloading a list of usernames and passwords and comparing your inputs in JavaScript in your browser. Another sends a hash out to an endpoint in the cloud. To the user - using your line of reasoning - they're both the same, right? This isn't supposed to be a technical comparison, I just want to point out that the implementation matters as much (or more) than the customers' experience.

BiteCode_dev 5y ago

I don't use the cloud. I don't even sign in an email address on my android phone. That makes all the difference to me.