6 ms·

The thing that's shocking to me is that Google, Microsoft and all the big names in tech have been scanning everything in your account (email, cloud drive, phot

The thing that's shocking to me is that Google, Microsoft and all the big names in tech have been scanning everything in your account (email, cloud drive, photos, etc) for the past decade, without any noticeable uproar. Apple announces that it is going to start scanning iCloud Photos only, and that their system is set to ignore anything below a threshold of ~30 positives before triggering a human review, and people lose their minds.

rblatz 5y ago
Because it isn’t about the CSAM scanning. Apple keeps reframing the argument back to that. most people expect that google and Microsoft to scan files on their servers for CSAM, people object to Apple turning your phone against its owner, and once building out the capability to do this only policy prevents it from being abused.
websites2023 5y ago
BigCos, take note: you’re better off doing nefarious shit without telling anyone. Because, if you come clean, you’ll only invite an endless parade of bloggers who will misconstrue your technology to make you look bad.
- justin_oaks 5y ago
  It's important to keep nefarious stuff on the server side because eventually someone will reverse engineer what's on the client side. Imagine if Apple had done this on the client side without telling anyone, and later it was discovered. I think things would be a whole worse for Apple in that case.
  - websites2023 5y ago
    That being the case, why do it client side at all, when presumably every claim they make is verifiable?
  - squarefoot 5y ago
    Devices with proprietary OSes spend more and more time phoning home, then exchanging data officially for "updates". They probably tell the truth, but should one of them decide to hide users data exfiltration or other monitoring practices behind those updates, it would be quite hard to catch them. In other words we have no way to tell that they're not already doing this.
- __blockcipher__ 5y ago
  No misconstrual needed. This technology is genuinely bad. It scans images against an arbitrary government-owned black-box database. There’s no guarantee that it’s only CSAM.
  - websites2023 5y ago
    NECMEC isn’t owned by the government and the database of hashes is available from Apple.
    - tjfl 5y ago
      They're not owned by the federal government, but they do get a lot of federal government money. > The National Center for Missing & Exploited Children® was established in 1984 as a private, nonprofit 501(c)(3) organization. Today, NCMEC performs the following 15 specific programs of work, funded in part by federal grants (34 U.S.C. § 11293): Source: https://www.missingkids.org/footer/about https://www.missingkids.org/footer/about US DOJ OJJDP lists recent grants totaling $84,446,366 in FY19 and FY20. Source: https://ojjdp.ojp.gov/funding/awards/list?awardee=NATIONAL%20CENTER%20FOR%20MISSING%20AND%20EXPLOITED%20CHILDREN%2C%20THE https://ojjdp.ojp.gov/funding/awards/list?awardee=NATIONAL%2...
      - __blockcipher__ 5y ago
        And don’t forget, it’s way more than just money: https://www.law.cornell.edu/uscode/text/18/2258A https://www.law.cornell.edu/uscode/text/18/2258A You must report to them and only them. For the GP to claim they’re not government “owned” is a rhetorical trick at best and outright ignorant absurdity at worst.
    - commoner 5y ago
      > NECMEC isn’t owned by the government Even though NCMEC describes itself as "private", it was established by and has been heavily funded by the U.S. government. From an archive of NCMEC's own history page, cited on Wikipedia (https://web.archive.org/web/20121029010231/http://www.missingkids.com/missingkids/servlet/PageServlet?LanguageCountry=en_US&PageId=4362 https://web.archive.org/web/20121029010231/http://www.missin...): > In 1984, the U.S. Congress passed the Missing Children’s Assistance Act which established a National Resource Center and Clearinghouse on Missing and Exploited Children. The National Center for Missing & Exploited Children was designated to fulfill this role. > On June 13, 1984, the National Center for Missing & Exploited Children was opened by President Ronald Reagan in a White House Ceremony. The national 24-hour toll-free missing children’s hotline 1-800-THE-LOST opened as well. $40 million/year of U.S. government funding from a 2013 bill (https://en.wikipedia.org/wiki/Missing_Children%27s_Assistance_Reauthorization_Act_of_2013 https://en.wikipedia.org/wiki/Missing_Children%27s_Assistanc...): > The Missing Children's Assistance Reauthorization Act of 2013 (H.R. 3092) is a bill that was introduced into the United States House of Representatives during the 113th United States Congress. The Missing Children's Assistance Reauthorization Act of 2013 reauthorizes the Missing Children's Assistance Act and authorizes $40 million a year to fund the National Center for Missing and Exploited Children.
OnlineGladiator 5y ago
The other companies didn't advertise and pride themselves on being privacy focused. Part of the appeal of Apple was that you could avoid that issue and they touted it regularly. Now they're telling their customers to go fuck themselves (so long as they're 18 or older).
- GeekyBear 5y ago
  Conducting the scan on the user's device instead of on the companies server is more private. Apple can't decrypt the results of the scan until the ~30 image threshold is crossed and a human review is triggered. Given Google's reluctance to hire humans when a poorly performing algorithm is cheaper, are they turning over every single false positive without a human review?
  - deleted 5y ago
    [deleted]
  - lifty 5y ago
    I guess people had a sense of ownership of these devices and they feel that it’s doing some they they don’t want. If ownership means control, maybe in the digital age full ownership/control is not really possible on a managed device like the iPhone. There are other example like the John Deer tractor issues.
    - GeekyBear 5y ago
      I have a sense of ownership over my non-publicly-accessible data when it's backed up to a cloud drive. Apple isn't scanning that. Google and Microsoft are.
      - lifty 5y ago
        Sure, you own the data, but you don’t have any control on what’s happening to it. Just like iPhone users, we own the physical device but we only control certain aspects of what it’s doing, and the boundaries are not set by us, but by Apple.
heavyset_go 5y ago
This seems like a common deflection, but get back to me when either company puts programs in my pocket that scan my data for crimes and snitch on me to authorities.
- bobthepanda 5y ago
  At least Google does. https://protectingchildren.google/intl/en/ https://protectingchildren.google/intl/en/ > CSAI Match is our proprietary technology, developed by the YouTube team, for combating child sexual abuse imagery (CSAI) in video content online. It was the first technology to use hash-matching to identify known violative videos and allows us to identify this type of violative content amid a high volume of non-violative video content. When a match of violative content is found, it is then flagged to partners to responsibly report in accordance to local laws and regulations. Through YouTube, we make CSAI Match available for free to NGOs and industry partners like Adobe, Reddit, and Tumblr, who use it to counter the spread of online child exploitation videos on their platforms as well. > We devote significant resources—technology, people, and time—to detecting, deterring, removing, and reporting child sexual exploitation content and behavior. Since 2008, we’ve used “hashing” technology, which creates a unique digital ID for each known child sexual abuse image, to identify copies of images on our services that may exist elsewhere.
  - arsome 5y ago
    Note that they're not pushing it to devices that consumers have purchased, just running it on their cloud services. I think it's an important distinction here with what Apple is doing.
  - heavyset_go 5y ago
    No, that happens on Google's servers and isn't an agent in my pocket that runs on and invades my personal property. It's also only for videos.
    - bobthepanda 5y ago
      The images being scanned are the ones that go to iCloud. The Google page has a section later down that also says they use hashing of images.
brandon272 5y ago
Here's the reason for the sudden uproar: The scanning of things on third party servers was just barely tolerated by a lot of people, whether it's for advertising purposes or government intrusion. People accept it because it's considered reasonable for these third parties to scan data in exchange for providing a service for free (e.g. Google), or because they need to have some degree of accountability for what is on their servers. When that scanning gets moved from the cloud to being on your device, a boundary is violated. When that boundary is violated by a company who makes extreme privacy claims like saying that privacy is a "fundamental human right"[1], yes, people will "lose their minds" over it. This shouldn't be shocking at all. [1] https://apple.com/privacy https://apple.com/privacy
- GeekyBear 5y ago
  When the scanning gets moved from the cloud to being on device, Apple itself cannot see the results of the scan until the risk that the result is only a false positive is greatly reduced. You would have to have 30 false positives before Apple can see anything, which is unlikely, but the next step is still a human review, since it's not impossible.
  - OnlineGladiator 5y ago
    I don't care. It's my device (at least that's how Apple used to advertise it) and I disagree with Apple's policy, full stop. I don't care about "think of the children" (especially since they will be scanning pictures of my own children), and furthermore I don't trust Apple not to change the policy in the future. They've created a backdoor and now the device is compromised and privacy is broken. If you want to trust Apple go ahead. They have eroded my trust and I doubt they could regain it within a decade.
    - GeekyBear 5y ago
      >I don't care If anything, you should be outraged that Google and Microsoft have been scanning much more of your data, and doing so in a much more intrusive way. Apple only scans iCloud Photos and they do so in a way that they can't see the results until they can be reasonably sure it's not just a false positive.
mightybyte 5y ago
I think the difference here is that it's ON YOUR DEVICE. I think there's a pretty clear understanding that if you upload stuff to a cloud provider they can do whatever they want with it. This is different. This is reaching into what has up until now mostly been considered a private place. Law enforcement often has to get warrants to search this kind of thing. This is the difference between putting CSAM on a sign in your front yard (maybe not quite front yard but I can't come up with quite the same physical equivalent to a cloud provider) and keeping it in a password protected vault in your basement. One of those things is protected in the U.S. by laws against unlawful search and seizure. Cloud and on your device are two very different things and consumers are right to be alarmed. I'll say it again, if you are concerned with this privacy violation, sell your Apple stock and categorically refuse to purchase Apple devices. Also go to https://www.nospyphone.com/ https://www.nospyphone.com/ and make your voice heard there.
- GeekyBear 5y ago
  ON YOUR DEVICE (where it's encrypted in a way that Apple can't read until the 30 image threshold is crossed) is more private than doing the same scan on server where a single false positive can be misused by anyone who can get a subpoena.
  - pasquinelli 5y ago
    what kind of encryption can't be decrypted until some threshold has been crossed? maybe you mean to say that apple says they won't read it until that threshold has been crossed.
    - GeekyBear 5y ago
      No, I'm saying they designed the system so that they don't have the key to decypt the scan result "vouchers" until after the device tells them that the 30 image threshold is crossed. >Apple is unable to process individual vouchers; instead, all the properties of our system mean that it’s only once an account has accumulated a collection of vouchers associated with illegal, known CSAM images that we are able to learn anything about the user’s account. Now, why to do it is because, as you said, this is something that will provide that detection capability while preserving user privacy. https://techcrunch.com/2021/08/10/interview-apples-head-of-privacy-details-child-abuse-detection-and-messages-safety-features/ https://techcrunch.com/2021/08/10/interview-apples-head-of-p... Meanwhile, a single false positive from an on server scan is open to malicious use by anyone who can get a subpeona.
Copernicron 5y ago
Most people don't pay attention to whether or not their stuff is being scanned. Those of us who do pay attention have known for a long time that it's being scanned. Especially by Google and Facebook. Basically anyone whose business model is based off of advertising. My default assumption is anything I upload is scanned.
GuB-42 5y ago
Google is open about the fact they scan and track everything, they even make it a point of convenience "hey, we looked at your email and found a plane ticket, based on where you are, you should leave in 2 hours if you want to be at the airport on time". Facebook, even more so, they are explicitly anti-privacy to the point of being insulting. Microsoft will happily show you everything they may send when you install Windows, you can sometimes refuse, but not always. They are a bit less explicit than Google, but privacy is rarely on the menu. As for Amazon, their cloud offers are mostly for businesses, different market, but still, for consumers, they don't really insist on privacy either. So that if any of these company scan your pictures for child porn, it won't shock anyone, because we know it is what they do. But Apple claims privacy as a core value, half of their ads are along the lines of "we are not like the others, we respect your privacy, everything on your device stays on your device, etc...", they announce every (often legitimate) privacy feature with great fanfare, etc... So much that people start to believe it. But with that, people realize that Apple is not so different from the others after all, and if they bought an overpriced device based on that promise, I understand why they are pissed off.
xdennis 5y ago
> all the big names in tech have been scanning everything in your account [...] without any noticeable uproar In part because people didn't know. And if you were one of innocent people caught by them, you wouldn't want people to know.

rblatz 5y ago

Because it isn’t about the CSAM scanning. Apple keeps reframing the argument back to that. most people expect that google and Microsoft to scan files on their servers for CSAM, people object to Apple turning your phone against its owner, and once building out the capability to do this only policy prevents it from being abused.

websites2023 5y ago

BigCos, take note: you’re better off doing nefarious shit without telling anyone. Because, if you come clean, you’ll only invite an endless parade of bloggers who will misconstrue your technology to make you look bad.

justin_oaks 5y ago
It's important to keep nefarious stuff on the server side because eventually someone will reverse engineer what's on the client side. Imagine if Apple had done this on the client side without telling anyone, and later it was discovered. I think things would be a whole worse for Apple in that case.
- websites2023 5y ago
  That being the case, why do it client side at all, when presumably every claim they make is verifiable?
- squarefoot 5y ago
  Devices with proprietary OSes spend more and more time phoning home, then exchanging data officially for "updates". They probably tell the truth, but should one of them decide to hide users data exfiltration or other monitoring practices behind those updates, it would be quite hard to catch them. In other words we have no way to tell that they're not already doing this.
__blockcipher__ 5y ago
No misconstrual needed. This technology is genuinely bad. It scans images against an arbitrary government-owned black-box database. There’s no guarantee that it’s only CSAM.
- websites2023 5y ago
  NECMEC isn’t owned by the government and the database of hashes is available from Apple.
  - tjfl 5y ago
    They're not owned by the federal government, but they do get a lot of federal government money. > The National Center for Missing & Exploited Children® was established in 1984 as a private, nonprofit 501(c)(3) organization. Today, NCMEC performs the following 15 specific programs of work, funded in part by federal grants (34 U.S.C. § 11293): Source: https://www.missingkids.org/footer/about https://www.missingkids.org/footer/about US DOJ OJJDP lists recent grants totaling $84,446,366 in FY19 and FY20. Source: https://ojjdp.ojp.gov/funding/awards/list?awardee=NATIONAL%20CENTER%20FOR%20MISSING%20AND%20EXPLOITED%20CHILDREN%2C%20THE https://ojjdp.ojp.gov/funding/awards/list?awardee=NATIONAL%2...
    - __blockcipher__ 5y ago
      And don’t forget, it’s way more than just money: https://www.law.cornell.edu/uscode/text/18/2258A https://www.law.cornell.edu/uscode/text/18/2258A You must report to them and only them. For the GP to claim they’re not government “owned” is a rhetorical trick at best and outright ignorant absurdity at worst.
  - commoner 5y ago
    > NECMEC isn’t owned by the government Even though NCMEC describes itself as "private", it was established by and has been heavily funded by the U.S. government. From an archive of NCMEC's own history page, cited on Wikipedia (https://web.archive.org/web/20121029010231/http://www.missingkids.com/missingkids/servlet/PageServlet?LanguageCountry=en_US&PageId=4362 https://web.archive.org/web/20121029010231/http://www.missin...): > In 1984, the U.S. Congress passed the Missing Children’s Assistance Act which established a National Resource Center and Clearinghouse on Missing and Exploited Children. The National Center for Missing & Exploited Children was designated to fulfill this role. > On June 13, 1984, the National Center for Missing & Exploited Children was opened by President Ronald Reagan in a White House Ceremony. The national 24-hour toll-free missing children’s hotline 1-800-THE-LOST opened as well. $40 million/year of U.S. government funding from a 2013 bill (https://en.wikipedia.org/wiki/Missing_Children%27s_Assistance_Reauthorization_Act_of_2013 https://en.wikipedia.org/wiki/Missing_Children%27s_Assistanc...): > The Missing Children's Assistance Reauthorization Act of 2013 (H.R. 3092) is a bill that was introduced into the United States House of Representatives during the 113th United States Congress. The Missing Children's Assistance Reauthorization Act of 2013 reauthorizes the Missing Children's Assistance Act and authorizes $40 million a year to fund the National Center for Missing and Exploited Children.

OnlineGladiator 5y ago

The other companies didn't advertise and pride themselves on being privacy focused. Part of the appeal of Apple was that you could avoid that issue and they touted it regularly. Now they're telling their customers to go fuck themselves (so long as they're 18 or older).

GeekyBear 5y ago
Conducting the scan on the user's device instead of on the companies server is more private. Apple can't decrypt the results of the scan until the ~30 image threshold is crossed and a human review is triggered. Given Google's reluctance to hire humans when a poorly performing algorithm is cheaper, are they turning over every single false positive without a human review?
- deleted 5y ago
  [deleted]
- lifty 5y ago
  I guess people had a sense of ownership of these devices and they feel that it’s doing some they they don’t want. If ownership means control, maybe in the digital age full ownership/control is not really possible on a managed device like the iPhone. There are other example like the John Deer tractor issues.
  - GeekyBear 5y ago
    I have a sense of ownership over my non-publicly-accessible data when it's backed up to a cloud drive. Apple isn't scanning that. Google and Microsoft are.
    - lifty 5y ago
      Sure, you own the data, but you don’t have any control on what’s happening to it. Just like iPhone users, we own the physical device but we only control certain aspects of what it’s doing, and the boundaries are not set by us, but by Apple.

heavyset_go 5y ago

This seems like a common deflection, but get back to me when either company puts programs in my pocket that scan my data for crimes and snitch on me to authorities.

bobthepanda 5y ago
At least Google does. https://protectingchildren.google/intl/en/ https://protectingchildren.google/intl/en/ > CSAI Match is our proprietary technology, developed by the YouTube team, for combating child sexual abuse imagery (CSAI) in video content online. It was the first technology to use hash-matching to identify known violative videos and allows us to identify this type of violative content amid a high volume of non-violative video content. When a match of violative content is found, it is then flagged to partners to responsibly report in accordance to local laws and regulations. Through YouTube, we make CSAI Match available for free to NGOs and industry partners like Adobe, Reddit, and Tumblr, who use it to counter the spread of online child exploitation videos on their platforms as well. > We devote significant resources—technology, people, and time—to detecting, deterring, removing, and reporting child sexual exploitation content and behavior. Since 2008, we’ve used “hashing” technology, which creates a unique digital ID for each known child sexual abuse image, to identify copies of images on our services that may exist elsewhere.
- arsome 5y ago
  Note that they're not pushing it to devices that consumers have purchased, just running it on their cloud services. I think it's an important distinction here with what Apple is doing.
- heavyset_go 5y ago
  No, that happens on Google's servers and isn't an agent in my pocket that runs on and invades my personal property. It's also only for videos.
  - bobthepanda 5y ago
    The images being scanned are the ones that go to iCloud. The Google page has a section later down that also says they use hashing of images.

brandon272 5y ago

Here's the reason for the sudden uproar: The scanning of things on third party servers was just barely tolerated by a lot of people, whether it's for advertising purposes or government intrusion. People accept it because it's considered reasonable for these third parties to scan data in exchange for providing a service for free (e.g. Google), or because they need to have some degree of accountability for what is on their servers. When that scanning gets moved from the cloud to being on your device, a boundary is violated. When that boundary is violated by a company who makes extreme privacy claims like saying that privacy is a "fundamental human right"[1], yes, people will "lose their minds" over it. This shouldn't be shocking at all. [1] https://apple.com/privacy https://apple.com/privacy

GeekyBear 5y ago
When the scanning gets moved from the cloud to being on device, Apple itself cannot see the results of the scan until the risk that the result is only a false positive is greatly reduced. You would have to have 30 false positives before Apple can see anything, which is unlikely, but the next step is still a human review, since it's not impossible.
- OnlineGladiator 5y ago
  I don't care. It's my device (at least that's how Apple used to advertise it) and I disagree with Apple's policy, full stop. I don't care about "think of the children" (especially since they will be scanning pictures of my own children), and furthermore I don't trust Apple not to change the policy in the future. They've created a backdoor and now the device is compromised and privacy is broken. If you want to trust Apple go ahead. They have eroded my trust and I doubt they could regain it within a decade.
  - GeekyBear 5y ago
    >I don't care If anything, you should be outraged that Google and Microsoft have been scanning much more of your data, and doing so in a much more intrusive way. Apple only scans iCloud Photos and they do so in a way that they can't see the results until they can be reasonably sure it's not just a false positive.

mightybyte 5y ago

I think the difference here is that it's ON YOUR DEVICE. I think there's a pretty clear understanding that if you upload stuff to a cloud provider they can do whatever they want with it. This is different. This is reaching into what has up until now mostly been considered a private place. Law enforcement often has to get warrants to search this kind of thing. This is the difference between putting CSAM on a sign in your front yard (maybe not quite front yard but I can't come up with quite the same physical equivalent to a cloud provider) and keeping it in a password protected vault in your basement. One of those things is protected in the U.S. by laws against unlawful search and seizure. Cloud and on your device are two very different things and consumers are right to be alarmed. I'll say it again, if you are concerned with this privacy violation, sell your Apple stock and categorically refuse to purchase Apple devices. Also go to https://www.nospyphone.com/ https://www.nospyphone.com/ and make your voice heard there.

GeekyBear 5y ago
ON YOUR DEVICE (where it's encrypted in a way that Apple can't read until the 30 image threshold is crossed) is more private than doing the same scan on server where a single false positive can be misused by anyone who can get a subpoena.
- pasquinelli 5y ago
  what kind of encryption can't be decrypted until some threshold has been crossed? maybe you mean to say that apple says they won't read it until that threshold has been crossed.
  - GeekyBear 5y ago
    No, I'm saying they designed the system so that they don't have the key to decypt the scan result "vouchers" until after the device tells them that the 30 image threshold is crossed. >Apple is unable to process individual vouchers; instead, all the properties of our system mean that it’s only once an account has accumulated a collection of vouchers associated with illegal, known CSAM images that we are able to learn anything about the user’s account. Now, why to do it is because, as you said, this is something that will provide that detection capability while preserving user privacy. https://techcrunch.com/2021/08/10/interview-apples-head-of-privacy-details-child-abuse-detection-and-messages-safety-features/ https://techcrunch.com/2021/08/10/interview-apples-head-of-p... Meanwhile, a single false positive from an on server scan is open to malicious use by anyone who can get a subpeona.

Copernicron 5y ago

Most people don't pay attention to whether or not their stuff is being scanned. Those of us who do pay attention have known for a long time that it's being scanned. Especially by Google and Facebook. Basically anyone whose business model is based off of advertising. My default assumption is anything I upload is scanned.

GuB-42 5y ago

Google is open about the fact they scan and track everything, they even make it a point of convenience "hey, we looked at your email and found a plane ticket, based on where you are, you should leave in 2 hours if you want to be at the airport on time". Facebook, even more so, they are explicitly anti-privacy to the point of being insulting. Microsoft will happily show you everything they may send when you install Windows, you can sometimes refuse, but not always. They are a bit less explicit than Google, but privacy is rarely on the menu. As for Amazon, their cloud offers are mostly for businesses, different market, but still, for consumers, they don't really insist on privacy either. So that if any of these company scan your pictures for child porn, it won't shock anyone, because we know it is what they do. But Apple claims privacy as a core value, half of their ads are along the lines of "we are not like the others, we respect your privacy, everything on your device stays on your device, etc...", they announce every (often legitimate) privacy feature with great fanfare, etc... So much that people start to believe it. But with that, people realize that Apple is not so different from the others after all, and if they bought an overpriced device based on that promise, I understand why they are pissed off.

xdennis 5y ago

> all the big names in tech have been scanning everything in your account [...] without any noticeable uproar In part because people didn't know. And if you were one of innocent people caught by them, you wouldn't want people to know.