7 ms·

The "scanning" feature that underpins this has been deployed for years, and I dare say wouldn't even be considered unique for photo library managers. If you can

by quitit 5y ago

The "scanning" feature that underpins this has been deployed for years, and I dare say wouldn't even be considered unique for photo library managers. If you can search for "cat" and get photos of cats, then your photo library already has the tools necessary for CSAM detection. I feel the fears about this implementation are largely overblown - equivalent features are not being misused on Google and Meta/Facebook's properties, despite having a much larger audience and less procedural oversight. Rather Apple is the outlier here for being not just late, but also by having minimum thresholds that must be passed and an unskippable step of human review. I'm yet to hear a coherent argument about the potential dangers of the system that understood the implementation, nor the state of image scanning amongst cloud providers - even this thread seems to believe that Apple is the first here. Apple's approach was the most conservative, Google's goes much farther: https://support.google.com/transparencyreport/answer/10330933?hl=en#zippy=%2Chow-does-google-identify-csam-on-its-platform https://support.google.com/transparencyreport/answer/1033093...

ItsBob 5y ago
> The "scanning" feature that underpins this has been deployed for years Yes, I agree that your phone has the ability to find stuff like this based on ML etc. However, if their algorithm finds something that matches a hash of child porn, you'll be reported (after notification or something, can't remember). Can you imagine what that would do to people? The risk of fucking this up and destroying someone's life is far too high. It's a stigma that will never go away even if it's proven false. And what if the government decides that they want to ruin a political adversary? The technology is there to report them, rightly or wrongly, for child porn on their phone and ruin them. Again, proving that it was false will not be enough if it leaked out in the first place. In my opinion, it's my phone (yes, yes, I know they all dial home to the mothership, I only lease it etc.) and I should have complete say over it and this is one more step away from that goal. It's on the dystopian highway imo.
- sweetheart 5y ago
  > matches a hash of child porn, you'll be reported After 30 hits (so 30 false positives), and then the photos are manually reviewed by a human being. How did people never understand this? How on _Earth_ would people's live be randomly destroyed by this? The reviewer would take one look (or 30) and immediately realize it's not illicit. Crisis averted. But the chances of those 30 false positives actually all being in someone's iCloud was like astronomically unlikely. > And what if the government decides that they want to ruin a political adversary? Then you're already fucked, and the child porn scanning software won't be what makes it possible to ruin your life.
  - ItsBob 5y ago
    And if the government makes it illegal to have a certain document, or a book, what happens then? You can guarantee Apple will report you. They have to. It will be the law. It will be used for human rights abuses. Guaranteed.
    - sweetheart 5y ago
      This falls under the "you're already fucked" part of my argument. If it becomes illegal to have a file, or a book, or an image, then the entire tech ecosystem we all use everyday is completely untrustworthy. Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope.
      - oarsinsync 5y ago
        > If it becomes illegal to have a file, or a book, or an image This is already true in certain circumstances. > then the entire tech ecosystem we all use everyday is completely untrustworthy. This depends on what tech you use. The degree to which you cannot trust it, or need to understand that it is actively working against you, varies by platform. Nearly all platforms undermine you in different ways. Some of those platforms do so in a way in which you're ok with. > Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope. Agree to disagree. I don't mind cloud services (someone else's computer) scanning my data and working against me. I do mind my computer scanning my data and working against me.
  - krapp 5y ago
    >After 30 hits, the photos are manually reviewed by a human being. How did people never understand this? How on _Earth_ would people's live be randomly destroyed by this? Many people here simply assume all governments and corporations are simultaneously that evil and that incompetent. The aggressive cynicism of this forum and its reticence to engage with content before commenting undermines its pretense at informed technical acumen and civil discussion in many ways.
    - 5y ago
- threeseed 5y ago
  I really just love these comments. You do know that Apple is scanning your photos server-side, right ? Just like Google, AWS, Yahoo etc anyone that stores your photos in the cloud.
  - ItsBob 5y ago
    > You do know that Apple is scanning your photos server-side, right ? Yes. They have every right to. It's their servers. However, this is MY phone (ok, not mine, I don't use Apple devices but you get my point)
  - IcyClAyMptHe 5y ago
    There's a fundamental difference between you sending files to a company server which are then scanned, and your own hardware constantly monitoring you for illegal activity so that it can report you to the authorities.
BrazzVuvuzela 5y ago
> equivalent features are not being misused on Google and Meta/Facebook's properties The objections come when such systems are to be run not on corporate property, but on end user property. I don't see anybody objecting to corporations scanning content uploaded to the corporation's own computers. Scanning content on the user's computer, which is the user's property, is what unnerves people. Furthermore, while Apple may have implemented this feature in a relatively safe way (I don't fully understand it so I'm not confident of that, but smart people here think it so I'll assume it for the sake of argument), I believe that the general public does not really understand this nuance and Apple normalizing the practice of scanning personal computers will clear the way for other corporations to implement scanning in shittier way. If Microsoft were to start scanning all documents on a Windows computer by uploading those documents to their own servers, is that a distinction that would necessarily be made clear to the general public? I don't think so. The rhetoric will be "Apple does it and that's fine, so Microsoft can do it too". The technical differences in the two implementations will be understood in forums like this, but not by the general public at large.
- quitit 5y ago
  I agree that there is a need for such changes to be public and debated - my grind comes from those which makes arguments which are neither factual, nor pay attention to the status quo. (Thus leaping to fantasy assumptions which are without basis.) For example Google and Meta both scan all material that is on their services, both of these providers actually go further by utilising AI to hunt imagery which may not be identified yet. So the idea of this compelling others to scan is false: Apple is last here. Similar to Google and Meta, Apple's CSAM processes only come into play when the material is uploaded to Apple's servers. The minutiae of where the scanning occurs is irrelevant because Apple's implementation is merely a way of performing the function without giving up the user's privacy (by scanning locally rather than sending all of your photos to Apple's servers for review.) The scanning process is merely the addition of verified hashes to a system which already uses this same technique to categorise ones photos into searchable images ("waterfalls" "dogs" "bicycles" etc.), because this is an existing system there is no impact to performance/device usage.
  - 5y ago
simion314 5y ago
The difference is that there is only 1 IF statement or local/remote config flag that will enable this scanning on your device even if you don't use the iCloud photo sharing feature. It is super well documented that Apple ALWAYS follows the laws of the land, so the scenario is that a government adds some new hashes that might not be CSAM as you and I define it but might be political stuff(but the citizens can't check). Also now that the code is there Apple can't refuse to add this hashes or enable the feature forcefully when a legal warrant for that territory is presented. some more config changes and it can start scanning for text in documents too.
- threeseed 5y ago
  Given that Apple wrote OSX, this scenario has been possible for over 20 years. And it hasn't eventuated. But yet now it is something to panic over ?
  - simion314 5y ago
    Until recently you had a lot of freedom on your OSX device, you could install and uninstall whatever you wanted , you could have a firewall that blocked applications. On iOS and slowly on OSX this freedoms are lost, already when you open your computer, open an application Apple is notified (for your safety), only a few more steps are needed until you will get laptops with iOS like lockdowns and maybe a PRO expensive version for the ones that want the key for the gates.
    - jeromegv 5y ago
      You can blame them when that happens but it’s odd to blame OS X for being locked when clearly OS X is not… locked.
      - simion314 5y ago
        >You can blame them when that happens but it’s odd to blame OS X for being locked when clearly OS X is not… locked. OSX is getting more and more locked, iOS is locked so why shouldn't people complain when things get worse. I was just explaining that the excuse "OSX is 20 years old and it was not locked yet so it will never be locked q.e.d " , this is wrong, you can see clearly how things get more locked in steps and you don't have to wait untill last step to complain. Keep in mind that you have better keyboards on Apple laptops today because people complained, the CSAM shit was delayed because people complained, the Apple tax was lowered in some cases because developers and Epic complained, so complaining sometimes works.
codydh 5y ago
It seems that many of the arguments against this system rely on believing governments (or similarly powerful groups) will strong-arm Apple and fabricate evidence. I fail to see how this system (which I'm not a big fan of, to be clear) makes it easier if you assume governments can and will fabricate evidence. Doesn't seem you need an iPhone, smartphone, or to have owned or done anything at all in that case.

by quitit 5y ago

ItsBob 5y ago

> The "scanning" feature that underpins this has been deployed for years Yes, I agree that your phone has the ability to find stuff like this based on ML etc. However, if their algorithm finds something that matches a hash of child porn, you'll be reported (after notification or something, can't remember). Can you imagine what that would do to people? The risk of fucking this up and destroying someone's life is far too high. It's a stigma that will never go away even if it's proven false. And what if the government decides that they want to ruin a political adversary? The technology is there to report them, rightly or wrongly, for child porn on their phone and ruin them. Again, proving that it was false will not be enough if it leaked out in the first place. In my opinion, it's my phone (yes, yes, I know they all dial home to the mothership, I only lease it etc.) and I should have complete say over it and this is one more step away from that goal. It's on the dystopian highway imo.

sweetheart 5y ago
> matches a hash of child porn, you'll be reported After 30 hits (so 30 false positives), and then the photos are manually reviewed by a human being. How did people never understand this? How on _Earth_ would people's live be randomly destroyed by this? The reviewer would take one look (or 30) and immediately realize it's not illicit. Crisis averted. But the chances of those 30 false positives actually all being in someone's iCloud was like astronomically unlikely. > And what if the government decides that they want to ruin a political adversary? Then you're already fucked, and the child porn scanning software won't be what makes it possible to ruin your life.
- ItsBob 5y ago
  And if the government makes it illegal to have a certain document, or a book, what happens then? You can guarantee Apple will report you. They have to. It will be the law. It will be used for human rights abuses. Guaranteed.
  - sweetheart 5y ago
    This falls under the "you're already fucked" part of my argument. If it becomes illegal to have a file, or a book, or an image, then the entire tech ecosystem we all use everyday is completely untrustworthy. Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope.
    - oarsinsync 5y ago
      > If it becomes illegal to have a file, or a book, or an image This is already true in certain circumstances. > then the entire tech ecosystem we all use everyday is completely untrustworthy. This depends on what tech you use. The degree to which you cannot trust it, or need to understand that it is actively working against you, varies by platform. Nearly all platforms undermine you in different ways. Some of those platforms do so in a way in which you're ok with. > Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope. Agree to disagree. I don't mind cloud services (someone else's computer) scanning my data and working against me. I do mind my computer scanning my data and working against me.
- krapp 5y ago
  >After 30 hits, the photos are manually reviewed by a human being. How did people never understand this? How on _Earth_ would people's live be randomly destroyed by this? Many people here simply assume all governments and corporations are simultaneously that evil and that incompetent. The aggressive cynicism of this forum and its reticence to engage with content before commenting undermines its pretense at informed technical acumen and civil discussion in many ways.
  - 5y ago
threeseed 5y ago
I really just love these comments. You do know that Apple is scanning your photos server-side, right ? Just like Google, AWS, Yahoo etc anyone that stores your photos in the cloud.
- ItsBob 5y ago
  > You do know that Apple is scanning your photos server-side, right ? Yes. They have every right to. It's their servers. However, this is MY phone (ok, not mine, I don't use Apple devices but you get my point)
- IcyClAyMptHe 5y ago
  There's a fundamental difference between you sending files to a company server which are then scanned, and your own hardware constantly monitoring you for illegal activity so that it can report you to the authorities.

BrazzVuvuzela 5y ago

> equivalent features are not being misused on Google and Meta/Facebook's properties The objections come when such systems are to be run not on corporate property, but on end user property. I don't see anybody objecting to corporations scanning content uploaded to the corporation's own computers. Scanning content on the user's computer, which is the user's property, is what unnerves people. Furthermore, while Apple may have implemented this feature in a relatively safe way (I don't fully understand it so I'm not confident of that, but smart people here think it so I'll assume it for the sake of argument), I believe that the general public does not really understand this nuance and Apple normalizing the practice of scanning personal computers will clear the way for other corporations to implement scanning in shittier way. If Microsoft were to start scanning all documents on a Windows computer by uploading those documents to their own servers, is that a distinction that would necessarily be made clear to the general public? I don't think so. The rhetoric will be "Apple does it and that's fine, so Microsoft can do it too". The technical differences in the two implementations will be understood in forums like this, but not by the general public at large.

quitit 5y ago
I agree that there is a need for such changes to be public and debated - my grind comes from those which makes arguments which are neither factual, nor pay attention to the status quo. (Thus leaping to fantasy assumptions which are without basis.) For example Google and Meta both scan all material that is on their services, both of these providers actually go further by utilising AI to hunt imagery which may not be identified yet. So the idea of this compelling others to scan is false: Apple is last here. Similar to Google and Meta, Apple's CSAM processes only come into play when the material is uploaded to Apple's servers. The minutiae of where the scanning occurs is irrelevant because Apple's implementation is merely a way of performing the function without giving up the user's privacy (by scanning locally rather than sending all of your photos to Apple's servers for review.) The scanning process is merely the addition of verified hashes to a system which already uses this same technique to categorise ones photos into searchable images ("waterfalls" "dogs" "bicycles" etc.), because this is an existing system there is no impact to performance/device usage.
- 5y ago

simion314 5y ago

The difference is that there is only 1 IF statement or local/remote config flag that will enable this scanning on your device even if you don't use the iCloud photo sharing feature. It is super well documented that Apple ALWAYS follows the laws of the land, so the scenario is that a government adds some new hashes that might not be CSAM as you and I define it but might be political stuff(but the citizens can't check). Also now that the code is there Apple can't refuse to add this hashes or enable the feature forcefully when a legal warrant for that territory is presented. some more config changes and it can start scanning for text in documents too.

threeseed 5y ago
Given that Apple wrote OSX, this scenario has been possible for over 20 years. And it hasn't eventuated. But yet now it is something to panic over ?
- simion314 5y ago
  Until recently you had a lot of freedom on your OSX device, you could install and uninstall whatever you wanted , you could have a firewall that blocked applications. On iOS and slowly on OSX this freedoms are lost, already when you open your computer, open an application Apple is notified (for your safety), only a few more steps are needed until you will get laptops with iOS like lockdowns and maybe a PRO expensive version for the ones that want the key for the gates.
  - jeromegv 5y ago
    You can blame them when that happens but it’s odd to blame OS X for being locked when clearly OS X is not… locked.
    - simion314 5y ago
      >You can blame them when that happens but it’s odd to blame OS X for being locked when clearly OS X is not… locked. OSX is getting more and more locked, iOS is locked so why shouldn't people complain when things get worse. I was just explaining that the excuse "OSX is 20 years old and it was not locked yet so it will never be locked q.e.d " , this is wrong, you can see clearly how things get more locked in steps and you don't have to wait untill last step to complain. Keep in mind that you have better keyboards on Apple laptops today because people complained, the CSAM shit was delayed because people complained, the Apple tax was lowered in some cases because developers and Epic complained, so complaining sometimes works.

codydh 5y ago

It seems that many of the arguments against this system rely on believing governments (or similarly powerful groups) will strong-arm Apple and fabricate evidence. I fail to see how this system (which I'm not a big fan of, to be clear) makes it easier if you assume governments can and will fabricate evidence. Doesn't seem you need an iPhone, smartphone, or to have owned or done anything at all in that case.