7 ms·

Problem with something like this is the cat's out of the bag. The code has been written, tested and merged etc. The project has reached a point where it's read

by ItsBob 5y ago

Problem with something like this is the cat's out of the bag. The code has been written, tested and merged etc. The project has reached a point where it's ready to ship. Apple have done their homework and they will not release ANYTHING unless they think they'll either make money from it or at the very least, not LOSE money from it. It's all about money, nothing else. It's coming whether we like it or not now :( And you can be sure that others will follow. It will be in Android at some point and possibly Windows and MacOS too. I try not to be negative and cynical these days but the tech oligarchs make that incredibly difficult.

quitit 5y ago
The "scanning" feature that underpins this has been deployed for years, and I dare say wouldn't even be considered unique for photo library managers. If you can search for "cat" and get photos of cats, then your photo library already has the tools necessary for CSAM detection. I feel the fears about this implementation are largely overblown - equivalent features are not being misused on Google and Meta/Facebook's properties, despite having a much larger audience and less procedural oversight. Rather Apple is the outlier here for being not just late, but also by having minimum thresholds that must be passed and an unskippable step of human review. I'm yet to hear a coherent argument about the potential dangers of the system that understood the implementation, nor the state of image scanning amongst cloud providers - even this thread seems to believe that Apple is the first here. Apple's approach was the most conservative, Google's goes much farther: https://support.google.com/transparencyreport/answer/10330933?hl=en#zippy=%2Chow-does-google-identify-csam-on-its-platform https://support.google.com/transparencyreport/answer/1033093...
- ItsBob 5y ago
  > The "scanning" feature that underpins this has been deployed for years Yes, I agree that your phone has the ability to find stuff like this based on ML etc. However, if their algorithm finds something that matches a hash of child porn, you'll be reported (after notification or something, can't remember). Can you imagine what that would do to people? The risk of fucking this up and destroying someone's life is far too high. It's a stigma that will never go away even if it's proven false. And what if the government decides that they want to ruin a political adversary? The technology is there to report them, rightly or wrongly, for child porn on their phone and ruin them. Again, proving that it was false will not be enough if it leaked out in the first place. In my opinion, it's my phone (yes, yes, I know they all dial home to the mothership, I only lease it etc.) and I should have complete say over it and this is one more step away from that goal. It's on the dystopian highway imo.
  - sweetheart 5y ago
    > matches a hash of child porn, you'll be reported After 30 hits (so 30 false positives), and then the photos are manually reviewed by a human being. How did people never understand this? How on _Earth_ would people's live be randomly destroyed by this? The reviewer would take one look (or 30) and immediately realize it's not illicit. Crisis averted. But the chances of those 30 false positives actually all being in someone's iCloud was like astronomically unlikely. > And what if the government decides that they want to ruin a political adversary? Then you're already fucked, and the child porn scanning software won't be what makes it possible to ruin your life.
    - ItsBob 5y ago
      And if the government makes it illegal to have a certain document, or a book, what happens then? You can guarantee Apple will report you. They have to. It will be the law. It will be used for human rights abuses. Guaranteed.
      - sweetheart 5y ago
        This falls under the "you're already fucked" part of my argument. If it becomes illegal to have a file, or a book, or an image, then the entire tech ecosystem we all use everyday is completely untrustworthy. Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope.
        oarsinsync 5y ago
        > If it becomes illegal to have a file, or a book, or an image This is already true in certain circumstances. > then the entire tech ecosystem we all use everyday is completely untrustworthy. This depends on what tech you use. The degree to which you cannot trust it, or need to understand that it is actively working against you, varies by platform. Nearly all platforms undermine you in different ways. Some of those platforms do so in a way in which you're ok with. > Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope. Agree to disagree. I don't mind cloud services (someone else's computer) scanning my data and working against me. I do mind my computer scanning my data and working against me.
- BrazzVuvuzela 5y ago
  > equivalent features are not being misused on Google and Meta/Facebook's properties The objections come when such systems are to be run not on corporate property, but on end user property. I don't see anybody objecting to corporations scanning content uploaded to the corporation's own computers. Scanning content on the user's computer, which is the user's property, is what unnerves people. Furthermore, while Apple may have implemented this feature in a relatively safe way (I don't fully understand it so I'm not confident of that, but smart people here think it so I'll assume it for the sake of argument), I believe that the general public does not really understand this nuance and Apple normalizing the practice of scanning personal computers will clear the way for other corporations to implement scanning in shittier way. If Microsoft were to start scanning all documents on a Windows computer by uploading those documents to their own servers, is that a distinction that would necessarily be made clear to the general public? I don't think so. The rhetoric will be "Apple does it and that's fine, so Microsoft can do it too". The technical differences in the two implementations will be understood in forums like this, but not by the general public at large.
  - quitit 5y ago
    I agree that there is a need for such changes to be public and debated - my grind comes from those which makes arguments which are neither factual, nor pay attention to the status quo. (Thus leaping to fantasy assumptions which are without basis.) For example Google and Meta both scan all material that is on their services, both of these providers actually go further by utilising AI to hunt imagery which may not be identified yet. So the idea of this compelling others to scan is false: Apple is last here. Similar to Google and Meta, Apple's CSAM processes only come into play when the material is uploaded to Apple's servers. The minutiae of where the scanning occurs is irrelevant because Apple's implementation is merely a way of performing the function without giving up the user's privacy (by scanning locally rather than sending all of your photos to Apple's servers for review.) The scanning process is merely the addition of verified hashes to a system which already uses this same technique to categorise ones photos into searchable images ("waterfalls" "dogs" "bicycles" etc.), because this is an existing system there is no impact to performance/device usage.
    - 5y ago
- simion314 5y ago
  The difference is that there is only 1 IF statement or local/remote config flag that will enable this scanning on your device even if you don't use the iCloud photo sharing feature. It is super well documented that Apple ALWAYS follows the laws of the land, so the scenario is that a government adds some new hashes that might not be CSAM as you and I define it but might be political stuff(but the citizens can't check). Also now that the code is there Apple can't refuse to add this hashes or enable the feature forcefully when a legal warrant for that territory is presented. some more config changes and it can start scanning for text in documents too.
  - threeseed 5y ago
    Given that Apple wrote OSX, this scenario has been possible for over 20 years. And it hasn't eventuated. But yet now it is something to panic over ?
    - simion314 5y ago
      Until recently you had a lot of freedom on your OSX device, you could install and uninstall whatever you wanted , you could have a firewall that blocked applications. On iOS and slowly on OSX this freedoms are lost, already when you open your computer, open an application Apple is notified (for your safety), only a few more steps are needed until you will get laptops with iOS like lockdowns and maybe a PRO expensive version for the ones that want the key for the gates.
      - jeromegv 5y ago
        You can blame them when that happens but it’s odd to blame OS X for being locked when clearly OS X is not… locked.
        simion314 5y ago
        >You can blame them when that happens but it’s odd to blame OS X for being locked when clearly OS X is not… locked. OSX is getting more and more locked, iOS is locked so why shouldn't people complain when things get worse. I was just explaining that the excuse "OSX is 20 years old and it was not locked yet so it will never be locked q.e.d " , this is wrong, you can see clearly how things get more locked in steps and you don't have to wait untill last step to complain. Keep in mind that you have better keyboards on Apple laptops today because people complained, the CSAM shit was delayed because people complained, the Apple tax was lowered in some cases because developers and Epic complained, so complaining sometimes works.
- codydh 5y ago
  It seems that many of the arguments against this system rely on believing governments (or similarly powerful groups) will strong-arm Apple and fabricate evidence. I fail to see how this system (which I'm not a big fan of, to be clear) makes it easier if you assume governments can and will fabricate evidence. Doesn't seem you need an iPhone, smartphone, or to have owned or done anything at all in that case.
m12k 5y ago
> Apple have done their homework and they will not release ANYTHING unless they think they'll either make money from it or at the very least, not LOSE money from it. It's all about money, nothing else. > It's coming whether we like it or not now :( Um, I think the public outcry showed them pretty clearly that they will lose customers and money precisely if they go ahead with this. I think the people championing this inside Apple (some rumors say Cook himself) were expecting this to be some kind of feel-good, "we're thinking of the children" PR piece - not the shitstorm it turned out to be.
- derwiki 5y ago
  Now they know to be quieter about this. It will be a quiet line item in an upcoming dot release. And not enough people will fuss (eventually).
  - sharklazer 5y ago
    I have a feeling it already came in 15.2. They are scanning texts for “nudity” in images. Not that they are the same thing. Or being used in the same way. But I have a feeling CSAM scanning just needs it’s flag turned on to work. https://petapixel.com/2021/12/13/apple-to-release-nudity-detection-but-other-safety-features-still-on-hold/ https://petapixel.com/2021/12/13/apple-to-release-nudity-det...
    - jeromegv 5y ago
      Thank you for your “feelings”. Except your feelings have no basis in reality. Why would you blame a company for something that they clearly haven’t done?
      - sharklazer 5y ago
        Well, their gonna push it at some point. Do you think they’ll tell you when they do, given the backlash. Tell you what, put up 20k USD and I’ll go through the formal process of verifying binaries and whatnot. Otherwise, I think my “feeling” still contributes to the discussion. I don’t need to put in 200 hours to prove something I’m not misrepresenting as fact. CSAM scanning will come. Apple probably won’t tell you.
duxup 5y ago
> The code has been written, tested and merged etc. The project has reached a point where it's ready to ship. Are we talking about anything that scans photos? If so that’s pretty common… everywhere. I have some technically capable friends who were up in arms about this situation. Then they share their “vacation at X with friends “ Google photos album with me. Cat isn’t just out of the bag….
sharklazer 5y ago
In 15.2, they are scanning text messages for “nudity”. Oddly, no post here on HN about that. My guess is it’s already quietly pushed, but the flag needs turned on. https://petapixel.com/2021/12/13/apple-to-release-nudity-detection-but-other-safety-features-still-on-hold/ https://petapixel.com/2021/12/13/apple-to-release-nudity-det...
- kevinmgranger 5y ago
  Is it local-only? If so, it has far fewer privacy implications than CSAM scanning.
  - sharklazer 5y ago
    Yes, local only as far as I understand. But how many additional steps does it take to turn the process into CSAM scanning? It looks really close, even without squinting. Caveat that I have no certain info though.
    - JimDabell 5y ago
      > But how many additional steps does it take to turn the process into CSAM scanning? It looks really close, even without squinting. It’s a different feature that works in an entirely different way. There’s plenty of descriptions of how both features work out there, there’s no need to guess at what they might do.
- jaywalk 5y ago
  There has been talk about it here, but it coincided with the CSAM scanning which got much more coverage.
  - sharklazer 5y ago
    Thanks, I guess I missed it. Too many things going on in this modern world to have a complete picture.
- mlindner 5y ago
dkdbejwi383 5y ago
> It's all about money, nothing else. Not being funny, but I am not sure why this comes as a surprise. Of course everything Apple (and any other for-profit org) does is to make money.
deleted 5y ago
[deleted]

by ItsBob 5y ago

quitit 5y ago

The "scanning" feature that underpins this has been deployed for years, and I dare say wouldn't even be considered unique for photo library managers. If you can search for "cat" and get photos of cats, then your photo library already has the tools necessary for CSAM detection. I feel the fears about this implementation are largely overblown - equivalent features are not being misused on Google and Meta/Facebook's properties, despite having a much larger audience and less procedural oversight. Rather Apple is the outlier here for being not just late, but also by having minimum thresholds that must be passed and an unskippable step of human review. I'm yet to hear a coherent argument about the potential dangers of the system that understood the implementation, nor the state of image scanning amongst cloud providers - even this thread seems to believe that Apple is the first here. Apple's approach was the most conservative, Google's goes much farther: https://support.google.com/transparencyreport/answer/10330933?hl=en#zippy=%2Chow-does-google-identify-csam-on-its-platform https://support.google.com/transparencyreport/answer/1033093...

ItsBob 5y ago
> The "scanning" feature that underpins this has been deployed for years Yes, I agree that your phone has the ability to find stuff like this based on ML etc. However, if their algorithm finds something that matches a hash of child porn, you'll be reported (after notification or something, can't remember). Can you imagine what that would do to people? The risk of fucking this up and destroying someone's life is far too high. It's a stigma that will never go away even if it's proven false. And what if the government decides that they want to ruin a political adversary? The technology is there to report them, rightly or wrongly, for child porn on their phone and ruin them. Again, proving that it was false will not be enough if it leaked out in the first place. In my opinion, it's my phone (yes, yes, I know they all dial home to the mothership, I only lease it etc.) and I should have complete say over it and this is one more step away from that goal. It's on the dystopian highway imo.
- sweetheart 5y ago
  > matches a hash of child porn, you'll be reported After 30 hits (so 30 false positives), and then the photos are manually reviewed by a human being. How did people never understand this? How on _Earth_ would people's live be randomly destroyed by this? The reviewer would take one look (or 30) and immediately realize it's not illicit. Crisis averted. But the chances of those 30 false positives actually all being in someone's iCloud was like astronomically unlikely. > And what if the government decides that they want to ruin a political adversary? Then you're already fucked, and the child porn scanning software won't be what makes it possible to ruin your life.
  - ItsBob 5y ago
    And if the government makes it illegal to have a certain document, or a book, what happens then? You can guarantee Apple will report you. They have to. It will be the law. It will be used for human rights abuses. Guaranteed.
    - sweetheart 5y ago
      This falls under the "you're already fucked" part of my argument. If it becomes illegal to have a file, or a book, or an image, then the entire tech ecosystem we all use everyday is completely untrustworthy. Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope.
      - oarsinsync 5y ago
        > If it becomes illegal to have a file, or a book, or an image This is already true in certain circumstances. > then the entire tech ecosystem we all use everyday is completely untrustworthy. This depends on what tech you use. The degree to which you cannot trust it, or need to understand that it is actively working against you, varies by platform. Nearly all platforms undermine you in different ways. Some of those platforms do so in a way in which you're ok with. > Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope. Agree to disagree. I don't mind cloud services (someone else's computer) scanning my data and working against me. I do mind my computer scanning my data and working against me.
BrazzVuvuzela 5y ago
> equivalent features are not being misused on Google and Meta/Facebook's properties The objections come when such systems are to be run not on corporate property, but on end user property. I don't see anybody objecting to corporations scanning content uploaded to the corporation's own computers. Scanning content on the user's computer, which is the user's property, is what unnerves people. Furthermore, while Apple may have implemented this feature in a relatively safe way (I don't fully understand it so I'm not confident of that, but smart people here think it so I'll assume it for the sake of argument), I believe that the general public does not really understand this nuance and Apple normalizing the practice of scanning personal computers will clear the way for other corporations to implement scanning in shittier way. If Microsoft were to start scanning all documents on a Windows computer by uploading those documents to their own servers, is that a distinction that would necessarily be made clear to the general public? I don't think so. The rhetoric will be "Apple does it and that's fine, so Microsoft can do it too". The technical differences in the two implementations will be understood in forums like this, but not by the general public at large.
- quitit 5y ago
  I agree that there is a need for such changes to be public and debated - my grind comes from those which makes arguments which are neither factual, nor pay attention to the status quo. (Thus leaping to fantasy assumptions which are without basis.) For example Google and Meta both scan all material that is on their services, both of these providers actually go further by utilising AI to hunt imagery which may not be identified yet. So the idea of this compelling others to scan is false: Apple is last here. Similar to Google and Meta, Apple's CSAM processes only come into play when the material is uploaded to Apple's servers. The minutiae of where the scanning occurs is irrelevant because Apple's implementation is merely a way of performing the function without giving up the user's privacy (by scanning locally rather than sending all of your photos to Apple's servers for review.) The scanning process is merely the addition of verified hashes to a system which already uses this same technique to categorise ones photos into searchable images ("waterfalls" "dogs" "bicycles" etc.), because this is an existing system there is no impact to performance/device usage.
  - 5y ago
simion314 5y ago
The difference is that there is only 1 IF statement or local/remote config flag that will enable this scanning on your device even if you don't use the iCloud photo sharing feature. It is super well documented that Apple ALWAYS follows the laws of the land, so the scenario is that a government adds some new hashes that might not be CSAM as you and I define it but might be political stuff(but the citizens can't check). Also now that the code is there Apple can't refuse to add this hashes or enable the feature forcefully when a legal warrant for that territory is presented. some more config changes and it can start scanning for text in documents too.
- threeseed 5y ago
  Given that Apple wrote OSX, this scenario has been possible for over 20 years. And it hasn't eventuated. But yet now it is something to panic over ?
  - simion314 5y ago
    Until recently you had a lot of freedom on your OSX device, you could install and uninstall whatever you wanted , you could have a firewall that blocked applications. On iOS and slowly on OSX this freedoms are lost, already when you open your computer, open an application Apple is notified (for your safety), only a few more steps are needed until you will get laptops with iOS like lockdowns and maybe a PRO expensive version for the ones that want the key for the gates.
    - jeromegv 5y ago
      You can blame them when that happens but it’s odd to blame OS X for being locked when clearly OS X is not… locked.
      - simion314 5y ago
        >You can blame them when that happens but it’s odd to blame OS X for being locked when clearly OS X is not… locked. OSX is getting more and more locked, iOS is locked so why shouldn't people complain when things get worse. I was just explaining that the excuse "OSX is 20 years old and it was not locked yet so it will never be locked q.e.d " , this is wrong, you can see clearly how things get more locked in steps and you don't have to wait untill last step to complain. Keep in mind that you have better keyboards on Apple laptops today because people complained, the CSAM shit was delayed because people complained, the Apple tax was lowered in some cases because developers and Epic complained, so complaining sometimes works.
codydh 5y ago
It seems that many of the arguments against this system rely on believing governments (or similarly powerful groups) will strong-arm Apple and fabricate evidence. I fail to see how this system (which I'm not a big fan of, to be clear) makes it easier if you assume governments can and will fabricate evidence. Doesn't seem you need an iPhone, smartphone, or to have owned or done anything at all in that case.

m12k 5y ago

> Apple have done their homework and they will not release ANYTHING unless they think they'll either make money from it or at the very least, not LOSE money from it. It's all about money, nothing else. > It's coming whether we like it or not now :( Um, I think the public outcry showed them pretty clearly that they will lose customers and money precisely if they go ahead with this. I think the people championing this inside Apple (some rumors say Cook himself) were expecting this to be some kind of feel-good, "we're thinking of the children" PR piece - not the shitstorm it turned out to be.

derwiki 5y ago
Now they know to be quieter about this. It will be a quiet line item in an upcoming dot release. And not enough people will fuss (eventually).
- sharklazer 5y ago
  I have a feeling it already came in 15.2. They are scanning texts for “nudity” in images. Not that they are the same thing. Or being used in the same way. But I have a feeling CSAM scanning just needs it’s flag turned on to work. https://petapixel.com/2021/12/13/apple-to-release-nudity-detection-but-other-safety-features-still-on-hold/ https://petapixel.com/2021/12/13/apple-to-release-nudity-det...
  - jeromegv 5y ago
    Thank you for your “feelings”. Except your feelings have no basis in reality. Why would you blame a company for something that they clearly haven’t done?
    - sharklazer 5y ago
      Well, their gonna push it at some point. Do you think they’ll tell you when they do, given the backlash. Tell you what, put up 20k USD and I’ll go through the formal process of verifying binaries and whatnot. Otherwise, I think my “feeling” still contributes to the discussion. I don’t need to put in 200 hours to prove something I’m not misrepresenting as fact. CSAM scanning will come. Apple probably won’t tell you.

duxup 5y ago

> The code has been written, tested and merged etc. The project has reached a point where it's ready to ship. Are we talking about anything that scans photos? If so that’s pretty common… everywhere. I have some technically capable friends who were up in arms about this situation. Then they share their “vacation at X with friends “ Google photos album with me. Cat isn’t just out of the bag….

sharklazer 5y ago

In 15.2, they are scanning text messages for “nudity”. Oddly, no post here on HN about that. My guess is it’s already quietly pushed, but the flag needs turned on. https://petapixel.com/2021/12/13/apple-to-release-nudity-detection-but-other-safety-features-still-on-hold/ https://petapixel.com/2021/12/13/apple-to-release-nudity-det...

kevinmgranger 5y ago
Is it local-only? If so, it has far fewer privacy implications than CSAM scanning.
- sharklazer 5y ago
  Yes, local only as far as I understand. But how many additional steps does it take to turn the process into CSAM scanning? It looks really close, even without squinting. Caveat that I have no certain info though.
  - JimDabell 5y ago
    > But how many additional steps does it take to turn the process into CSAM scanning? It looks really close, even without squinting. It’s a different feature that works in an entirely different way. There’s plenty of descriptions of how both features work out there, there’s no need to guess at what they might do.
jaywalk 5y ago
There has been talk about it here, but it coincided with the CSAM scanning which got much more coverage.
- sharklazer 5y ago
  Thanks, I guess I missed it. Too many things going on in this modern world to have a complete picture.
mlindner 5y ago

dkdbejwi383 5y ago

> It's all about money, nothing else. Not being funny, but I am not sure why this comes as a surprise. Of course everything Apple (and any other for-profit org) does is to make money.

deleted 5y ago

[deleted]