8 ms·

I’m not sure what to think of the backlash here. I’m sure that people aren’t trying to minimize the evil of child pornography and exploitation. But anything tha

I’m not sure what to think of the backlash here. I’m sure that people aren’t trying to minimize the evil of child pornography and exploitation. But anything that has the potential to stop or slow it should be fairly considered. People complaining that Apple is scanning your photos, they are already doing that. Where was this backlash when they released the memories feature? Why is scanning your photos for good pictures of your dog different that scanning your pictures for exploitive pictures of children? Is the problem that they could then share that information with the authorities? But of course this feature could be abused. But Apple already has all the power it needs to be abusive. They can push whatever software they want to your devices. They can remotely lock and wipe all your data. They already have the power to do all of these things. This statement is simply an announcement that they will be using some of this power to try to stop one of the worst evils in our world. Until they prove that they will abuse this power, I suggest that we let them try.

xvector 5y ago
The concern is less about outing people with CSAM and more about Apple building a powerful tool for government surveillance that will most certainly be abused by countries like China and Saudi Arabia when they request that politically dangerous images be added to the database if Apple wants to continue business in their country. It is incredibly myopic of Apple to implement this feature. You must consider, as an engineer, how your constructions will be abused and used for bad as well as good.
- simondotau 5y ago
  This isn't a powerful tool, it's more akin to a rudimentary hash check. It's not going to match photos that aren't already known to authorities. And any time a user is flagged, the material in question is sighted by an Apple employee before a decision is made whether to forward it onto the relevant authorities.
  - dmitrygr 5y ago
    Ok, so what stops the CCP from submitting hashes of photos of the "tank man" to this hash set? And then jailing all those reported to possess it?
    - Klonoar 5y ago
      What's stopping them already...? It operates on iCloud photos. Those are already scanned. If a nation state wanted to flex this before they could have done so.
      - dmitrygr 5y ago
        Same reason you do not sue someone for patent infringement until it is too late for them to turn back?
      - deleted 5y ago
        [deleted]
      - meowster 5y ago
        That's a poor argument, and possibly some kind of fallacy. 'No one has done it before, so they probably won't do it in the future now that it we made it easier.'
        Jolter 5y ago
        It’s not easier though. They moved the scanning onto the device instead of doing it server side. That all.
    - simondotau 5y ago
      Because when the Apple employee who reviews the flagged accounts sees pictures of Tank Man and not CSAM, they won't forward them on.
      - xvector 5y ago
        Unless their business is threatened
hvocode 5y ago
I think the point is that yes, they have always had the power to do these things - but they haven't thus far. We rely on entities with power (companies, governments, people) to exercise restraint in how they exercise it. Those that DO exercise restraint gain trust since exercising restraint demonstrates an understanding of the consequences of not doing so. What the Apple move is doing is showing that they are willing to relax their restraint. It gets tricky because everyone agrees that the specific goal here is honorable, but the manner by which they are using their power to achieve it is generalizable to areas that are less honorable. Once they are willing to use their power to accomplish one highly honorable goal, it's not a big ask for them to use it for a slightly less honorable goal in the future. Iterate that a few times and you can find yourself in a very bad place. It's the classic slippery slope argument - when you know there is a slope that leads to dangerous places, you need to not ever start down it no matter how righteous the motive is in starting down that path in the first place. There's a reason we have the old saying "the path to hell is paved with good intentions". The existence of power isn't what matters: it's the intention and willingness to exercise it. Apple is now demonstrating that they have changed their stance in how they choose to exercise their extreme power. That's worthy of scrutiny. For a concrete example of where I expect this to naturally lead to: instead of a database of child pornography being the source of the hashes to search for, the Chinese government provides a set of hashes of all known digital photos of the Tianenmen square protests of 1989. Does it really seem implausible for a government like China's to NOT use this kind of technology for that purpose? It's not hard to cook up similar examples all over the place.
- jasonlfunk 5y ago
  Yeah, but they are already doing this for pointless things. They already use facial recognition on all the photos on your phone. That’s what I don’t understand. The only new thing is what they are looking for and their willingness to alert the authorities. The slippery slope argument that this will eventually be used by China to arrest journalists is scare tactics. We have zero evidence that Apple would allow such a thing to happen. And the only thing stopping them is Apple’s word. The fact that they are announcing this should actually give confidence that they aren’t doing it in the shadows for China. They didn’t have to say anything about this. The fact that they should give you confidence that they are respecting your rights, not evidence that they aren’t.
dmitrygr 5y ago
But anything that has the potential to stop or slow it should be fairly considered Strip searching everyone hourly will stop most contraband, including child pornography. And, according to you, it should be fairly considered. Shall we start with you? (And as people do have naked photos of themselves on their phones often[1], make no mistake, the strip search analogy is NOT an exaggeration) [1] https://en.wikipedia.org/wiki/ICloud_leaks_of_celebrity_photos https://en.wikipedia.org/wiki/ICloud_leaks_of_celebrity_phot...
- jasonlfunk 5y ago
  Considered, sure. But easily rejected. Let’s be grownups here.
  - dmitrygr 5y ago
    Yes, indeed, let's! Grownups recognize that strip searches are unwarranted in this and most other cases, be they physical OR digital... And certainly you've heard of software bugs. You do not think that a small one like "accidentally" "forgetting" to check that a photo had been synced cannot happen, and an "accidental" scan of ALL photos couldn't possibly happen in any situation no matter what bug/corruption/etc? Surely you'll share with the class where YOU hire infallible programmers. An accidental strip search is a strip search nonetheless. Luckily, this can all be solved by NOT writing such code. Ever.
    - jpxw 5y ago
      You must surely realise that there’s a fairly large difference between being strip-searched and having a hash computed on one of your photos that you’re uploading to iCloud.
      - dmitrygr 5y ago
        You must surely realize that slippery slopes exist. And as soon as adversarial code exists in your device, it will only expand. Reference: history of literally every dictatorship includes many "reasonable" expansions of power in the name of security/safety/etc...
matwood 5y ago
It's important not to conflate the new features. CSAM uses hashes of known photos and is only run on photos going to iCloud (turning off iCloud turns off CSAM). Photos sent to iCloud have been checked against CSAM for years on the server. The change here is moving it from server to client (which I hope is to make iCloud photos E2E encrypted). Completely agree with your second point. All the 'what ifs' have existed forever. Either iOS users trust Apple will only do what stated or they don't. Nothing has changed.

xvector 5y ago

The concern is less about outing people with CSAM and more about Apple building a powerful tool for government surveillance that will most certainly be abused by countries like China and Saudi Arabia when they request that politically dangerous images be added to the database if Apple wants to continue business in their country. It is incredibly myopic of Apple to implement this feature. You must consider, as an engineer, how your constructions will be abused and used for bad as well as good.

simondotau 5y ago
This isn't a powerful tool, it's more akin to a rudimentary hash check. It's not going to match photos that aren't already known to authorities. And any time a user is flagged, the material in question is sighted by an Apple employee before a decision is made whether to forward it onto the relevant authorities.
- dmitrygr 5y ago
  Ok, so what stops the CCP from submitting hashes of photos of the "tank man" to this hash set? And then jailing all those reported to possess it?
  - Klonoar 5y ago
    What's stopping them already...? It operates on iCloud photos. Those are already scanned. If a nation state wanted to flex this before they could have done so.
    - dmitrygr 5y ago
      Same reason you do not sue someone for patent infringement until it is too late for them to turn back?
    - deleted 5y ago
      [deleted]
    - meowster 5y ago
      That's a poor argument, and possibly some kind of fallacy. 'No one has done it before, so they probably won't do it in the future now that it we made it easier.'
      - Jolter 5y ago
        It’s not easier though. They moved the scanning onto the device instead of doing it server side. That all.
  - simondotau 5y ago
    Because when the Apple employee who reviews the flagged accounts sees pictures of Tank Man and not CSAM, they won't forward them on.
    - xvector 5y ago
      Unless their business is threatened

hvocode 5y ago

I think the point is that yes, they have always had the power to do these things - but they haven't thus far. We rely on entities with power (companies, governments, people) to exercise restraint in how they exercise it. Those that DO exercise restraint gain trust since exercising restraint demonstrates an understanding of the consequences of not doing so. What the Apple move is doing is showing that they are willing to relax their restraint. It gets tricky because everyone agrees that the specific goal here is honorable, but the manner by which they are using their power to achieve it is generalizable to areas that are less honorable. Once they are willing to use their power to accomplish one highly honorable goal, it's not a big ask for them to use it for a slightly less honorable goal in the future. Iterate that a few times and you can find yourself in a very bad place. It's the classic slippery slope argument - when you know there is a slope that leads to dangerous places, you need to not ever start down it no matter how righteous the motive is in starting down that path in the first place. There's a reason we have the old saying "the path to hell is paved with good intentions". The existence of power isn't what matters: it's the intention and willingness to exercise it. Apple is now demonstrating that they have changed their stance in how they choose to exercise their extreme power. That's worthy of scrutiny. For a concrete example of where I expect this to naturally lead to: instead of a database of child pornography being the source of the hashes to search for, the Chinese government provides a set of hashes of all known digital photos of the Tianenmen square protests of 1989. Does it really seem implausible for a government like China's to NOT use this kind of technology for that purpose? It's not hard to cook up similar examples all over the place.

jasonlfunk 5y ago
Yeah, but they are already doing this for pointless things. They already use facial recognition on all the photos on your phone. That’s what I don’t understand. The only new thing is what they are looking for and their willingness to alert the authorities. The slippery slope argument that this will eventually be used by China to arrest journalists is scare tactics. We have zero evidence that Apple would allow such a thing to happen. And the only thing stopping them is Apple’s word. The fact that they are announcing this should actually give confidence that they aren’t doing it in the shadows for China. They didn’t have to say anything about this. The fact that they should give you confidence that they are respecting your rights, not evidence that they aren’t.

dmitrygr 5y ago

But anything that has the potential to stop or slow it should be fairly considered Strip searching everyone hourly will stop most contraband, including child pornography. And, according to you, it should be fairly considered. Shall we start with you? (And as people do have naked photos of themselves on their phones often[1], make no mistake, the strip search analogy is NOT an exaggeration) [1] https://en.wikipedia.org/wiki/ICloud_leaks_of_celebrity_photos https://en.wikipedia.org/wiki/ICloud_leaks_of_celebrity_phot...

jasonlfunk 5y ago
Considered, sure. But easily rejected. Let’s be grownups here.
- dmitrygr 5y ago
  Yes, indeed, let's! Grownups recognize that strip searches are unwarranted in this and most other cases, be they physical OR digital... And certainly you've heard of software bugs. You do not think that a small one like "accidentally" "forgetting" to check that a photo had been synced cannot happen, and an "accidental" scan of ALL photos couldn't possibly happen in any situation no matter what bug/corruption/etc? Surely you'll share with the class where YOU hire infallible programmers. An accidental strip search is a strip search nonetheless. Luckily, this can all be solved by NOT writing such code. Ever.
  - jpxw 5y ago
    You must surely realise that there’s a fairly large difference between being strip-searched and having a hash computed on one of your photos that you’re uploading to iCloud.
    - dmitrygr 5y ago
      You must surely realize that slippery slopes exist. And as soon as adversarial code exists in your device, it will only expand. Reference: history of literally every dictatorship includes many "reasonable" expansions of power in the name of security/safety/etc...

matwood 5y ago

It's important not to conflate the new features. CSAM uses hashes of known photos and is only run on photos going to iCloud (turning off iCloud turns off CSAM). Photos sent to iCloud have been checked against CSAM for years on the server. The change here is moving it from server to client (which I hope is to make iCloud photos E2E encrypted). Completely agree with your second point. All the 'what ifs' have existed forever. Either iOS users trust Apple will only do what stated or they don't. Nothing has changed.