10 ms·

How do you know there aren’t bad actors working at the NCMEC? If I know that adding a hash to a list will get it flagged, and I could conveniently arrest or dis

How do you know there aren’t bad actors working at the NCMEC? If I know that adding a hash to a list will get it flagged, and I could conveniently arrest or discredit anyone I wanted, I would certainly send people to work there. How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t. And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights? The point is that it is completely an easy way to get tagged by a government or bad actors as a pedophile. It’s sickening that Apple would let this “technology” into their products.

stetrain 5y ago
If you don’t trust what Apple says about this, why even argue? Apple could be doing all of this and more without telling. I agree with you on the point that the concern here is what various governments may mandate, but if we’re going to argue about Apple’s specific implementation you should probably understand it.
- farmerstan 5y ago
  There's nothing to argue. I'm incredibly disappointed in Apple and feel betrayed. I went all-in with the Apple ecosystem because I stupidly and naively believed their commitment to privacy.
  - stetrain 5y ago
    You don’t seem to have read what Apple has said on the issue so that feels a bit extreme. And for the record I’m not for this, but my concerns are more about what various governments may start mandating as this capability becomes an option. If you want on-device and cloud backup of data that isn’t checked for illegal content, I think that change needs to happen at the legislature not the phone store.
    - farmerstan 5y ago
      I’m telling you what is going to happen in the future. What they write today is meaningless. In 2016 they fought the fbi in terms of unencrypting data. Then they decided to not encrypt iCloud backups. They didn’t mention in 2016 they wouldn’t encrypt iCloud backups just like they won’t say today that they won’t bend to the Chinese government tweaking their algorithm or their hash list in a few yearsnn
      - stetrain 5y ago
        The only logical reason for Apple to implement such a complex system is to give them a defensive political tool to allow them to do things like encrypt iCloud backups and photos. “But CSAM” is a common political tool used against such encryption. I don’t see this move from Apple making any measurable difference in how well a government can scan your device for arbitrary data. If it happens it was going to happen anyway, your prediction of the future comes true with or without CSAM scanning, if you are allowing for new government orders and legislation.
        nieve 5y ago
        The difference is that Apple has built the infrastructure for _global_, continuous image scanning. No need to hack individual phones or know who your targets are, all you have to do is get some images added to list. If you think the Chinese (or most likely American) government wouldn't love to be able to find out everyone who had copies of certain images, say from protests or inside camps, you're far too optimistic. This is Apple creating a mechanism for worldwide surveillance and declaring they'd never compromise because they're good people who we can trust. Note that they're not making any actual promises that might land them in court if the system is abuses, they're just repeating that they're good people who'd never cooperate with authoritarian regimes.
        stetrain 5y ago
        All you have to do is: - Get your image added to the list - Order Apple to scan all images not just ones being uploaded to iCloud - Order Apple to change the reporting threshold or remove the safety voucher system unless you only care about users with many matching images - Compromise or order Apple to skip the human review and pass the reports directly to your government agency Really you have to compromise or order Apple to change every step in the pipeline except “compare images to hashes” which is the part an intern could do in a weekend. I’m sure governments would love to know which users have “undesirable” images on their device. I’m just saying instead of doing the above steps to take advantage of the CSAM system, they could just order Apple to scan for the images they want scanned for. Apple’s choice is the same, comply or risk participation in that market.
cucumb3rrelish 5y ago
By the same logic, how do you know there aren't bad actors working at Apple's software teams? Or your insurance company or bank?
- farmerstan 5y ago
  If you think that FAANG and other tech companies aren't filled with spies from around the world, you're incredibly naive.
nicce 5y ago
> How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t Spoiler alert: they are building the hashing algorithm, so there is at least someone to confirm everything in the beginning. They are not hashing hashes. Stakes are so high in this game that they must be very careful in review. This is not about Youtube copyright claims. > And what about in China? I don’t think that China even cares a lot. It is mandatory to install some apps by law for some minorities. They have the surveillance already.
fortenforge 5y ago
> How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t. As I said, the flagged content is reviewed by an Apple employee before it actually triggers an external report. If the flagged material is not in fact CSAM, it will not be reported. > And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights? First of all, the volume of flagged CSAM content is much, much smaller than the volume of YouTube copyright claims. It's entirely plausible to ensure that a human reviews all flags. Second, Apple is actually constitutionally-barred from automating this step entirely. You can thank Neil Gorsuch's decision in United States v. Ackerman for this. [1] The crux is that since NCMEC is a qausi-governmental entity, automatically sending CSAM-matched content to NCMEC without an Apple employee first inspecting the content would constitute an unreasonable search and seizure and would violate the 4th amendment. [1] https://library.law.virginia.edu/gorsuchproject/united-states-v-ackerman/ https://library.law.virginia.edu/gorsuchproject/united-state...

stetrain 5y ago

If you don’t trust what Apple says about this, why even argue? Apple could be doing all of this and more without telling. I agree with you on the point that the concern here is what various governments may mandate, but if we’re going to argue about Apple’s specific implementation you should probably understand it.

farmerstan 5y ago
There's nothing to argue. I'm incredibly disappointed in Apple and feel betrayed. I went all-in with the Apple ecosystem because I stupidly and naively believed their commitment to privacy.
- stetrain 5y ago
  You don’t seem to have read what Apple has said on the issue so that feels a bit extreme. And for the record I’m not for this, but my concerns are more about what various governments may start mandating as this capability becomes an option. If you want on-device and cloud backup of data that isn’t checked for illegal content, I think that change needs to happen at the legislature not the phone store.
  - farmerstan 5y ago
    I’m telling you what is going to happen in the future. What they write today is meaningless. In 2016 they fought the fbi in terms of unencrypting data. Then they decided to not encrypt iCloud backups. They didn’t mention in 2016 they wouldn’t encrypt iCloud backups just like they won’t say today that they won’t bend to the Chinese government tweaking their algorithm or their hash list in a few yearsnn
    - stetrain 5y ago
      The only logical reason for Apple to implement such a complex system is to give them a defensive political tool to allow them to do things like encrypt iCloud backups and photos. “But CSAM” is a common political tool used against such encryption. I don’t see this move from Apple making any measurable difference in how well a government can scan your device for arbitrary data. If it happens it was going to happen anyway, your prediction of the future comes true with or without CSAM scanning, if you are allowing for new government orders and legislation.
      - nieve 5y ago
        The difference is that Apple has built the infrastructure for _global_, continuous image scanning. No need to hack individual phones or know who your targets are, all you have to do is get some images added to list. If you think the Chinese (or most likely American) government wouldn't love to be able to find out everyone who had copies of certain images, say from protests or inside camps, you're far too optimistic. This is Apple creating a mechanism for worldwide surveillance and declaring they'd never compromise because they're good people who we can trust. Note that they're not making any actual promises that might land them in court if the system is abuses, they're just repeating that they're good people who'd never cooperate with authoritarian regimes.
        stetrain 5y ago
        All you have to do is: - Get your image added to the list - Order Apple to scan all images not just ones being uploaded to iCloud - Order Apple to change the reporting threshold or remove the safety voucher system unless you only care about users with many matching images - Compromise or order Apple to skip the human review and pass the reports directly to your government agency Really you have to compromise or order Apple to change every step in the pipeline except “compare images to hashes” which is the part an intern could do in a weekend. I’m sure governments would love to know which users have “undesirable” images on their device. I’m just saying instead of doing the above steps to take advantage of the CSAM system, they could just order Apple to scan for the images they want scanned for. Apple’s choice is the same, comply or risk participation in that market.

cucumb3rrelish 5y ago

By the same logic, how do you know there aren't bad actors working at Apple's software teams? Or your insurance company or bank?

farmerstan 5y ago
If you think that FAANG and other tech companies aren't filled with spies from around the world, you're incredibly naive.

nicce 5y ago

> How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t Spoiler alert: they are building the hashing algorithm, so there is at least someone to confirm everything in the beginning. They are not hashing hashes. Stakes are so high in this game that they must be very careful in review. This is not about Youtube copyright claims. > And what about in China? I don’t think that China even cares a lot. It is mandatory to install some apps by law for some minorities. They have the surveillance already.

fortenforge 5y ago

> How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t. As I said, the flagged content is reviewed by an Apple employee before it actually triggers an external report. If the flagged material is not in fact CSAM, it will not be reported. > And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights? First of all, the volume of flagged CSAM content is much, much smaller than the volume of YouTube copyright claims. It's entirely plausible to ensure that a human reviews all flags. Second, Apple is actually constitutionally-barred from automating this step entirely. You can thank Neil Gorsuch's decision in United States v. Ackerman for this. [1] The crux is that since NCMEC is a qausi-governmental entity, automatically sending CSAM-matched content to NCMEC without an Apple employee first inspecting the content would constitute an unreasonable search and seizure and would violate the 4th amendment. [1] https://library.law.virginia.edu/gorsuchproject/united-states-v-ackerman/ https://library.law.virginia.edu/gorsuchproject/united-state...