3 ms·

There are numerous incorrect statements in your comment. First: Apple has disclosed who gets to curate the hash list. The answer is NCMEC and other child safet

There are numerous incorrect statements in your comment. First: Apple has disclosed who gets to curate the hash list. The answer is NCMEC and other child safety organizations. https://twitter.com/AlexMartin/status/1424703642913935374/photo/1 https://twitter.com/AlexMartin/status/1424703642913935374/ph... Apple states point-blank that they will refuse any demands to add non-CSAM content to the lists. Second: Why can't the FBI / CCCP inject a hash into the list. Here's a tweet thread gamifying that scenario: https://twitter.com/pwnallthethings/status/1424873629003702273?s=20 https://twitter.com/pwnallthethings/status/14248736290037022... The short answer is that at some point an Apple employee must visually review the flagged photo, and confirm that it does represent CSAM content. If it does not, then Apple is under no legal obligation to report it. Third: You claim that abusers will simply opt not to use iPhones to distribute their CSAM content rendering the feature useless. This is in fact not how things have played out on other platforms like Google and Facebook that do already scan for CSAM. These organizations report on the order of millions of flagged images per year. [1] Clearly the abusers have simply not moved on to a different platform. [1] https://www.businessinsider.com/facebook-instagram-report-20-million-child-sexual-abuse-images-2021-2 https://www.businessinsider.com/facebook-instagram-report-20...

farmerstan 5y ago
How do you know there aren’t bad actors working at the NCMEC? If I know that adding a hash to a list will get it flagged, and I could conveniently arrest or discredit anyone I wanted, I would certainly send people to work there. How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t. And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights? The point is that it is completely an easy way to get tagged by a government or bad actors as a pedophile. It’s sickening that Apple would let this “technology” into their products.
- stetrain 5y ago
  If you don’t trust what Apple says about this, why even argue? Apple could be doing all of this and more without telling. I agree with you on the point that the concern here is what various governments may mandate, but if we’re going to argue about Apple’s specific implementation you should probably understand it.
  - farmerstan 5y ago
    There's nothing to argue. I'm incredibly disappointed in Apple and feel betrayed. I went all-in with the Apple ecosystem because I stupidly and naively believed their commitment to privacy.
    - stetrain 5y ago
      You don’t seem to have read what Apple has said on the issue so that feels a bit extreme. And for the record I’m not for this, but my concerns are more about what various governments may start mandating as this capability becomes an option. If you want on-device and cloud backup of data that isn’t checked for illegal content, I think that change needs to happen at the legislature not the phone store.
      - farmerstan 5y ago
        I’m telling you what is going to happen in the future. What they write today is meaningless. In 2016 they fought the fbi in terms of unencrypting data. Then they decided to not encrypt iCloud backups. They didn’t mention in 2016 they wouldn’t encrypt iCloud backups just like they won’t say today that they won’t bend to the Chinese government tweaking their algorithm or their hash list in a few yearsnn
        stetrain 5y ago
        The only logical reason for Apple to implement such a complex system is to give them a defensive political tool to allow them to do things like encrypt iCloud backups and photos. “But CSAM” is a common political tool used against such encryption. I don’t see this move from Apple making any measurable difference in how well a government can scan your device for arbitrary data. If it happens it was going to happen anyway, your prediction of the future comes true with or without CSAM scanning, if you are allowing for new government orders and legislation.
- cucumb3rrelish 5y ago
  By the same logic, how do you know there aren't bad actors working at Apple's software teams? Or your insurance company or bank?
  - farmerstan 5y ago
    If you think that FAANG and other tech companies aren't filled with spies from around the world, you're incredibly naive.
- nicce 5y ago
  > How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t Spoiler alert: they are building the hashing algorithm, so there is at least someone to confirm everything in the beginning. They are not hashing hashes. Stakes are so high in this game that they must be very careful in review. This is not about Youtube copyright claims. > And what about in China? I don’t think that China even cares a lot. It is mandatory to install some apps by law for some minorities. They have the surveillance already.
- fortenforge 5y ago
  > How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t. As I said, the flagged content is reviewed by an Apple employee before it actually triggers an external report. If the flagged material is not in fact CSAM, it will not be reported. > And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights? First of all, the volume of flagged CSAM content is much, much smaller than the volume of YouTube copyright claims. It's entirely plausible to ensure that a human reviews all flags. Second, Apple is actually constitutionally-barred from automating this step entirely. You can thank Neil Gorsuch's decision in United States v. Ackerman for this. [1] The crux is that since NCMEC is a qausi-governmental entity, automatically sending CSAM-matched content to NCMEC without an Apple employee first inspecting the content would constitute an unreasonable search and seizure and would violate the 4th amendment. [1] https://library.law.virginia.edu/gorsuchproject/united-states-v-ackerman/ https://library.law.virginia.edu/gorsuchproject/united-state...
alfalfasprout 5y ago
You've done nothing to address OP's concerns. The linked twitter thread assumes each actor (NCMEC, FBI, Apple) act in a certain way. There's no "provable" guarantee against an actor acting in bad faith or in a manner inconsistent with certain interpretations of the law (which we've seen routinely with the NSA). The FBI/NSA can absolutely inject something into the hash list. You're assuming that NCMEC needs to be involved. Or that it would be broadly known to Apple. The reality is that the hash list needs to be updated on a different cadence than iOS itself. So it's likely downloaded rather than baked into the OS build permanently. That means that you can't necessarily rely on an iOS build being signed to know if you have a different hash list from everyone else. Ultimately, a small team at Apple cooperating with a secret court order could release a different hashlist to a select set of devices. There's nothing really stopping that. Even if Apple didn't comply, we've seen recently how sophisticated cybersecurity companies armed with zero days can manipulate devices easily. If the mechanism for hash lists scanning the device is already built in all it takes is an exploit changing the hashlist and where it reports to which might be much simpler than gaining full access to the device.
- nieve 5y ago
  It's worse than that. They don't even need to release a different hash list for you, all they need to do is add a few images they know you'll probably have (say from your Facebook or Instagram posts) to the regular DB to meet the match threshold. The people running the database aren't going to be continually going back through old images to double-check they're actually CSAM/theoretically CSAM-related.
blendergeek 5y ago
> Apple states point-blank that they will refuse any demands to add non-CSAM content to the lists. How would Apple know if non-CSAM was added to the list? Apple does not and cannot curate the list. Apple only receives hashes from NCMEC (and other unnamed government agencies). The government does not allow Apple to verify that this list only contains CSAM. This pledge from Apple is at best misguided at worst intentionally dishonest. Of course nobody can make Apple add non-CSAM to the list: Apple doesn't maintain the list.
system2 5y ago
Do you work for Apple or the government? You just sounded like a lawyer talking about similar but unrelated things to win the case.

farmerstan 5y ago

How do you know there aren’t bad actors working at the NCMEC? If I know that adding a hash to a list will get it flagged, and I could conveniently arrest or discredit anyone I wanted, I would certainly send people to work there. How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t. And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights? The point is that it is completely an easy way to get tagged by a government or bad actors as a pedophile. It’s sickening that Apple would let this “technology” into their products.

stetrain 5y ago
If you don’t trust what Apple says about this, why even argue? Apple could be doing all of this and more without telling. I agree with you on the point that the concern here is what various governments may mandate, but if we’re going to argue about Apple’s specific implementation you should probably understand it.
- farmerstan 5y ago
  There's nothing to argue. I'm incredibly disappointed in Apple and feel betrayed. I went all-in with the Apple ecosystem because I stupidly and naively believed their commitment to privacy.
  - stetrain 5y ago
    You don’t seem to have read what Apple has said on the issue so that feels a bit extreme. And for the record I’m not for this, but my concerns are more about what various governments may start mandating as this capability becomes an option. If you want on-device and cloud backup of data that isn’t checked for illegal content, I think that change needs to happen at the legislature not the phone store.
    - farmerstan 5y ago
      I’m telling you what is going to happen in the future. What they write today is meaningless. In 2016 they fought the fbi in terms of unencrypting data. Then they decided to not encrypt iCloud backups. They didn’t mention in 2016 they wouldn’t encrypt iCloud backups just like they won’t say today that they won’t bend to the Chinese government tweaking their algorithm or their hash list in a few yearsnn
      - stetrain 5y ago
        The only logical reason for Apple to implement such a complex system is to give them a defensive political tool to allow them to do things like encrypt iCloud backups and photos. “But CSAM” is a common political tool used against such encryption. I don’t see this move from Apple making any measurable difference in how well a government can scan your device for arbitrary data. If it happens it was going to happen anyway, your prediction of the future comes true with or without CSAM scanning, if you are allowing for new government orders and legislation.
cucumb3rrelish 5y ago
By the same logic, how do you know there aren't bad actors working at Apple's software teams? Or your insurance company or bank?
- farmerstan 5y ago
  If you think that FAANG and other tech companies aren't filled with spies from around the world, you're incredibly naive.
nicce 5y ago
> How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t Spoiler alert: they are building the hashing algorithm, so there is at least someone to confirm everything in the beginning. They are not hashing hashes. Stakes are so high in this game that they must be very careful in review. This is not about Youtube copyright claims. > And what about in China? I don’t think that China even cares a lot. It is mandatory to install some apps by law for some minorities. They have the surveillance already.
fortenforge 5y ago
> How will Apple know whether a hash is for non-CSAM content? Spoiler alert: they won’t. As I said, the flagged content is reviewed by an Apple employee before it actually triggers an external report. If the flagged material is not in fact CSAM, it will not be reported. > And Apple claims it will be reviewed by a human. Sure, just like YouTube copyright claims? Or will it get automated in the near future? And what about in China? Or Saudi Arabia or other countries with less human rights? First of all, the volume of flagged CSAM content is much, much smaller than the volume of YouTube copyright claims. It's entirely plausible to ensure that a human reviews all flags. Second, Apple is actually constitutionally-barred from automating this step entirely. You can thank Neil Gorsuch's decision in United States v. Ackerman for this. [1] The crux is that since NCMEC is a qausi-governmental entity, automatically sending CSAM-matched content to NCMEC without an Apple employee first inspecting the content would constitute an unreasonable search and seizure and would violate the 4th amendment. [1] https://library.law.virginia.edu/gorsuchproject/united-states-v-ackerman/ https://library.law.virginia.edu/gorsuchproject/united-state...

alfalfasprout 5y ago

You've done nothing to address OP's concerns. The linked twitter thread assumes each actor (NCMEC, FBI, Apple) act in a certain way. There's no "provable" guarantee against an actor acting in bad faith or in a manner inconsistent with certain interpretations of the law (which we've seen routinely with the NSA). The FBI/NSA can absolutely inject something into the hash list. You're assuming that NCMEC needs to be involved. Or that it would be broadly known to Apple. The reality is that the hash list needs to be updated on a different cadence than iOS itself. So it's likely downloaded rather than baked into the OS build permanently. That means that you can't necessarily rely on an iOS build being signed to know if you have a different hash list from everyone else. Ultimately, a small team at Apple cooperating with a secret court order could release a different hashlist to a select set of devices. There's nothing really stopping that. Even if Apple didn't comply, we've seen recently how sophisticated cybersecurity companies armed with zero days can manipulate devices easily. If the mechanism for hash lists scanning the device is already built in all it takes is an exploit changing the hashlist and where it reports to which might be much simpler than gaining full access to the device.

nieve 5y ago
It's worse than that. They don't even need to release a different hash list for you, all they need to do is add a few images they know you'll probably have (say from your Facebook or Instagram posts) to the regular DB to meet the match threshold. The people running the database aren't going to be continually going back through old images to double-check they're actually CSAM/theoretically CSAM-related.

blendergeek 5y ago

> Apple states point-blank that they will refuse any demands to add non-CSAM content to the lists. How would Apple know if non-CSAM was added to the list? Apple does not and cannot curate the list. Apple only receives hashes from NCMEC (and other unnamed government agencies). The government does not allow Apple to verify that this list only contains CSAM. This pledge from Apple is at best misguided at worst intentionally dishonest. Of course nobody can make Apple add non-CSAM to the list: Apple doesn't maintain the list.

system2 5y ago

Do you work for Apple or the government? You just sounded like a lawyer talking about similar but unrelated things to win the case.