5 ms·

> Who audits these databases? Where is the oversight to guarantee only appropriate content is included? They are famously opaque because the very viewing of the

by Shank 5y ago

> Who audits these databases? Where is the oversight to guarantee only appropriate content is included? They are famously opaque because the very viewing of the content is illegal. So how can we know that they contain what they are purported to contain? I wholeheartedly agree: there is an audit question here too. The contents of the database are by far the most dangerous part of this equation, malicious or not, targeted or not. I don't like the privacy implications about this, nor the potential for abuse. I would love to see some kind of way to audit the database, or ensure that it's only used "for good." I just don't know what that system is, and I know that PhotoDNA is already in use on other cloud providers. Matthew Green's ongoing analysis [0] is really worth keeping an eye on. For example, there's a good question: can you just scan against a different database for different people? These are the right questions given what we have right now. [0]: https://twitter.com/matthew_d_green/status/1423378285468209153 https://twitter.com/matthew_d_green/status/14233782854682091...

shuckles 5y ago
Matt should read the release before live tweeting FUD. The database is shipped in the iOS image, per the overview, so targeting users is not an issue (roughly).
- pseudalopex 5y ago
  You should understand the difference between a protocol and an easy to change implementation detail before throwing around words like FUD.
  - shuckles 5y ago
    Has Matt redacted any of the FUD from his tweets last night which aren’t true given the published details from today? For example, his claim that the method is vulnerable to black box attacks from GANs isn’t applicable to the protocol because the attacker can’t access model outputs. Furthermore, if “an easy to change implementation detail” in your threat model is anything which could be changed by iOS update, you should’ve stopped using iPhone about 14 years ago.
    - deleted 5y ago
      [deleted]
- pushrax 5y ago
  Is the database frozen or can they push out updates independently of iOS updates? If not, targeting individual users definitely doesn't seem possible unless you control OS signing.
  - shuckles 5y ago
    The database is shipped in the iOS image.
    - pushrax 5y ago
      That's what you wrote originally - and to me it doesn't indicate whether it can also be updated from other sources or not. Lots of content is shipped in the iOS image but can update independently.
      - shuckles 5y ago
        The technical summary provides a lot of detail. I don’t think Apple would omit remote update functionality from it if such capability existed, especially since database poisoning is a real risk to this type of program. I’m comfortable with interpreting the lack of evidence as evidence of absence of such a mechanism. Explicit clarification would certainly help though, but my original point stands: there is positive evidence in the docs which the FUD tweets don’t engage with. In particular, I’m referencing the figure which says that the database of CSAM hashes is “Blinded and embedded” into the client device. That does not sound like an asset the system remotely updates.
        marshf 5y ago
        Do you not see any scenerio where the CIA/OGA inserts a hash into the database to specially target one person or a group of people?
        shuckles 5y ago
        I agree database poisoning is a legitimate threat! Including the database in an iOS release (so it can’t be targeted and updated out of band) mitigates it somewhat. At the end of the day, though, more should be done to make NCMEC’s database transparent and trustworthy. And other databases too, if Apple decides to ship country-specific blacklists.
cwkoss 5y ago
Do the people authorized to query the database have access to view its contents? How many people in the world can certify that it only contains CSAM? I wonder if there are images of US troops committing warcrimes, or politicians doing cocaine, or honeypot intel op blackmail images in there too. Lots of powerful people would love to have an early warning system for leaks of embarrassing non-CSAM images.

by Shank 5y ago

shuckles 5y ago

Matt should read the release before live tweeting FUD. The database is shipped in the iOS image, per the overview, so targeting users is not an issue (roughly).

pseudalopex 5y ago
You should understand the difference between a protocol and an easy to change implementation detail before throwing around words like FUD.
- shuckles 5y ago
  Has Matt redacted any of the FUD from his tweets last night which aren’t true given the published details from today? For example, his claim that the method is vulnerable to black box attacks from GANs isn’t applicable to the protocol because the attacker can’t access model outputs. Furthermore, if “an easy to change implementation detail” in your threat model is anything which could be changed by iOS update, you should’ve stopped using iPhone about 14 years ago.
  - deleted 5y ago
    [deleted]
pushrax 5y ago
Is the database frozen or can they push out updates independently of iOS updates? If not, targeting individual users definitely doesn't seem possible unless you control OS signing.
- shuckles 5y ago
  The database is shipped in the iOS image.
  - pushrax 5y ago
    That's what you wrote originally - and to me it doesn't indicate whether it can also be updated from other sources or not. Lots of content is shipped in the iOS image but can update independently.
    - shuckles 5y ago
      The technical summary provides a lot of detail. I don’t think Apple would omit remote update functionality from it if such capability existed, especially since database poisoning is a real risk to this type of program. I’m comfortable with interpreting the lack of evidence as evidence of absence of such a mechanism. Explicit clarification would certainly help though, but my original point stands: there is positive evidence in the docs which the FUD tweets don’t engage with. In particular, I’m referencing the figure which says that the database of CSAM hashes is “Blinded and embedded” into the client device. That does not sound like an asset the system remotely updates.
      - marshf 5y ago
        Do you not see any scenerio where the CIA/OGA inserts a hash into the database to specially target one person or a group of people?
        shuckles 5y ago
        I agree database poisoning is a legitimate threat! Including the database in an iOS release (so it can’t be targeted and updated out of band) mitigates it somewhat. At the end of the day, though, more should be done to make NCMEC’s database transparent and trustworthy. And other databases too, if Apple decides to ship country-specific blacklists.

cwkoss 5y ago

Do the people authorized to query the database have access to view its contents? How many people in the world can certify that it only contains CSAM? I wonder if there are images of US troops committing warcrimes, or politicians doing cocaine, or honeypot intel op blackmail images in there too. Lots of powerful people would love to have an early warning system for leaks of embarrassing non-CSAM images.