6 ms·

The request from the FBI in the San Bernardino case was to change a passcode limit constant and retry timeouts. Those are about as trivial to implement as any o

The request from the FBI in the San Bernardino case was to change a passcode limit constant and retry timeouts. Those are about as trivial to implement as any of the convoluted government coercion database attacks against CSAM detection being proposed here.

ajsnigrutin 5y ago
The difference here is, that apple would have to develop a new feature for them, test it, and waste millions in lawers to protect themselves from accusations of tampering with the evidence (which a software update definitely is, and who knows what FBI wanted in that software update, maybe even to insert a fake sms to the sms database, or many other things a good defense lawyer could bring to the jury). Here, it's different.. let's say there's a new wikileaks, photos of secret documents... and again, first a few journalists get the data, start slowly writing articles, and FBI just adds the hashes to the database, and they can find out who has the photos, with metadata even who had the first, before the journalist, and they can find a leak.
- azinman2 5y ago
  But the FBI can’t just add the hashes to the db. That’s why it’s the intersection of two dbs in two jurisdictions… to prevent exactly that kind of attack. Then they need to pass a human reviewer as well.
  - xkcd-sucks 5y ago
    Five Eyes
    - simondotau 5y ago
      You are suggesting that another country could launder the request on behalf of the USA in order to circumvent the 4th Amendment. Okay then, let's play that out. Australia contacts Apple and demands they augment CSAM detection so that every iPhone in the USA is now scanning for image hashes supplied by Australia. Apple says no. End of hypothetical.
      - salawat 5y ago
        You haven't heard of how GCHQ would happily hand over intelligence they had on U.S. Citizens? I know for a fact there are efforts at creating fusion centers across national boundaries. The question you need to ask is not if but what will make you interesting enough to mobilize against. Thou shalt not build the effing Panopticon, nor it's predecessors. Is that so hard to not do.
        simondotau 5y ago
        I've no doubt that GCHQ would hand whatever they like to whomever they like. But why would Apple comply with a demand from the GCHQ to spy on US citizens?
        brokenmachine 5y ago
        They will come up with some kind of fuckery to get around the rules, like some crypto ridiculousness that makes files that match a certain hash go to servers in a different jurisdiction? Far-fetched, I know...
      - nonlocutor 5y ago
        https://www.lawfareblog.com/legal-tetris-and-fbis-anom-program https://www.lawfareblog.com/legal-tetris-and-fbis-anom-progr... “ The Australian Telecommunications and Other Legislation Amendment (Assistance and Access) Act 2018 (TOLA) allows government agencies to issue “technical assistance” and “technical capability” notices to providers of communications services. The notices require that the providers give the authorities help in conducting criminal enforcement intercepts, and that they make changes in their systems to ensure that they can give that help.” … “In any event, the FBI chose a curiously roundabout way of getting access to ANOM messages. For ANOM devices operating outside of the United States, “an encrypted [blind carbon copy or BCC]” of each message that a user sent was transmitted to a server located outside of the United States, which then decrypted and reencrypted the message with an encryption key known to the FBI. Those reencrypted messages were sent to another server that was owned by the FBI, outside of the United States. In the summer of 2019, the FBI started negotiating to build the legal structure that would make this technical architecture work. In essence, the FBI went looking for a third country that would host the BCC server and could lawfully accept all of the decrypted messages and send the copies to the FBI. As the affidavit notes, “Unlike the Australian beta test, the third country would not review the content in the first instance.” This would have been a fascinating negotiation. Both participants wanted to make criminal cases and avoid privacy scandals. The U.S. would want to be sure that the third country had full legal authority to intercept the contents of every ANOM message, and that the country was also willing to share the full ANOM take with the U.S. in something like real time.”
    - azinman2 5y ago
      It hasn't been announced yet who else's db will be used. If it's not a 'five eyes' member... then what?
- snowwrestler 5y ago
  > FBI just adds the hashes to the database This is the crux of the argument right here, and I have yet to see a detailed description of how the FBI would go about doing that. The most detail I’ve seen is in this thread, which suggests that it would be difficult for the FBI to do it, or at least do it more than once. https://twitter.com/pwnallthethings/status/1424873629003702273 https://twitter.com/pwnallthethings/status/14248736290037022... Has anyone seen something like this in the other direction? Something that walks through “the FBI would do this, then this, etc. and now they’ve coopted Apple’s system”?
simondotau 5y ago
The passcode limit constant is enforced by the secure enclave. I don't know if it's been proven that the secure enclave component of the device can be changed without the device being unlocked. I'm not even sure it possible for any operating system updates to occur on a device which is locked.
- shuckles 5y ago
  Not on the iPhone 5C which was the phone used by the terrorist and did not have an SE. Locked iPhones can be updated from DFU, but I think SE firmware can’t.
- snowwrestler 5y ago
  The technical feasibility of the FBI’s request was never the question, nor the basis of Apple’s objection. Of course it’s even easier for Apple to say “no” to the government if they literally cannot do what the government is asking. That’s the basis of the EFF’s objection to Apple’s plans: they think that by implementing this CSAM system, Apple will turn an impossibility into a possibility.
  - shuckles 5y ago
    We were never in a world where Apple was unable to do what the government was asking. Nothing impossible has been made possible, and Apple has made a stand against small changes - like changing constants - before.
    - HWR_14 5y ago
      It's not "impossibility". It's cost. The FBI cannot force Apple to do free work to circumvent security. Can they force them to add one more hash? Is that work?
      - shuckles 5y ago
        I’m not a lawyer, but at the very least Apple would have to train its reviewers to recognize new image types, redact or update all statements where they explicitly drew a line in the sand about what the feature searched for, likely produce a custom build of iOS, update its knowledge base to distinguish between the root hash of the CSAM database and FBI Imagery database, perhaps re-present consent, add code to target this new database (or subset of the full database) to US users, etc. I’m not a product manager either, but superficially simple changes are often quite complex!

ajsnigrutin 5y ago

The difference here is, that apple would have to develop a new feature for them, test it, and waste millions in lawers to protect themselves from accusations of tampering with the evidence (which a software update definitely is, and who knows what FBI wanted in that software update, maybe even to insert a fake sms to the sms database, or many other things a good defense lawyer could bring to the jury). Here, it's different.. let's say there's a new wikileaks, photos of secret documents... and again, first a few journalists get the data, start slowly writing articles, and FBI just adds the hashes to the database, and they can find out who has the photos, with metadata even who had the first, before the journalist, and they can find a leak.

azinman2 5y ago
But the FBI can’t just add the hashes to the db. That’s why it’s the intersection of two dbs in two jurisdictions… to prevent exactly that kind of attack. Then they need to pass a human reviewer as well.
- xkcd-sucks 5y ago
  Five Eyes
  - simondotau 5y ago
    You are suggesting that another country could launder the request on behalf of the USA in order to circumvent the 4th Amendment. Okay then, let's play that out. Australia contacts Apple and demands they augment CSAM detection so that every iPhone in the USA is now scanning for image hashes supplied by Australia. Apple says no. End of hypothetical.
    - salawat 5y ago
      You haven't heard of how GCHQ would happily hand over intelligence they had on U.S. Citizens? I know for a fact there are efforts at creating fusion centers across national boundaries. The question you need to ask is not if but what will make you interesting enough to mobilize against. Thou shalt not build the effing Panopticon, nor it's predecessors. Is that so hard to not do.
      - simondotau 5y ago
        I've no doubt that GCHQ would hand whatever they like to whomever they like. But why would Apple comply with a demand from the GCHQ to spy on US citizens?
        brokenmachine 5y ago
        They will come up with some kind of fuckery to get around the rules, like some crypto ridiculousness that makes files that match a certain hash go to servers in a different jurisdiction? Far-fetched, I know...
    - nonlocutor 5y ago
      https://www.lawfareblog.com/legal-tetris-and-fbis-anom-program https://www.lawfareblog.com/legal-tetris-and-fbis-anom-progr... “ The Australian Telecommunications and Other Legislation Amendment (Assistance and Access) Act 2018 (TOLA) allows government agencies to issue “technical assistance” and “technical capability” notices to providers of communications services. The notices require that the providers give the authorities help in conducting criminal enforcement intercepts, and that they make changes in their systems to ensure that they can give that help.” … “In any event, the FBI chose a curiously roundabout way of getting access to ANOM messages. For ANOM devices operating outside of the United States, “an encrypted [blind carbon copy or BCC]” of each message that a user sent was transmitted to a server located outside of the United States, which then decrypted and reencrypted the message with an encryption key known to the FBI. Those reencrypted messages were sent to another server that was owned by the FBI, outside of the United States. In the summer of 2019, the FBI started negotiating to build the legal structure that would make this technical architecture work. In essence, the FBI went looking for a third country that would host the BCC server and could lawfully accept all of the decrypted messages and send the copies to the FBI. As the affidavit notes, “Unlike the Australian beta test, the third country would not review the content in the first instance.” This would have been a fascinating negotiation. Both participants wanted to make criminal cases and avoid privacy scandals. The U.S. would want to be sure that the third country had full legal authority to intercept the contents of every ANOM message, and that the country was also willing to share the full ANOM take with the U.S. in something like real time.”
  - azinman2 5y ago
    It hasn't been announced yet who else's db will be used. If it's not a 'five eyes' member... then what?
snowwrestler 5y ago
> FBI just adds the hashes to the database This is the crux of the argument right here, and I have yet to see a detailed description of how the FBI would go about doing that. The most detail I’ve seen is in this thread, which suggests that it would be difficult for the FBI to do it, or at least do it more than once. https://twitter.com/pwnallthethings/status/1424873629003702273 https://twitter.com/pwnallthethings/status/14248736290037022... Has anyone seen something like this in the other direction? Something that walks through “the FBI would do this, then this, etc. and now they’ve coopted Apple’s system”?

simondotau 5y ago

The passcode limit constant is enforced by the secure enclave. I don't know if it's been proven that the secure enclave component of the device can be changed without the device being unlocked. I'm not even sure it possible for any operating system updates to occur on a device which is locked.

shuckles 5y ago
Not on the iPhone 5C which was the phone used by the terrorist and did not have an SE. Locked iPhones can be updated from DFU, but I think SE firmware can’t.
snowwrestler 5y ago
The technical feasibility of the FBI’s request was never the question, nor the basis of Apple’s objection. Of course it’s even easier for Apple to say “no” to the government if they literally cannot do what the government is asking. That’s the basis of the EFF’s objection to Apple’s plans: they think that by implementing this CSAM system, Apple will turn an impossibility into a possibility.
- shuckles 5y ago
  We were never in a world where Apple was unable to do what the government was asking. Nothing impossible has been made possible, and Apple has made a stand against small changes - like changing constants - before.
  - HWR_14 5y ago
    It's not "impossibility". It's cost. The FBI cannot force Apple to do free work to circumvent security. Can they force them to add one more hash? Is that work?
    - shuckles 5y ago
      I’m not a lawyer, but at the very least Apple would have to train its reviewers to recognize new image types, redact or update all statements where they explicitly drew a line in the sand about what the feature searched for, likely produce a custom build of iOS, update its knowledge base to distinguish between the root hash of the CSAM database and FBI Imagery database, perhaps re-present consent, add code to target this new database (or subset of the full database) to US users, etc. I’m not a product manager either, but superficially simple changes are often quite complex!