7 ms·

"Investigators said content stored on the encrypted hard drive matched file hashes for known child pornography content." If the disk is encrypted how can they

by xtanx 9y ago

"Investigators said content stored on the encrypted hard drive matched file hashes for known child pornography content." If the disk is encrypted how can they match file hashes? Do they encrypt known CP files with the FileVault key and then compare? If so, isn't that enough to convict him?

infinisil 9y ago
> Do they encrypt known CP files with the FileVault key and then compare? This shouldn't work because if they had the key (which should be encrypted with the password) then they could also just decrypt the rest. Somebody on IRC said that maybe the encrypted filesystem saves hashes of the files unencrypted, but not sure if Apple's FileVault does this.
- Pywarrior 9y ago
  Storing hashes of unencrypted files, would that allow FileVault to verify it was decrypted without error?
  - poizan42 9y ago
    The correct order is hash then encrypt, exactly so you can't do that. Now I don't know if FileVault is doing this correctly, but hopefully it does. Edit: So two people have downvoted me without explanation. Is what I'm saying wrong?
    - maaaats 9y ago
      You can't unhash, so you could never retrieve the original file again.
      - poizan42 9y ago
        What do you mean? You obviously store the encrypted file together with the encrypted hash.
        maaaats 9y ago
        Then I guess me and the downvoters thought you meant hashing the files for encryption :)
    - mdekkers 9y ago
      two people have downvoted me without explanation Sadly, that is the new normal for HN (and, I'll be downvoted for saying something like this)
    - slaymaker1907 9y ago
      You could also hash, encrypt, then hash again if you wanted extra integrity without decrypting. This is technically done with HTTPS through the TCP hash.
stephengillie 9y ago
Something doesn't add up in this story. Encryption and hashing are different processes with different algorithms - SHA vs MD5. For example, IPSec VPNs hash a packet with MD5 to prevent tampering, then encrypt the hash with SHA256 to prevent viewing. (Because the message could be modified while encrypted, were it not also hashed.) Isn't the point of encryption that it doesn't create a reliable hash - that 2 identical files will appear different while encrypted, as part of the larger encrypted drive? Or are encrypted-hash collisions possible when small files are encrypted individually?
- Neverbolt 9y ago
  There are several misconceptions in this comment, first and foremost that SHA is encryption, which it is not. It is a hashing algorithm, not unlike MD5, though "stronger". Secondly, when you have two files that are exactly the same and encrypt both with the same key, method and parameters then both will have the same hash. ( Though I could imagine Apple doing stuff with padding, and other parameters to make this not happen)
  - chatmasta 9y ago
    Right... so for the authorities to "compare" the hash of an encrypted file with that of a known original, they would need to encrypt the original with the same private key used to encrypt the encrypted file. If they had that private key, wouldn't it be sufficient to unlock the drive? They wouldn't need his cooperation to decrypt the drive if they had a private key. So it seems like a catch-22 compelling him to decrypt the drive based on a hash collision.
    - deleted 9y ago
      [deleted]
    - weinzierl 9y ago
      Exactly, but the original quote doesn't say that they compared decrypted content with known hashes. It doesn't say anything about how they learned about the "content stored on the encrypted hard drive". "Investigators said content stored on the encrypted hard drive matched file hashes for known child pornography content." I read it like this: They figured out that the disk had some incriminating files, as I described in another comment of this thread. To make this work hashes are of no use, they need the original files. For various reasons they might not want to admit that they are in possession of the original files, hence the cryptic and vague phrasing.
deleted 9y ago
[deleted]
ivanbakel 9y ago
Hashes might not fall under being "beyond reasonable doubt" - I'd be interested to see the legal history, if any, of them being used in convictions. There'll always be the argument of the pigeonhole principle. On a similar note, I wonder if this will spur interest in a kind of file-doping program to confuse hashes of drive contents. A few pixels won't make a difference if you're planning on just viewing some images.
- geofft 9y ago
  I think they're also looking for other files, whose hashes they don't have forensic evidence of, to secure a stronger conviction (e.g., they might suspect him of producing child pornography and not just downloading it, and so they're looking for files whose hashes they don't already know).
- kakarot 9y ago
  I wonder if the same kind of error-tolerant fingerprinting used in technology like Shazam (recognizing songs from ambient recordings) could be generalized to combat such a doping program. You might have to do more than change a few pixels here and a unicode character or meta tag there. I even suspect things like color grading and, say, something like batching multiple images or text files together into one file, could be accounted for. Not to mention, such a doping program would preferably alter files in an imperceptible (aka, less mutated) fashion. I am a bit of an audiophile with my music and go to great lengths to rip high-quality, lossless, perfectly-encoded tracks for the sake of historical preservation. I imagine some pedophiles feel the same way about their data and the thought of tampering with the data's original state is abhorrent. Even a colorshift or one or two changed pixels might make it worthless in their eyes, much like a bad rip with barely perceptible clicks or an altered noise floor is worthless to me.
  - shpx 9y ago
    https://en.wikipedia.org/wiki/PhotoDNA https://en.wikipedia.org/wiki/PhotoDNA They usually aren't using SHA (maybe they are in this specific case).
nashashmi 9y ago
Depends on the kind of encryption used. If the encryption is convergent encryption, the hash ID stays.
geofft 9y ago
Some Googling finds http://caselaw.findlaw.com/us-3rd-circuit/1853477.html http://caselaw.findlaw.com/us-3rd-circuit/1853477.html , which says there's both an internal hard disk and an external hard disk, and the external one is the one at issue: > The Forensic examination also disclosed that Doe had downloaded thousands of files known by their “hash” values to be child pornography.[3] The files, however, were not on the Mac Pro, but instead had been stored on the encrypted external hard drives. Accordingly, the files themselves could not be accessed. If he's downloading them or storing them by some content-addressable system (torrents, something rsync-like that generates hashes before syncing them, etc.), I can easily believe that there's forensic evidence on the internal hard drive that the files were copied, including the hash of the plaintext, but the files themselves aren't present in plaintext.
- wyager 9y ago
  Ding ding ding. This is the answer. I know OS X also stores hashes of at least some downloads for the purpose of checking certificates on downloaded apps and disk images. This is clearly enough evidence to convict the guy, so I imagine they're holding him for some political reason (like generating jurisprudence for violating the 5th amendment in the future).
  - geofft 9y ago
    I vaguely suspect they're holding him because they expect to get either a stronger conviction or more evidence for other cases from the contents of the external drive (e.g., maybe he's producing child porn or knows people who do).
ja27 9y ago
My understanding is that they know the file hashes and path they were written to from logs of the tool he used to download them.
- lafay 9y ago
  Yes, this. Whatever was used to share or transfer the files generated a cleartext log with MD5 hashes, which was written somewhere other than the encrypted volume.
cmiles74 9y ago
I was thinking perhaps the hash is from something like BitTorrent, where they have a list of hashes that actually identifies components of the file. Then they see that the data for a particular series of matching hashes looks like they were saved to an external drive.
campuscodi 9y ago
Article was updated with the following link to sustain the source of the information: https://arstechnica.com/wp-content/uploads/2017/03/rawlsopinion.pdf#page=5&zoom=auto,-99,637 https://arstechnica.com/wp-content/uploads/2017/03/rawlsopin...
- keehun 9y ago
  Great source. Here's the relevant bits: Agents from the Department of Homeland Security then applied for a federal search warrant to examine the seized devices. Doe voluntarily provided the password for the Apple iPhone 5S, but refused to provide the passwords to decrypt the Apple Mac Pro computer or the external hard drives. Despite Doe’s refusal, forensic analysts discovered the password to decrypt the Mac Pro Computer, but could not decrypt the external hard drives. Forensic examination of the Mac Pro revealed an image of a pubescent girl in a sexually provocative position and logs showing that the Mac Pro had been used to visit sites with titles common in child exploitation, such as "toddler_cp," "lolicam," "tor-childporn," and “pthc.” The Forensic examination also disclosed that Doe had downloaded thousands of files known by their “hash” values to be child pornography. The files, however, were not on the Mac Pro, but instead had been stored on the encrypted external hard drives. Accordingly, the files themselves could not be accessed. So it looks like they got the hashes from logs/forensic evidence collected from an decrypted Mac Pro.
weinzierl 9y ago
> If the disk is encrypted how can they match file hashes? This is an attack, which contemporary block based FDE doesn't really protect you well from. Bitlocker, FileFault, TrueCrypt, VeraCrypt basically operate on one disk block at at time and this means they cannot hide data patterns well. Or as Thomas Ptacek put it in his article "You Don't Want XTS" [1] >It’s ECB-like. It can’t do a perfect job of providing privacy. This is also why Thomas Ptacek and others are advocating that FDE is not a complete replacement for file based encryption. > But that’s the big problem: sector-level encryption sucks. It’s messy, provides fewer security guarantees than conventional message encryption, and makes tradeoffs tailored to the challenges of encrypting disk sectors. > Sector-level crypto is last-resort crypto.[1] The Wikipedia article about ECB[2] (which is not used in current FDE) has a dramatic example where the image of the Linux penguin is clearly recognizable in the ciphertext. [1] https://sockpuppet.org/blog/2014/04/30/you-dont-want-xts/ https://sockpuppet.org/blog/2014/04/30/you-dont-want-xts/ [2] https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation
- wyager 9y ago
  This is 100% wrong. XTS-mode AES absolutely protects you from known-plaintext attacks like the investigators apparently claim to have pulled off. I suspect we don't have the full description.
  - weinzierl 9y ago
    > XTS-mode AES absolutely protects you from known-plaintext attacks like the investigators apparently claim to have pulled off. 1. I didn't claim XTS-mode AES is vulnerable to known-plaintext attacks. 2. I don't believe the investigators claim to have pulled off a known-plaintext attack. What is your source for this?
    - wyager 9y ago
      You said: > Bitlocker, FileFault, TrueCrypt, VeraCrypt basically operate on one disk block at at time and this means they cannot hide data patterns well. This is wrong, but you claimed it. In the context of this thread (matching files on an encrypted disk with known unencrypted files), that's a known-plaintext attack. Maybe you weren't aware you were making this claim, but you did.

by xtanx 9y ago

infinisil 9y ago

> Do they encrypt known CP files with the FileVault key and then compare? This shouldn't work because if they had the key (which should be encrypted with the password) then they could also just decrypt the rest. Somebody on IRC said that maybe the encrypted filesystem saves hashes of the files unencrypted, but not sure if Apple's FileVault does this.

Pywarrior 9y ago
Storing hashes of unencrypted files, would that allow FileVault to verify it was decrypted without error?
- poizan42 9y ago
  The correct order is hash then encrypt, exactly so you can't do that. Now I don't know if FileVault is doing this correctly, but hopefully it does. Edit: So two people have downvoted me without explanation. Is what I'm saying wrong?
  - maaaats 9y ago
    You can't unhash, so you could never retrieve the original file again.
    - poizan42 9y ago
      What do you mean? You obviously store the encrypted file together with the encrypted hash.
      - maaaats 9y ago
        Then I guess me and the downvoters thought you meant hashing the files for encryption :)
  - mdekkers 9y ago
    two people have downvoted me without explanation Sadly, that is the new normal for HN (and, I'll be downvoted for saying something like this)
  - slaymaker1907 9y ago
    You could also hash, encrypt, then hash again if you wanted extra integrity without decrypting. This is technically done with HTTPS through the TCP hash.

stephengillie 9y ago

Something doesn't add up in this story. Encryption and hashing are different processes with different algorithms - SHA vs MD5. For example, IPSec VPNs hash a packet with MD5 to prevent tampering, then encrypt the hash with SHA256 to prevent viewing. (Because the message could be modified while encrypted, were it not also hashed.) Isn't the point of encryption that it doesn't create a reliable hash - that 2 identical files will appear different while encrypted, as part of the larger encrypted drive? Or are encrypted-hash collisions possible when small files are encrypted individually?

Neverbolt 9y ago
There are several misconceptions in this comment, first and foremost that SHA is encryption, which it is not. It is a hashing algorithm, not unlike MD5, though "stronger". Secondly, when you have two files that are exactly the same and encrypt both with the same key, method and parameters then both will have the same hash. ( Though I could imagine Apple doing stuff with padding, and other parameters to make this not happen)
- chatmasta 9y ago
  Right... so for the authorities to "compare" the hash of an encrypted file with that of a known original, they would need to encrypt the original with the same private key used to encrypt the encrypted file. If they had that private key, wouldn't it be sufficient to unlock the drive? They wouldn't need his cooperation to decrypt the drive if they had a private key. So it seems like a catch-22 compelling him to decrypt the drive based on a hash collision.
  - deleted 9y ago
    [deleted]
  - weinzierl 9y ago
    Exactly, but the original quote doesn't say that they compared decrypted content with known hashes. It doesn't say anything about how they learned about the "content stored on the encrypted hard drive". "Investigators said content stored on the encrypted hard drive matched file hashes for known child pornography content." I read it like this: They figured out that the disk had some incriminating files, as I described in another comment of this thread. To make this work hashes are of no use, they need the original files. For various reasons they might not want to admit that they are in possession of the original files, hence the cryptic and vague phrasing.

deleted 9y ago

[deleted]

ivanbakel 9y ago

Hashes might not fall under being "beyond reasonable doubt" - I'd be interested to see the legal history, if any, of them being used in convictions. There'll always be the argument of the pigeonhole principle. On a similar note, I wonder if this will spur interest in a kind of file-doping program to confuse hashes of drive contents. A few pixels won't make a difference if you're planning on just viewing some images.

geofft 9y ago
I think they're also looking for other files, whose hashes they don't have forensic evidence of, to secure a stronger conviction (e.g., they might suspect him of producing child pornography and not just downloading it, and so they're looking for files whose hashes they don't already know).
kakarot 9y ago
I wonder if the same kind of error-tolerant fingerprinting used in technology like Shazam (recognizing songs from ambient recordings) could be generalized to combat such a doping program. You might have to do more than change a few pixels here and a unicode character or meta tag there. I even suspect things like color grading and, say, something like batching multiple images or text files together into one file, could be accounted for. Not to mention, such a doping program would preferably alter files in an imperceptible (aka, less mutated) fashion. I am a bit of an audiophile with my music and go to great lengths to rip high-quality, lossless, perfectly-encoded tracks for the sake of historical preservation. I imagine some pedophiles feel the same way about their data and the thought of tampering with the data's original state is abhorrent. Even a colorshift or one or two changed pixels might make it worthless in their eyes, much like a bad rip with barely perceptible clicks or an altered noise floor is worthless to me.
- shpx 9y ago
  https://en.wikipedia.org/wiki/PhotoDNA https://en.wikipedia.org/wiki/PhotoDNA They usually aren't using SHA (maybe they are in this specific case).

nashashmi 9y ago

Depends on the kind of encryption used. If the encryption is convergent encryption, the hash ID stays.

geofft 9y ago

Some Googling finds http://caselaw.findlaw.com/us-3rd-circuit/1853477.html http://caselaw.findlaw.com/us-3rd-circuit/1853477.html , which says there's both an internal hard disk and an external hard disk, and the external one is the one at issue: > The Forensic examination also disclosed that Doe had downloaded thousands of files known by their “hash” values to be child pornography.[3] The files, however, were not on the Mac Pro, but instead had been stored on the encrypted external hard drives. Accordingly, the files themselves could not be accessed. If he's downloading them or storing them by some content-addressable system (torrents, something rsync-like that generates hashes before syncing them, etc.), I can easily believe that there's forensic evidence on the internal hard drive that the files were copied, including the hash of the plaintext, but the files themselves aren't present in plaintext.

wyager 9y ago
Ding ding ding. This is the answer. I know OS X also stores hashes of at least some downloads for the purpose of checking certificates on downloaded apps and disk images. This is clearly enough evidence to convict the guy, so I imagine they're holding him for some political reason (like generating jurisprudence for violating the 5th amendment in the future).
- geofft 9y ago
  I vaguely suspect they're holding him because they expect to get either a stronger conviction or more evidence for other cases from the contents of the external drive (e.g., maybe he's producing child porn or knows people who do).

ja27 9y ago

My understanding is that they know the file hashes and path they were written to from logs of the tool he used to download them.

lafay 9y ago
Yes, this. Whatever was used to share or transfer the files generated a cleartext log with MD5 hashes, which was written somewhere other than the encrypted volume.

cmiles74 9y ago

I was thinking perhaps the hash is from something like BitTorrent, where they have a list of hashes that actually identifies components of the file. Then they see that the data for a particular series of matching hashes looks like they were saved to an external drive.

campuscodi 9y ago

Article was updated with the following link to sustain the source of the information: https://arstechnica.com/wp-content/uploads/2017/03/rawlsopinion.pdf#page=5&zoom=auto,-99,637 https://arstechnica.com/wp-content/uploads/2017/03/rawlsopin...

keehun 9y ago
Great source. Here's the relevant bits: Agents from the Department of Homeland Security then applied for a federal search warrant to examine the seized devices. Doe voluntarily provided the password for the Apple iPhone 5S, but refused to provide the passwords to decrypt the Apple Mac Pro computer or the external hard drives. Despite Doe’s refusal, forensic analysts discovered the password to decrypt the Mac Pro Computer, but could not decrypt the external hard drives. Forensic examination of the Mac Pro revealed an image of a pubescent girl in a sexually provocative position and logs showing that the Mac Pro had been used to visit sites with titles common in child exploitation, such as "toddler_cp," "lolicam," "tor-childporn," and “pthc.” The Forensic examination also disclosed that Doe had downloaded thousands of files known by their “hash” values to be child pornography. The files, however, were not on the Mac Pro, but instead had been stored on the encrypted external hard drives. Accordingly, the files themselves could not be accessed. So it looks like they got the hashes from logs/forensic evidence collected from an decrypted Mac Pro.

weinzierl 9y ago

> If the disk is encrypted how can they match file hashes? This is an attack, which contemporary block based FDE doesn't really protect you well from. Bitlocker, FileFault, TrueCrypt, VeraCrypt basically operate on one disk block at at time and this means they cannot hide data patterns well. Or as Thomas Ptacek put it in his article "You Don't Want XTS" [1] >It’s ECB-like. It can’t do a perfect job of providing privacy. This is also why Thomas Ptacek and others are advocating that FDE is not a complete replacement for file based encryption. > But that’s the big problem: sector-level encryption sucks. It’s messy, provides fewer security guarantees than conventional message encryption, and makes tradeoffs tailored to the challenges of encrypting disk sectors. > Sector-level crypto is last-resort crypto.[1] The Wikipedia article about ECB[2] (which is not used in current FDE) has a dramatic example where the image of the Linux penguin is clearly recognizable in the ciphertext. [1] https://sockpuppet.org/blog/2014/04/30/you-dont-want-xts/ https://sockpuppet.org/blog/2014/04/30/you-dont-want-xts/ [2] https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation

wyager 9y ago
This is 100% wrong. XTS-mode AES absolutely protects you from known-plaintext attacks like the investigators apparently claim to have pulled off. I suspect we don't have the full description.
- weinzierl 9y ago
  > XTS-mode AES absolutely protects you from known-plaintext attacks like the investigators apparently claim to have pulled off. 1. I didn't claim XTS-mode AES is vulnerable to known-plaintext attacks. 2. I don't believe the investigators claim to have pulled off a known-plaintext attack. What is your source for this?
  - wyager 9y ago
    You said: > Bitlocker, FileFault, TrueCrypt, VeraCrypt basically operate on one disk block at at time and this means they cannot hide data patterns well. This is wrong, but you claimed it. In the context of this thread (matching files on an encrypted disk with known unencrypted files), that's a known-plaintext attack. Maybe you weren't aware you were making this claim, but you did.