4 ms·

> Encrypting Data Percival, 2009: AES-CTR with HMAC. I just found that AES-CTR + HMAC is very simple and doable in in languages like Go and Javascript for larg

> Encrypting Data Percival, 2009: AES-CTR with HMAC. I just found that AES-CTR + HMAC is very simple and doable in in languages like Go and Javascript for large objects which can't (or shouldn't) all be put into memory at the same time. In fact, a popular google drive client uses this: https://github.com/odeke-em/drive/wiki/End-to-End-Encryption https://github.com/odeke-em/drive/wiki/End-to-End-Encryption (assuming standard secure key generation for both the AES-CTR stream and the HMAC - along with a unique IV) However, reading this article it seems this might no longer be recommended.

dchest 9y ago
It's not recommended not because it's insecure, but because it's easy to get wrong. XSalsa20Poly1305 can also be implemented in a streaming/buffered fashion, it's just that most libraries implement the easier to use interface. If you're encrypting objects that won't fit in RAM, it's worth considering encrypting separate chunks, making sure that the order cannot be changed (e.g. by incrementing a part of nonce for each chunk and setting some flag for the last chunk, see https://github.com/dchest/nacl-stream-js https://github.com/dchest/nacl-stream-js or https://download.libsodium.org/doc/secret-key_cryptography/secretstream.html https://download.libsodium.org/doc/secret-key_cryptography/s...). This allows detecting tampering early without writing malicious results into a file. Again, should be done carefully, as it's easy to get wrong.
- Xeoncross 9y ago
  Can you explain what is easy to get wrong? You want a CSPRNG to generate the IV, HMAC key, and AES key (which Go and node both have). Are their other implementation details that are complex?
  - dchest 9y ago
    I think the most common mistake I've seen is that people forget to use constant-time comparison function to verify MAC. AE/AEAD interfaces just have an open() function which returns NULL instead of the decrypted message if MACs don't match. Also, AES-CTR has only space for 128-bit "IV" (counter that is encrypted), which is sometimes split into 96-bit nonce and 32-bit block counter, sometimes into 64-bit nonce and 64-bit block counter, sometimes IV is used directly and then incremented for each block. 96-bit is on the edge of collisions for randomly generated numbers, 64-bit random nonce is not safe, with 128-bit you're also limited with how much data you can encrypt without collisions... meh. XSalsaPoly accepts 24-byte nonce (can be random) and is good for 2^72 - 32 bytes (practically unlimited). You also mention having separate keys for HMAC and AES, which is good, but you have to care about it (theoretically). With AE/AEAD you don't have to care about it. That is, it seems like you can implement the system you described properly, but many people can't.
  - lvh 9y ago
    Example: if you don't specify that something is the last token, an attacker can trivially truncate.
    - Xeoncross 9y ago
      That is what the 512bit MAC is for
      - lvh 9y ago
        I don't see which 512 bit MAC you're referring to. I'm referring to the chunked encryption scheme 'dchest mentioned.
- jedisct1 9y ago
  https://download.libsodium.org/doc/secret-key_cryptography/encrypted-messages.html https://download.libsodium.org/doc/secret-key_cryptography/e... also provides some guidance on how to do it when the libraries doesn't provide anything off the shelf. Libhydrogen takes a different approach, and supports an optional "message id" (which is AD in disguise) https://github.com/jedisct1/libhydrogen/wiki/Secret-key-encryption https://github.com/jedisct1/libhydrogen/wiki/Secret-key-encr...
tptacek 9y ago
Go already gives you an AEAD API with "Seal" and "Open" and implements it with GCM, which is what you should use in a clean-slate designs if you're sticking to the standard library.
- Xeoncross 9y ago
  This requires the whole plaintext/cipher text to fit into memory. For small blobs I would agree that GCM is perfect, but for anything larger you end up with extra projects like https://github.com/minio/sio https://github.com/minio/sio trying to break streams into smaller ordered chunks. They are on v2 now.

dchest 9y ago

It's not recommended not because it's insecure, but because it's easy to get wrong. XSalsa20Poly1305 can also be implemented in a streaming/buffered fashion, it's just that most libraries implement the easier to use interface. If you're encrypting objects that won't fit in RAM, it's worth considering encrypting separate chunks, making sure that the order cannot be changed (e.g. by incrementing a part of nonce for each chunk and setting some flag for the last chunk, see https://github.com/dchest/nacl-stream-js https://github.com/dchest/nacl-stream-js or https://download.libsodium.org/doc/secret-key_cryptography/secretstream.html https://download.libsodium.org/doc/secret-key_cryptography/s...). This allows detecting tampering early without writing malicious results into a file. Again, should be done carefully, as it's easy to get wrong.

Xeoncross 9y ago
Can you explain what is easy to get wrong? You want a CSPRNG to generate the IV, HMAC key, and AES key (which Go and node both have). Are their other implementation details that are complex?
- dchest 9y ago
  I think the most common mistake I've seen is that people forget to use constant-time comparison function to verify MAC. AE/AEAD interfaces just have an open() function which returns NULL instead of the decrypted message if MACs don't match. Also, AES-CTR has only space for 128-bit "IV" (counter that is encrypted), which is sometimes split into 96-bit nonce and 32-bit block counter, sometimes into 64-bit nonce and 64-bit block counter, sometimes IV is used directly and then incremented for each block. 96-bit is on the edge of collisions for randomly generated numbers, 64-bit random nonce is not safe, with 128-bit you're also limited with how much data you can encrypt without collisions... meh. XSalsaPoly accepts 24-byte nonce (can be random) and is good for 2^72 - 32 bytes (practically unlimited). You also mention having separate keys for HMAC and AES, which is good, but you have to care about it (theoretically). With AE/AEAD you don't have to care about it. That is, it seems like you can implement the system you described properly, but many people can't.
- lvh 9y ago
  Example: if you don't specify that something is the last token, an attacker can trivially truncate.
  - Xeoncross 9y ago
    That is what the 512bit MAC is for
    - lvh 9y ago
      I don't see which 512 bit MAC you're referring to. I'm referring to the chunked encryption scheme 'dchest mentioned.
jedisct1 9y ago
https://download.libsodium.org/doc/secret-key_cryptography/encrypted-messages.html https://download.libsodium.org/doc/secret-key_cryptography/e... also provides some guidance on how to do it when the libraries doesn't provide anything off the shelf. Libhydrogen takes a different approach, and supports an optional "message id" (which is AD in disguise) https://github.com/jedisct1/libhydrogen/wiki/Secret-key-encryption https://github.com/jedisct1/libhydrogen/wiki/Secret-key-encr...

tptacek 9y ago

Go already gives you an AEAD API with "Seal" and "Open" and implements it with GCM, which is what you should use in a clean-slate designs if you're sticking to the standard library.

Xeoncross 9y ago
This requires the whole plaintext/cipher text to fit into memory. For small blobs I would agree that GCM is perfect, but for anything larger you end up with extra projects like https://github.com/minio/sio https://github.com/minio/sio trying to break streams into smaller ordered chunks. They are on v2 now.