4 ms·

Hm, I understand the use cases, but I don't understand this: The only way to get the AD is to decrypt the ciphertext, right? Otherwise the data is unauthenticat

Hm, I understand the use cases, but I don't understand this: The only way to get the AD is to decrypt the ciphertext, right? Otherwise the data is unauthenticated, so I assume it's a big no-no to access it. If you need to decrypt the ciphertext to access the AD, why do you care if it was encrypted or not? Basically, I'm not sure why `encrypt(key, nonce, (data, associated data))` (ie adding the AD to your ciphertext, with the encryption framework being unaware of it) is that different from `encrypt(key, nonce, data, associated data)` (ie the AD being a first-class citizen). EDIT: I saw your other message, and this makes it click for me: > authenticated data can include data that doesn't even appear in the message, but rather is derived from the context at the time the message is encrypted and decrypted So the AD can be an additional envelope-level thing at encryption/decryption time, that helps a lot, thanks!

tptacek 2y ago
This is why message routing headers are kind of a fucky example (you can make it make sense but it begs for this confusion). Instead, just take the chunked large-file encryption use case I gave in that comment. The chunk offset isn't recorded anywhere in the ciphertext. It's derived contextually while you decrypt the file. The AD ensures that the decryption will explode if you try to cut and paste chunks of the file into different positions.
- stavros 2y ago
  Yeah, you're right, I was thinking about it in a case where the implementation had the ciphertext being `(block data, chunk offset)` so it did make it part of the message, but it's more elegant for the associated data to be separate from the ciphertext.
  - tptacek 2y ago
    It's a really good question, because, in order to verify the AD, you have to have the same key you need to decrypt it.
    - stavros 2y ago
      Yep, that's the part that throws me. Is it fair to say that it's a more elegant way to include metadata in the ciphertext, without really messing with the plaintext itself? Ie it's basically "just" a way to distinguish the message from its metadata?
      - edoceo 2y ago
        Does that make it like some kind of HMAC?
        hxtk 2y ago
        Yes, in fact, one construction of the AEAD primitive is to use AES-CTR with HMAC to "bolt on" authentication after the fact (AES-CTR on its own is an unauthenticated stream cipher). You can find an implementation of AES-CTR-HMAC (at a high level where AES-CTR and HMAC are both given) here: https://github.com/tink-crypto/tink-go/blob/main/aead/aesctrhmac/aead.go#L75 https://github.com/tink-crypto/tink-go/blob/main/aead/aesctr...
  - firesteelrain 2y ago
    Message headers have a tendency to want to mutate so it makes the problem more complicated to solve for but decrypting chunks in the right order is a good example to grasp because it’s referring to essential metadata that needs to stay open so readers of the data know what to do with it. AD is bound to the cipher text.
- vlovich123 2y ago
  The strategy I used instead was to HKDF derive different keys for each chunk using the offset as part of the info to derive the key. No AD needed.
  - tatersolid 2y ago
    Two calls to SHA-256 for each block would be very slow compared with a modern AEAD.
    - vlovich123 2y ago
      Hmm… it seemed to run at line speed on our machines. I’m also not sure where you’re getting two calls for sha256 from? Like 1 to derive the key (which is sha256 on a very small amount of data) and the second is?
      - tptacek 2y ago
        What does it mean for an AEAD construction to "run at line speed"? Over how many different sessions and with what message sizes and on what hardware?
        vlovich123 2y ago
        As in it's processing at the speed that data can be fed into the CPU. This particular use case was files coming in from the network on 10Gbps hardware but that was about the speed the AES HW ran via openssl perf tests. How many sessions and message sizes are irrelevant. Hardware was AMD EPYC 7642.
        tptacek 2y ago
        It's irrelevant to your use case. It's not irrelevant to the broader CS question of why people don't just do what you're doing.
        2y ago
- andrewflnr 2y ago
  > The AD ensures that the decryption will explode if you try to cut and paste chunks of the file into different positions. Ah, that's the key bit of perspective. Just talking about "context" is so abstract. That's a case where you don't even need to transmit the AD, right? Do you ever have cases where the AD is a mix of transmitted and locally/"contextually" derived data?

tptacek 2y ago

This is why message routing headers are kind of a fucky example (you can make it make sense but it begs for this confusion). Instead, just take the chunked large-file encryption use case I gave in that comment. The chunk offset isn't recorded anywhere in the ciphertext. It's derived contextually while you decrypt the file. The AD ensures that the decryption will explode if you try to cut and paste chunks of the file into different positions.

stavros 2y ago
Yeah, you're right, I was thinking about it in a case where the implementation had the ciphertext being `(block data, chunk offset)` so it did make it part of the message, but it's more elegant for the associated data to be separate from the ciphertext.
- tptacek 2y ago
  It's a really good question, because, in order to verify the AD, you have to have the same key you need to decrypt it.
  - stavros 2y ago
    Yep, that's the part that throws me. Is it fair to say that it's a more elegant way to include metadata in the ciphertext, without really messing with the plaintext itself? Ie it's basically "just" a way to distinguish the message from its metadata?
    - edoceo 2y ago
      Does that make it like some kind of HMAC?
      - hxtk 2y ago
        Yes, in fact, one construction of the AEAD primitive is to use AES-CTR with HMAC to "bolt on" authentication after the fact (AES-CTR on its own is an unauthenticated stream cipher). You can find an implementation of AES-CTR-HMAC (at a high level where AES-CTR and HMAC are both given) here: https://github.com/tink-crypto/tink-go/blob/main/aead/aesctrhmac/aead.go#L75 https://github.com/tink-crypto/tink-go/blob/main/aead/aesctr...
- firesteelrain 2y ago
  Message headers have a tendency to want to mutate so it makes the problem more complicated to solve for but decrypting chunks in the right order is a good example to grasp because it’s referring to essential metadata that needs to stay open so readers of the data know what to do with it. AD is bound to the cipher text.
vlovich123 2y ago
The strategy I used instead was to HKDF derive different keys for each chunk using the offset as part of the info to derive the key. No AD needed.
- tatersolid 2y ago
  Two calls to SHA-256 for each block would be very slow compared with a modern AEAD.
  - vlovich123 2y ago
    Hmm… it seemed to run at line speed on our machines. I’m also not sure where you’re getting two calls for sha256 from? Like 1 to derive the key (which is sha256 on a very small amount of data) and the second is?
    - tptacek 2y ago
      What does it mean for an AEAD construction to "run at line speed"? Over how many different sessions and with what message sizes and on what hardware?
      - vlovich123 2y ago
        As in it's processing at the speed that data can be fed into the CPU. This particular use case was files coming in from the network on 10Gbps hardware but that was about the speed the AES HW ran via openssl perf tests. How many sessions and message sizes are irrelevant. Hardware was AMD EPYC 7642.
        tptacek 2y ago
        It's irrelevant to your use case. It's not irrelevant to the broader CS question of why people don't just do what you're doing.
        2y ago
andrewflnr 2y ago
> The AD ensures that the decryption will explode if you try to cut and paste chunks of the file into different positions. Ah, that's the key bit of perspective. Just talking about "context" is so abstract. That's a case where you don't even need to transmit the AD, right? Do you ever have cases where the AD is a mix of transmitted and locally/"contextually" derived data?