7 ms·

GCM (ie. AES-GCM) has the following problems, which extended variants - those that deterministically randomize (key,nonce) pair - do not solve: Inability to en

GCM (ie. AES-GCM) has the following problems, which extended variants - those that deterministically randomize (key,nonce) pair - do not solve: Inability to encrypt more than 64Gb with the same (key,nonce) pair. Lack of commitment (whether key-commitment, or key+nonce+ad commitment). If one is seriously considering breaking away from existing GCM standards to create yet-another-standard, such proposal would need to offer improvements in all areas (ex. a proposed standard for converting any AEAD into streaming chunk-based AEAD with practically unlimited message sizes under the same (key,nonce) and unlimited message counts. GCM-256 is ubiquitous and is often the preferred choice for all the reasons mentioned by the author, but that very argument is what makes non-standard GCM with 11 AES-rounds silly. In 2023 we should be working on new standards that "wrap" existing crypto-primitives (which are already implemented/available in countless hardware-accelerated libraries/APIs) to get additional features/benefits/capabilities - not musing about AES with 10+1 rounds or SHA-512-really-fast with 80-1 rounds..

tptacek 3y ago
Presumably, if you're going to do an extended-nonce GCM, you could reformat the counter block (since the nonce is encoded in the key anyways) to get rid of the 64Gb limit --- but that complicates the FIPS story, I guess?
- sdrapkin 3y ago
  Indeed. But FIPS is not the only problem. Both the McGrew/Viega spec and subsequent NIST spec of GCM mandate a 4-byte counter - any departure from that would be "no longer GCM".
  - tptacek 3y ago
    Yep. Gross.
  - motohagiography 3y ago
    Is the argument for a small counter that nobody serious will treat it as a significant diversification component or reliable source of entropy, especially in a streaming mode? It's a counter whose function is necesesarily finite and predictable (and reversible?), if not explicitly linear. Intuitively, any substitutions or convolutions derived from it would weaken subsequent operations, no?
- oconnor663 3y ago
  I think the message size limit is a bit of a red herring. Anyone using AES-GCM with messages that large is probably doing sketchy things with unauthenticated plaintext on the decryption side. A non-hazmat/not-just-for-experts cipher really needs to be chunked.
  - colmmacc 3y ago
    I've seen operating on unauthenticated plaintext enough times to list it as my own pet peeve with AES-GCM. But it's a problem for chunked messages too. A few years ago we released a SCRAM mode that makes very minimal changes to AES-GCM so that it mathematically can't operate on unauthenticated plaintext. https://github.com/aws/s2n-tls/tree/main/scram https://github.com/aws/s2n-tls/tree/main/scram
    - oconnor663 3y ago
      > But it's a problem for chunked messages too. I'm curious to hear more about what you've seen. My naive hope was that a proper streaming decrypt API would be enough of a pit of success that developers wouldn't be tempted to sabotage themselves.
      - colmmacc 3y ago
        A great amount of modern systems is copying data from A to B to C. The construct of frontends and backends implies it, or middleware, or proxies, or distributed storage, or blockchains. Even in the most complex systems, latency is one of the easiest core metrics to measure, and is always a priority. It is always lower latency to prefetch the inter-system pipelines, or to use optimistic concurrency and to preprocess data before it has been authenticated. Chunked streaming can make the difference smaller, but even that "small" difference is beyond what is relevant to say ... filling an L1 cache, or waiting a round-trip. Some of the cases of "read before auth" I've seen have been on very small messages, but in contexts where the incentives are even further driven up, like trading or bidding protocols. It just left me thinking that we should enforce AEAD mathematically. Many practitioners often assume it already is enforced!
vlovich123 3y ago
> Inability to encrypt more than 64Gb with the same (key,nonce) pair. I think a better way is to derive the content per file part and then use a ratcheting nonce to encrypt the subparts. That also gives you random access into the entire file in ~O(1) (i.e. no need to decrypt the entire file) and the ability to interrupt and resume decryption. Unfortunately, there's no standard that describes how the output should be serialized, so tool interop becomes a problem. Although, to be fair, there's no serialization standard for AES either (i.e. what do you do with the nonce?) so it's probably not a big deal.
- goalieca 3y ago
  The issue with 64GB is one thing. Not many people encrypt single 64GB files. But people do use aes-gcm for tunnels and encrypting billions of messages and you can only use the key so many times before you have to rotate. Many people screw this part up.
- fargle 3y ago
  I think it's easier than that. When using the IV using a 96 bit fixed field and 32 bit counter, FIPS SP800-38d says you are limited to 2**32 "invocations of the authenticated encryption function with any given fixed field and key". What they call the fixed field is the common 96 bit IV or nonce. So you can do 512Gb or 64GB under one nonce. Then simply increment the fixed field and run the next 64GB under the same key and new nonce (nonce+1) and so on. In essence, it's the same thing is just making the fixed field smaller and the counter bigger, but meets the letter and intent of the law. The "fixed field" can be anything the user wants, including being "constructed from two or more smaller fields". And it is not constrained to remain the same under multiple invocations. Still compatible with FIPS and common implementations. It doesn't have to be some fancy ratcheting scheme. The initial "fixed field" or nonce could even always just be all zeros [1]. It doesn't matter, it's not secret. If for some reason you want to encrypt that much under one key, which I think you really don't. 1: well, in most cases especially AES-256: https://crypto.stackexchange.com/questions/68774/can-a-zero-nonce-be-safely-used-with-aes-gcm-if-the-key-is-random-and-never-used https://crypto.stackexchange.com/questions/68774/can-a-zero-...
  - 3y ago

tptacek 3y ago

Presumably, if you're going to do an extended-nonce GCM, you could reformat the counter block (since the nonce is encoded in the key anyways) to get rid of the 64Gb limit --- but that complicates the FIPS story, I guess?

sdrapkin 3y ago
Indeed. But FIPS is not the only problem. Both the McGrew/Viega spec and subsequent NIST spec of GCM mandate a 4-byte counter - any departure from that would be "no longer GCM".
- tptacek 3y ago
  Yep. Gross.
- motohagiography 3y ago
  Is the argument for a small counter that nobody serious will treat it as a significant diversification component or reliable source of entropy, especially in a streaming mode? It's a counter whose function is necesesarily finite and predictable (and reversible?), if not explicitly linear. Intuitively, any substitutions or convolutions derived from it would weaken subsequent operations, no?
oconnor663 3y ago
I think the message size limit is a bit of a red herring. Anyone using AES-GCM with messages that large is probably doing sketchy things with unauthenticated plaintext on the decryption side. A non-hazmat/not-just-for-experts cipher really needs to be chunked.
- colmmacc 3y ago
  I've seen operating on unauthenticated plaintext enough times to list it as my own pet peeve with AES-GCM. But it's a problem for chunked messages too. A few years ago we released a SCRAM mode that makes very minimal changes to AES-GCM so that it mathematically can't operate on unauthenticated plaintext. https://github.com/aws/s2n-tls/tree/main/scram https://github.com/aws/s2n-tls/tree/main/scram
  - oconnor663 3y ago
    > But it's a problem for chunked messages too. I'm curious to hear more about what you've seen. My naive hope was that a proper streaming decrypt API would be enough of a pit of success that developers wouldn't be tempted to sabotage themselves.
    - colmmacc 3y ago
      A great amount of modern systems is copying data from A to B to C. The construct of frontends and backends implies it, or middleware, or proxies, or distributed storage, or blockchains. Even in the most complex systems, latency is one of the easiest core metrics to measure, and is always a priority. It is always lower latency to prefetch the inter-system pipelines, or to use optimistic concurrency and to preprocess data before it has been authenticated. Chunked streaming can make the difference smaller, but even that "small" difference is beyond what is relevant to say ... filling an L1 cache, or waiting a round-trip. Some of the cases of "read before auth" I've seen have been on very small messages, but in contexts where the incentives are even further driven up, like trading or bidding protocols. It just left me thinking that we should enforce AEAD mathematically. Many practitioners often assume it already is enforced!

vlovich123 3y ago

> Inability to encrypt more than 64Gb with the same (key,nonce) pair. I think a better way is to derive the content per file part and then use a ratcheting nonce to encrypt the subparts. That also gives you random access into the entire file in ~O(1) (i.e. no need to decrypt the entire file) and the ability to interrupt and resume decryption. Unfortunately, there's no standard that describes how the output should be serialized, so tool interop becomes a problem. Although, to be fair, there's no serialization standard for AES either (i.e. what do you do with the nonce?) so it's probably not a big deal.

goalieca 3y ago
The issue with 64GB is one thing. Not many people encrypt single 64GB files. But people do use aes-gcm for tunnels and encrypting billions of messages and you can only use the key so many times before you have to rotate. Many people screw this part up.
fargle 3y ago
I think it's easier than that. When using the IV using a 96 bit fixed field and 32 bit counter, FIPS SP800-38d says you are limited to 2**32 "invocations of the authenticated encryption function with any given fixed field and key". What they call the fixed field is the common 96 bit IV or nonce. So you can do 512Gb or 64GB under one nonce. Then simply increment the fixed field and run the next 64GB under the same key and new nonce (nonce+1) and so on. In essence, it's the same thing is just making the fixed field smaller and the counter bigger, but meets the letter and intent of the law. The "fixed field" can be anything the user wants, including being "constructed from two or more smaller fields". And it is not constrained to remain the same under multiple invocations. Still compatible with FIPS and common implementations. It doesn't have to be some fancy ratcheting scheme. The initial "fixed field" or nonce could even always just be all zeros [1]. It doesn't matter, it's not secret. If for some reason you want to encrypt that much under one key, which I think you really don't. 1: well, in most cases especially AES-256: https://crypto.stackexchange.com/questions/68774/can-a-zero-nonce-be-safely-used-with-aes-gcm-if-the-key-is-random-and-never-used https://crypto.stackexchange.com/questions/68774/can-a-zero-...
- 3y ago