8 ms·

Go 1.20 Cryptography

204 points by mfrw 4y ago

kibwen 4y ago
I'm under the impression that Go's original crypto libs were written by none other than djb himself. Is there a rationale for the deficiencies laid out in the OP, or am I putting djb on too much of a pedestal?
- FiloSottile 4y ago
  The libraries were originally written by Adam Langley, who did a great job compared to the state of the art at the time. They have been serving us well for over 13 years. Bernstein was never involved.
  - kibwen 4y ago
    Interesting, could have sworn I heard this back in 2011 or so, but searching for a source now I have no clue how I got that impression. :) Thank you!
- remus 4y ago
  > Is there a rationale for the deficiencies laid out in the OP Yes. The previous package had a dependency on the general purpose math/big package which was not constant-time and had a large surface area (that was not required for crypto). This has lead to security bugs in the crypto package.
coder543 4y ago
> What it does not support is other curves, which I am ok with. Ed25519 is relatively nice, but I do wish it also supported Ed448, or at least future-proofed the API so that Ed448 support would be possible as a drop-in replacement. Ed25519 is "128-bit" security, while Ed448 is "224-bit" security. Both are ECDH. 128-bit "ought to be enough for anybody", but for the many applications that asymmetric crypto isn't the bottleneck, it's nice to have the option to pick a stronger curve. Why place this arbitrary limit on the standard library API for the (probably) next decade? It feels counterproductive.
- FiloSottile 4y ago
  The API can easily accommodate adding a curve to crypto/ecdh, just not doing it outside the standard library. The reason not to implement X448 is simply that ~no one uses it, and it would be a lot of maintainer time to make it safe and make it fast. FWIW, no one uses X448 because if X25519 falls it's unlikely to be due to something that leaves us with any confidence in the similar-but-bigger Ed448 curve. It will have to be a cryptanalysis breakthrough or progress in quantum computers, both of which are more likely to break both than only one. The original Ed448 paper is a good read for understanding the "paranoia" motivation behind the curve. https://eprint.iacr.org/2015/625.pdf https://eprint.iacr.org/2015/625.pdf
  - coder543 4y ago
    I'm not exactly a crypto expert, just someone who pays attention to this kind of stuff, so I might be incorrect or have some misconceptions, but even that paper's justification agrees that there are possible outcomes where a stronger curve could prevail in the face of a weakness of Ed25519. There are certainly scenarios where "all crypto becomes invalid" or "ECC becomes invalid" or "curves similar to Ed25519 become invalid", but with Defense In Depth... if all other things are equal, a stronger curve is better, and that is effectively the case here. The only downside of Ed448 that I'm aware of is a slight performance penalty, which is irrelevant in most applications, so there's no reason to actively choose a weaker curve. The "overkill" option is the sensible option in most cases. AFAIK, no one is actually encrypting large sums of data using an asymmetric algorithm like we're discussing. It's typically just used to encrypt a symmetric key, which is then lightning fast to use on the bulk of the data. The performance of the asymmetric algorithm is only important in very specific scenarios. I also believe that "no one" uses Ed448 in much the same way that "no one" uses JPEG XL; a lack of ecosystem support drastically hinders adoption, and it has nothing to do with people believing that Ed448 has no advantage over Ed25519. But, that's just like, my opinion. JPEG XL would be drastically better than the other image format options, but ecosystem support is a chicken and egg problem. Measuring the existing adoption of a poorly supported option doesn't give much insight. If people appreciate Ed25519 (and they really seem to), then offering the stronger version of Ed25519 seems like an obvious next step, even though it is a lot of work (which is why my original comment mentioned future proofing the API, but you seem to indicate that it is, so that's good). If the option existed and were exactly as easy to use, then why wouldn't people pick it for projects where the curve is open for selection?
jupp0r 4y ago
I love the style and detail of this post. There is so much high quality engineering going into the Go ecosystem.
- petercooper 4y ago
  It also helps that Filippo is very diligent and observant. He is one of the most frequent people to send in corrections or extra background information to my Go newsletter :-)
  - FiloSottile 4y ago
    <3 to both :) Love the newsletter. https://golangweekly.com https://golangweekly.com
- codesniperjoe 4y ago
  Go is great. But if you look close enough, go also has some dark corners. The reason the biggest parts of go/crypto is well designed, good readable code, is mostly the work of one person (the author).
  - jupp0r 4y ago
    Any particular dark corners you know of?
    - Jalad 4y ago
      [flagged]
      - Thaxll 4y ago
        "Great" is far fetch especially "you shouldn't use Go for production services", which does not make any sense, also the same author works for a compagny that uses Go in places, which is kind of funny.
    - codesniperjoe 4y ago
fiatjaf 4y ago
Where is secp256k1, the most important curve in the history of cryptography and the one most used by real-world money applications in the world today?
- KAMSPioneer 4y ago
  I honestly can't tell if this is sarcasm or not. In case it isn't, I'm not a cryptography engineer but those who are generally recommend against using secp256k1 in greenfield deployments, and it doesn't seem (to me at least) to be commonly used in existing software either (with one glaring exception of course).
  - 0x4FFC8F 4y ago
    [flagged]
  - adastra22 4y ago
    Two glaring exceptions. Bitcoin and Ethereum both use secp256k1. As well as a host of smaller copycat cryptocurrencies of course. I don’t believe your statement about seco256k1 being depreciated is correct. There are even tradeoffs between the two curve families, approximately equal software and hardware support, and legitimate reasons for preferring one over the other. They are approximately equal performant in practice, and both have high quality, high security implementations.
    - ilyt 4y ago
      Well the scamcoin guys can just use separate library for it. I doubt any of the crypto uses would care about timing attacks either
      - adastra22 4y ago
        The libsecp256k1 library has been ahead of standard curve25519 implementations in terms of protection against timing attacks, implementation foot-guns, and runtime efficiency.
codesniperjoe 4y ago
> Finally, reviewing the BoringCrypto integration by Russ. Do you think there is a rollback/cleanup possible of current boring/fips <enter most diplomatic/nice phrase here> situation? (eg. push it back into a seperate stash, guarded by a fixed build-time compiler directive?)
- FiloSottile 4y ago
  Heh. Look, no one likes FIPS 140. We don’t like it, those that need it don’t like it, sometimes I wonder if NIST likes it. But it is what it is, and the current situation is marginally better than having to fix the merge every time we touch anything. All Go+BoringCrypto code is behind the compile time GOEXPERIMENT, and mostly in its own files or in its own blocks. It could be worse.
  - candiddevmike 4y ago
    Will Go ever be able to encrypt/decrypt ECDH, similar to the RSA package?
codesniperjoe 4y ago
> TLS handshakes now return a CertificateVerificationError if they fail because of, well, certificate verification. ... as long the crypto/x509 CheckSignatureFrom ignores the pathlen contraint (the /ONLY/ way of an CA Owner pin down a delegated SubCA usage/raw-key-abuse!) im not sure that CertifictionVerification does what a high-level api user expects!?
- FiloSottile 4y ago
  CheckSignatureFrom is a low-level API that can't check anything about the path (including more important constraints such as nested EKUs and Name Constraints) because it can only see the immediate parent. The high-level certificate verification API is Verify, which does check all of the above, see https://pkg.go.dev/crypto/x509#TooManyIntermediates https://pkg.go.dev/crypto/x509#TooManyIntermediates. We should probably add a line to the docs, to avoid users getting confused like this, but I haven't seen misused in the wild. (I also disagree that maxPathLen does anything about raw key abuse, since once you have the key of an intermediate you can issue leaves arbitrarily, without needing to issue another intermediate, but that's besides the point.)
  - FiloSottile 4y ago
    There we go. https://go.dev/cl/460537 https://go.dev/cl/460537
    - codesniperjoe 4y ago
      Thank you! That is a quick (helpful) fix! Beside of checks for the integrity of SelfSigned Certificates, I do not used this function directly. My assumption about the TLS impact was based on the fact that [Certificate.Verify] uses internally [Certificate.buildChains] who calls [Certificate.CheckSignatureFrom], who has only the direct parents IsCA status, but not the full picture of the full path (and ignores it at the end, when missing on the root-Anchor)!? And yes, in general, the max pathLen helps only to restict the impact of a specific (delegated) subCA compromise, not for a full root CA key leak. But I expect to see a well prepared narrative about a (revokeable) SubCA as most likely response, to protect the Root-Anchor itself. (What a happy co-incidence that the most Root-Anchors do not declare the planned SubCA strategy via maxPathlen upfront.)
imiric 4y ago
Most of these cryptography changes are a bit over my head, but I'm grateful that Go has such diligent contributors. Every new version improves the language in significant ways, and 1.20 is shaping up to be a big milestone. Filippo is an inspiration as an open source developer. <3
Isolus 4y ago
I'm pretty sure these changes are great for e.g. standard TLS connections. But I don't quite understand what this means for other/custom curves. For example I'm often using brainpool curves. Currently I just set the CurveParams and I'm done. They are part of a lot of official standards (especially in Europe/Germany but also for e.g. travel documents in the ICAO standard) so I can't get around them. Do I have to implement everything for that curves myself? That would probably be more insecure than just using crypto/elliptic.
- deleted 4y ago
  [deleted]
Zamicol 4y ago
Thanks Filippo! I'm excited for Ed25519ph support. Ed25519ph is designed to work with digests, just like the ECDSA algorithms. (ES256, ES384, etc...) The [latest FIPS draft](https://csrc.nist.gov/publications/detail/fips/186/5/draft https://csrc.nist.gov/publications/detail/fips/186/5/draft) is requiring Ed25519ph, so I'm happy to see Go support out-of-the-box. [For reference, see section 7.8](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-5-draft.pdf https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-5-draft...). I've also been looking for Ed25519ph support for other languages. [Paul Miller](https://github.com/paulmillr https://github.com/paulmillr), who is the author of the noble libraries for Javascript has just added support in his newly released [curves](https://github.com/paulmillr/noble-ed25519/issues/63 https://github.com/paulmillr/noble-ed25519/issues/63) library. Paul has suggested on Twitter holding off on using "curves" until an audit, but most of his other work has already been audited and all his works are highly polished. Also, for all readers, we wrote an [online Ed25519 tool](https://cyphr.me/ed25519_applet/ed.html https://cyphr.me/ed25519_applet/ed.html), which is useful for testing and verifying. Previously the top result on Google, which has now been taken down, was sending the keys off to a server, which motivated us to write a tool that didn't openly phone home.

204 points by mfrw 4y ago

I'm under the impression that Go's original crypto libs were written by none other than djb himself. Is there a rationale for the deficiencies laid out in the OP, or am I putting djb on too much of a pedestal?

FiloSottile 4y ago
The libraries were originally written by Adam Langley, who did a great job compared to the state of the art at the time. They have been serving us well for over 13 years. Bernstein was never involved.
- kibwen 4y ago
  Interesting, could have sworn I heard this back in 2011 or so, but searching for a source now I have no clue how I got that impression. :) Thank you!
remus 4y ago
> Is there a rationale for the deficiencies laid out in the OP Yes. The previous package had a dependency on the general purpose math/big package which was not constant-time and had a large surface area (that was not required for crypto). This has lead to security bugs in the crypto package.

coder543 4y ago

> What it does not support is other curves, which I am ok with. Ed25519 is relatively nice, but I do wish it also supported Ed448, or at least future-proofed the API so that Ed448 support would be possible as a drop-in replacement. Ed25519 is "128-bit" security, while Ed448 is "224-bit" security. Both are ECDH. 128-bit "ought to be enough for anybody", but for the many applications that asymmetric crypto isn't the bottleneck, it's nice to have the option to pick a stronger curve. Why place this arbitrary limit on the standard library API for the (probably) next decade? It feels counterproductive.

FiloSottile 4y ago
The API can easily accommodate adding a curve to crypto/ecdh, just not doing it outside the standard library. The reason not to implement X448 is simply that ~no one uses it, and it would be a lot of maintainer time to make it safe and make it fast. FWIW, no one uses X448 because if X25519 falls it's unlikely to be due to something that leaves us with any confidence in the similar-but-bigger Ed448 curve. It will have to be a cryptanalysis breakthrough or progress in quantum computers, both of which are more likely to break both than only one. The original Ed448 paper is a good read for understanding the "paranoia" motivation behind the curve. https://eprint.iacr.org/2015/625.pdf https://eprint.iacr.org/2015/625.pdf
- coder543 4y ago
  I'm not exactly a crypto expert, just someone who pays attention to this kind of stuff, so I might be incorrect or have some misconceptions, but even that paper's justification agrees that there are possible outcomes where a stronger curve could prevail in the face of a weakness of Ed25519. There are certainly scenarios where "all crypto becomes invalid" or "ECC becomes invalid" or "curves similar to Ed25519 become invalid", but with Defense In Depth... if all other things are equal, a stronger curve is better, and that is effectively the case here. The only downside of Ed448 that I'm aware of is a slight performance penalty, which is irrelevant in most applications, so there's no reason to actively choose a weaker curve. The "overkill" option is the sensible option in most cases. AFAIK, no one is actually encrypting large sums of data using an asymmetric algorithm like we're discussing. It's typically just used to encrypt a symmetric key, which is then lightning fast to use on the bulk of the data. The performance of the asymmetric algorithm is only important in very specific scenarios. I also believe that "no one" uses Ed448 in much the same way that "no one" uses JPEG XL; a lack of ecosystem support drastically hinders adoption, and it has nothing to do with people believing that Ed448 has no advantage over Ed25519. But, that's just like, my opinion. JPEG XL would be drastically better than the other image format options, but ecosystem support is a chicken and egg problem. Measuring the existing adoption of a poorly supported option doesn't give much insight. If people appreciate Ed25519 (and they really seem to), then offering the stronger version of Ed25519 seems like an obvious next step, even though it is a lot of work (which is why my original comment mentioned future proofing the API, but you seem to indicate that it is, so that's good). If the option existed and were exactly as easy to use, then why wouldn't people pick it for projects where the curve is open for selection?

jupp0r 4y ago

I love the style and detail of this post. There is so much high quality engineering going into the Go ecosystem.

petercooper 4y ago
It also helps that Filippo is very diligent and observant. He is one of the most frequent people to send in corrections or extra background information to my Go newsletter :-)
- FiloSottile 4y ago
  <3 to both :) Love the newsletter. https://golangweekly.com https://golangweekly.com
codesniperjoe 4y ago
Go is great. But if you look close enough, go also has some dark corners. The reason the biggest parts of go/crypto is well designed, good readable code, is mostly the work of one person (the author).
- jupp0r 4y ago
  Any particular dark corners you know of?
  - Jalad 4y ago
    [flagged]
    - Thaxll 4y ago
      "Great" is far fetch especially "you shouldn't use Go for production services", which does not make any sense, also the same author works for a compagny that uses Go in places, which is kind of funny.
  - codesniperjoe 4y ago

fiatjaf 4y ago

Where is secp256k1, the most important curve in the history of cryptography and the one most used by real-world money applications in the world today?

KAMSPioneer 4y ago
I honestly can't tell if this is sarcasm or not. In case it isn't, I'm not a cryptography engineer but those who are generally recommend against using secp256k1 in greenfield deployments, and it doesn't seem (to me at least) to be commonly used in existing software either (with one glaring exception of course).
- 0x4FFC8F 4y ago
  [flagged]
- adastra22 4y ago
  Two glaring exceptions. Bitcoin and Ethereum both use secp256k1. As well as a host of smaller copycat cryptocurrencies of course. I don’t believe your statement about seco256k1 being depreciated is correct. There are even tradeoffs between the two curve families, approximately equal software and hardware support, and legitimate reasons for preferring one over the other. They are approximately equal performant in practice, and both have high quality, high security implementations.
  - ilyt 4y ago
    Well the scamcoin guys can just use separate library for it. I doubt any of the crypto uses would care about timing attacks either
    - adastra22 4y ago
      The libsecp256k1 library has been ahead of standard curve25519 implementations in terms of protection against timing attacks, implementation foot-guns, and runtime efficiency.

codesniperjoe 4y ago

> Finally, reviewing the BoringCrypto integration by Russ. Do you think there is a rollback/cleanup possible of current boring/fips <enter most diplomatic/nice phrase here> situation? (eg. push it back into a seperate stash, guarded by a fixed build-time compiler directive?)

FiloSottile 4y ago
Heh. Look, no one likes FIPS 140. We don’t like it, those that need it don’t like it, sometimes I wonder if NIST likes it. But it is what it is, and the current situation is marginally better than having to fix the merge every time we touch anything. All Go+BoringCrypto code is behind the compile time GOEXPERIMENT, and mostly in its own files or in its own blocks. It could be worse.
- candiddevmike 4y ago
  Will Go ever be able to encrypt/decrypt ECDH, similar to the RSA package?

codesniperjoe 4y ago

> TLS handshakes now return a CertificateVerificationError if they fail because of, well, certificate verification. ... as long the crypto/x509 CheckSignatureFrom ignores the pathlen contraint (the /ONLY/ way of an CA Owner pin down a delegated SubCA usage/raw-key-abuse!) im not sure that CertifictionVerification does what a high-level api user expects!?

FiloSottile 4y ago
CheckSignatureFrom is a low-level API that can't check anything about the path (including more important constraints such as nested EKUs and Name Constraints) because it can only see the immediate parent. The high-level certificate verification API is Verify, which does check all of the above, see https://pkg.go.dev/crypto/x509#TooManyIntermediates https://pkg.go.dev/crypto/x509#TooManyIntermediates. We should probably add a line to the docs, to avoid users getting confused like this, but I haven't seen misused in the wild. (I also disagree that maxPathLen does anything about raw key abuse, since once you have the key of an intermediate you can issue leaves arbitrarily, without needing to issue another intermediate, but that's besides the point.)
- FiloSottile 4y ago
  There we go. https://go.dev/cl/460537 https://go.dev/cl/460537
  - codesniperjoe 4y ago
    Thank you! That is a quick (helpful) fix! Beside of checks for the integrity of SelfSigned Certificates, I do not used this function directly. My assumption about the TLS impact was based on the fact that [Certificate.Verify] uses internally [Certificate.buildChains] who calls [Certificate.CheckSignatureFrom], who has only the direct parents IsCA status, but not the full picture of the full path (and ignores it at the end, when missing on the root-Anchor)!? And yes, in general, the max pathLen helps only to restict the impact of a specific (delegated) subCA compromise, not for a full root CA key leak. But I expect to see a well prepared narrative about a (revokeable) SubCA as most likely response, to protect the Root-Anchor itself. (What a happy co-incidence that the most Root-Anchors do not declare the planned SubCA strategy via maxPathlen upfront.)

imiric 4y ago

Most of these cryptography changes are a bit over my head, but I'm grateful that Go has such diligent contributors. Every new version improves the language in significant ways, and 1.20 is shaping up to be a big milestone. Filippo is an inspiration as an open source developer. <3

Isolus 4y ago

I'm pretty sure these changes are great for e.g. standard TLS connections. But I don't quite understand what this means for other/custom curves. For example I'm often using brainpool curves. Currently I just set the CurveParams and I'm done. They are part of a lot of official standards (especially in Europe/Germany but also for e.g. travel documents in the ICAO standard) so I can't get around them. Do I have to implement everything for that curves myself? That would probably be more insecure than just using crypto/elliptic.

deleted 4y ago
[deleted]

Zamicol 4y ago

Thanks Filippo! I'm excited for Ed25519ph support. Ed25519ph is designed to work with digests, just like the ECDSA algorithms. (ES256, ES384, etc...) The [latest FIPS draft](https://csrc.nist.gov/publications/detail/fips/186/5/draft https://csrc.nist.gov/publications/detail/fips/186/5/draft) is requiring Ed25519ph, so I'm happy to see Go support out-of-the-box. [For reference, see section 7.8](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-5-draft.pdf https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-5-draft...). I've also been looking for Ed25519ph support for other languages. [Paul Miller](https://github.com/paulmillr https://github.com/paulmillr), who is the author of the noble libraries for Javascript has just added support in his newly released [curves](https://github.com/paulmillr/noble-ed25519/issues/63 https://github.com/paulmillr/noble-ed25519/issues/63) library. Paul has suggested on Twitter holding off on using "curves" until an audit, but most of his other work has already been audited and all his works are highly polished. Also, for all readers, we wrote an [online Ed25519 tool](https://cyphr.me/ed25519_applet/ed.html https://cyphr.me/ed25519_applet/ed.html), which is useful for testing and verifying. Previously the top result on Google, which has now been taken down, was sending the keys off to a server, which motivated us to write a tool that didn't openly phone home.