10 ms·

How to Interchange Ed25519 Keys

69 points by tmfi 5y ago

kodablah 5y ago
Why use ASN.1? Just base64 and send. Besides a few bytes at the end, a Tor v3 onion service address is just a base32'd ed25519 public key.
agwa 5y ago
This is unnecessarily complicated. The beautiful thing about Ed25519 keys (private and public) is that they are simply opaque byte strings. Indeed, here is Go's definition of ed25519.PublicKey: type PublicKey []byte So if your application uses exclusively Ed25519, you can just pass around the raw bytes (encoded as base64 as desired). There is no need to muck around with ASN.1. And if you do need to transmit keys in ASN.1 structures, Go's standard library has functions for that: https://golang.org/pkg/crypto/x509/#MarshalPKIXPublicKey https://golang.org/pkg/crypto/x509/#MarshalPKIXPublicKey https://golang.org/pkg/crypto/x509/#ParsePKIXPublicKey https://golang.org/pkg/crypto/x509/#ParsePKIXPublicKey The author has just written worse versions of those functions.
- psanford 5y ago
  Came here to say that the example code in the docs for ParsePKIXPublicKey seems to be exactly what the author was looking for.
- userbinator 5y ago
  All keys are just opaque strings of bytes.
  - agwa 5y ago
    They actually aren't. RSA keys, for instance, are defined as a pair of integers (modulus and exponent)[1] and the algorithms are specified as taking these integers as input. ECDSA is similar. Therefore, to interchange RSA and ECDSA keys, you need some other specification that tells you how to serialize those integers. It's true that internally Ed25519 keys are also integers, but the encoding/decoding is encapsulated in the Ed25519 algorithm so to users of Ed25519 they are just byte strings. This is quite a bit nicer in practice than the approach taken by RSA and ECDSA. [1] https://tools.ietf.org/html/rfc8017#section-3 https://tools.ietf.org/html/rfc8017#section-3
    - toast0 5y ago
      Much fun[1] can be had because RSA keys are integers. If your key would be encoded with a zero most significant byte, some implementations are fine if that byte is omited, and some will give a nonsensical error instead. [1] Not actually fun, but I got a patch in OpenSSL, so I guess that's neat.
      - kaoD 5y ago
        Adding to the fun: some implementations bark at you if you include it!
  - deleted 5y ago
    [deleted]
  - jnwatson 5y ago
    If you're going there, all information can be opaque strings of bytes.
- runeks 5y ago
  > Indeed, here is Go's definition of ed25519.PublicKey […] Wouldn’t you need some sort of length restriction on that? Or are all byte sequences valid Ed25519 keys — e.g. the empty byte sequence and the byte sequence that comprises this[1] cat GIF? https://i.pinimg.com/originals/c3/2b/fa/c32bfa16bcf864e478d3ddfe32440268.gif https://i.pinimg.com/originals/c3/2b/fa/c32bfa16bcf864e478d3...
  - lawl 5y ago
    > Wouldn’t you need some sort of length restriction on that? https://github.com/golang/go/blob/master/src/crypto/ed25519/ed25519.go#L203 https://github.com/golang/go/blob/master/src/crypto/ed25519/... It seems it would indeed panic with an invalid size.
  - madars 5y ago
    Yup! From https://cr.yp.to/ecdh.html https://cr.yp.to/ecdh.html > How do I validate Curve25519 public keys? > Don't. The Curve25519 function was carefully designed to allow all 32-byte strings as Diffie-Hellman public keys. (minor exceptions follow)
  - agwa 5y ago
    Public keys are 32 bytes long, and private keys are 64 bytes long. The Ed25519 functions check to make sure the key is the correct length before using them: https://github.com/golang/go/blob/d4bfe006155ea0d9c2970e22fca5a027125c8a98/src/crypto/ed25519/ed25519.go#L204 https://github.com/golang/go/blob/d4bfe006155ea0d9c2970e22fc... The types would ideally use fixed-length arrays, but at the time the API was designed, Go didn't allow conversions between slices and array pointers, which would have made the API annoying to use: https://github.com/golang/go/issues/24376#issuecomment-372786394 https://github.com/golang/go/issues/24376#issuecomment-37278... Go is now adding support for such conversions, but the Ed25519 API can't be changed due to backwards compatibility guarantees.
- timbray 5y ago
  Fair enough, thanks, will adopt those. An interesting question is why heavy googling and stackoverflow diving looking for "how to publish ed25519" and related things failed to turn this up.
  - nabla9 5y ago
    Call me old fashioned, but I browse and search the documentation first. Starting from the index page. "pkix - Package pkix contains shared, low level structures used for ASN.1 parsing and serialization of X.509 certificates, CRL and OCSP."
    - timbray 5y ago
      Right, what path would have led you to that doc page. I read a lot of doc pages and none of the paths I followed led to that one.
      - nabla9 5y ago
        Just one click away. Go to https://golang.org/ https://golang.org/ click [Packages] Scroll down until you see crypto packages. Now you should see package named pkix and description of the package: pkix - contains shared, low level structures used for ASN.1 parsing and serialization of X.509 certificates, CRL and OCSP.
        timbray 5y ago
        Call me weird, but I tend to start by searching for what I think I'm looking for. I was looking for "serialize 25519" or something like that. None of "pkix", "ASN.1", "CRL", X.509, o "OCSP" were in my vocabulary when I started.
        nabla9 5y ago
        I see. 1. You needed to learn how the cryptographic serialization happens in practice. Including terminology. 2. You forgot to look again after you had learned all you needed to learn and wrote your own. It's easy to wander from learning to coding.
- ComodoHacker 5y ago
  >opaque byte strings The point was to have something you (an average Joe) can share on the Internet, and that's definitely not byte string. But the whole PKI thing is indeed very complicated. After such thorough analysis I was expecting some novel and simple solution like memorizable passphrase or QR code to scan.
- evanelias 5y ago
  As a non-crypto person who coincidentally just spent a couple days mucking with this in Go earlier this week, I absolutely agree with the author's assertion that it's very hard to find complete examples, and even articles that don't assume a decent baseline of domain expertise. fwiw my own thought/research path, after choosing Ed25519, looked something like this: 1. "How do I generate the key pair? I should probably use openssl since I trust that and have used it in the past." After googling, I see I can use `openssl genpkey -algorithm ed25519` for the private key. 2. "OK, that gave me a pem file. How do I deal with that in Go?" From here I found the same blainsmith.com article as OP, which is overall very helpful but unfortunately contains the unnecessary asn1.Unmarshal steps. 3. "Hmm, this looks overly complex." Luckily I took some extra time to look at other search results for "ed25519 pem golang" and found the crypto/x509 functions linked above. In retrospect, if I had just used ed25519.GenerateKey(nil) for step 1, I would have saved a ton of time. But this really wasn't initially clear to me at all. Exactly as the OP said, "Because as I stumbled through the undergrowth figuring this stuff out in the usual way, via Google and StackOverflow, I did not convince myself that the path I was following was the right one." And in this case the OP is a highly respected software engineer with his own wikipedia page, so at least I'm in good company for tripping all over this process :)
  - chasil 5y ago
    Since Microsoft strapped OpenSSH into Windows 10, we can assume ssh-keygen is the most ubiquitous ed25519 management tool; anything using OpenSSH format will be widely supported. Does this run just about everywhere? ssh-keygen -t ed25519 -f testkey
    - evanelias 5y ago
      That likely runs everywhere, but in my mind it doesn't solve the core problems here: * The keys are each really just 32 byte sequences. So any tool that serializes them in a more complicated structure is just adding unnecessary complexity, if the calling code only cares about ed25519 keys anyway. * Lack of end-to-end examples in documentation and blog posts.
- timbray 5y ago
  Just committed changes to use those functions. Didn't remove all the ceremony but saved a few lines of code. Thanks. Credited @agwa in the commit.
- FiloSottile 5y ago
  Came here to say this. Please don't use PKIX, ASN.1, and Base64-of-DER-without-PEM-headers for Ed25519. Those are all extra complexity from protocols that were misguidedly built with runtime agility, or from a different time in cryptography engineering in which we really felt the need to put dynamic types on everything. [0][1] An Ed25519 public key is a 32 bytes sequence. An Ed25519 private key (called seed in crypto/ed25519 for unfortunate historical standardization reasons) is a 32 byte sequence. That's it. Here they are encoded in Base64. public key: 5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ= private key: shhKyGTvTeLjXGDCjEQgHRA7ps3LRNNfoO5S714kinU= age [2] similarly uses Curve25519 keys encoded with Bech32 to make them easier to copy-paste and read aloud. Look how nice they look compared to PKIX blobs! $ age-keygen # created: 2021-04-23T12:31:14-04:00 # public key: age1gek0nrawzg9fhkrzcmt4ql7au0n6hwflz7lqqc8wwcvefn2vssgsa4uulp AGE-SECRET-KEY-19JFPFAY2DF3HJP5DFGVSY4A4G4YSRHG4ZCJMKNC5MFD9C9ZN5LCSG8VTDL I'll think about how to make the crypto/ed25519 godoc point people in the right direction once Go 1.17 enters feature freeze. [0] https://www.imperialviolet.org/2016/05/16/agility.html https://www.imperialviolet.org/2016/05/16/agility.html [1] https://buttondown.email/cryptography-dispatches/archive/cryptography-dispatches-registries-considered/ https://buttondown.email/cryptography-dispatches/archive/cry... [2] https://github.com/FiloSottile/age https://github.com/FiloSottile/age
  - timbray 5y ago
    This makes sense, but if I'm handing these things around probably some of the code [gasp] isn't in Go. So someone should write an RFC for the case where you don't need algorithm agility, and then devs don't need to know or care in the slightest what 25519 keys actually are, they just need to call APIs to serialize & deserialize them.
    - FiloSottile 5y ago
      I'd argue the RFC is already there and it's RFC 8032, which defines "32 bytes string" <-> "public key" and "32 bytes string" <-> "private key" APIs. Then you can use your preferred format to serialize a 32 bytes string if you need a text-safe encoding. (If you don't you're done!) For example, you can use RFC 4648. I am positive every language has Base64 code, and if there's an Ed25519 library out there that can't accept 32 byte strings we should talk to the author because it's broken as it doesn't implement RFC 8032. What else would a serialization RFC say, that is not already said in RFC 8032? Deciding that keys are encoded with Base64 and not with, say, hex seems like a weird thing to force on people.
- rauhl 5y ago
  The trouble with using just a bag of (potentially Base64-encoded) bytes is that it flirts with cryptographic doom: one relies on users not to make mistakes. For example, is 5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ= a Base64-encoded Ed25519 public key? Sure. Is it a Base64-encoded HMAC-SHA-256 key? Sure! It’s 256 bits, after all. Is it a valid AES-256 key? Yes again. OTOH, (public-key (ed25519 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly not an HMAC private key. For that matter, (secret-key (hmac-sha-256 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly not an Ed25519 public key. And (secret-key (aes-256 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly neither a public Ed25519 key nor a private HMAC-SHA-256 key. Software expecting one and getting one of the other will break. Why does this matter? Because using a public value such as a public key as an HMAC signing key means that anyone can generate a signature, and using a public value such as a public key as a secret encryption key means that anyone can decrypt the encrypted material. Why use the unusual formats I suggested above? Because this: (public-key (ed25519 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly preferable to this: -----BEGIN PUBLIC KEY----- MDYwBQYDK2VwAy0ANXVXN2FuRUdGMW5JakdmcDVwUzJraU4wY24ybUdZa3VTYStUQ0JvRkliUT0 -----END PUBLIC KEY-----
  - timbray 5y ago
    The conventional PKIX/PEM is logically equivalent to your s-expression there; I assume you're arguing for using the conventional packing rather than proposing a new s-expression based encoding?
    - rauhl 5y ago
      For your purposes, I think that your choice of PEM is probably the best. As you note, it is logically equivalent to the s-expression (it is typed, not a bag of bytes), and it has the advantage of being a well-known format. If æsthetic concerns are a higher priority than interoperability (say, for a personal hobby project), then I do recommend folks adopt the format I gave an example of. It has some history behind it, dating back to the 1997–1999 timeframe and the IETF’s SPKI Working Group (e.g. https://theworld.com/~cme/examples.txt https://theworld.com/~cme/examples.txt) which resulted in RFCs 2692 & 2693, as well as some great drafts (https://people.csail.mit.edu/rivest/Sexp.txt https://people.csail.mit.edu/rivest/Sexp.txt & https://theworld.com/~cme/spki.txt https://theworld.com/~cme/spki.txt) which never progressed because Reasons™. I still think that the SPKI project got PKI right. Oh well, what might have been!
- 1vuio0pswjnm7 5y ago
  Wouldnt it make more sense to use base32 instead of base64. Also whats wrong with simply using the hex string of the public key. It is not much longer than the base64/32 string. I thought the short length was an intentional part of djb's design. Small public keys, short hex strings. Easy to share.
formerly_proven 5y ago
Why exactly would you do any of this instead of a) just having a field of 32 bytes in the wire protocol (if binary) b) just turning it into a hex string (if text is needed) or base64/85 if you don't want hex for some reason
- viraptor 5y ago
  > Why exactly would you do any of this Because it's your first adventure in the land of crypto and you have no idea how to do things, which parts you need to learn about and which are not required. It seems trivial for me now, but there was a time I would end up on a similar journey. I don't know of a source which would do a quick walk through all of these levels. And if there is one, I don't know how you'd find it without knowing the right keywords. (Now I'm tempted to write up some "crypto and certificates and public keys and other things explained quickly and without depth so you can ask better questions next" post)
  - formerly_proven 5y ago
    Fair enough, if you don't know "the customs" it's probably very easy to end up looking for "public key formats" and fall into the PKIX/X.509/ASN.1 rabbit hole.
    - cordite 5y ago
      It’s actually quite an exciting rabbit hole to explore with not many useful Google search results to educate on. Opinions are my own :)
    - alfiedotwtf 5y ago
      Rabbit hole is a polite way to put it. Antiquated pile of shit is getting closer to the truth
  - Aachen 5y ago
    Not the person you replied to, but > Because it's your first adventure in the land of crypto and you have no idea how to do things is rather presumptuous. I do actually know how to do crypto things (I review and test such things for a living) and after reading their comment and skimming the post I'm left wondering the exact same thing.
latortuga 5y ago
There's a typo in the HN title, it should be 25519, not 25119.
- timbray 5y ago
  Fixed, thanks
- dang 5y ago
  Fixed, thanks :)
kfreds 5y ago
First of all, I enjoyed reading this article. Second, it inspired me to write the following, with curious beginners as the target audience. It was meant to be about the importance of reducing the computing power of recognizers / parsers / interfaces / meaning-makers, as the author brings up PEM, ASN.1, and X.509. It grew a bit. Enjoy. <rant> In my opinion security exists to facilitate trust, and security has a lot to do with managing complexity. One of the most powerful security technologies we have is math. Especially math that is easy to do in one direction and very difficult to do in the opposite direction. RSA, Ed25519, Diffie-Hellman, and SHA256 all rely on this concept. Another power tool is combining these cryptographic primitives to build protocols that provide e.g. authentication, forward secrecy, confidentiality, and integrity-protection. When using cryptographic primitives directly it is easy to make mistakes. For example, before Ed25519 there were other similar signature schemes that, in addition to a message to be signed, required a number that must not be repeated for different messages, called a nonce. If this number was repeated, an attacker could collect signatures and combine them to get the private key. Ed25519 removes this sharp corner by essentially using nonce=SHA512(message) internally. When designing a protocol like TLS, SSH, Signal, or WireGuard, multiple primitives of different types are combined to get the desired security properties. Cryptographers work hard to combine primitives such that the resulting construction is easy to reason about, and ideally prove that the construction will only break if one of the primitives break. Unfortunately this key insight is often ignored further up the stack. Many protocols, file formats, etc are unnecessarily complex. This makes them harder to reason about, which makes it harder to build a safe recognizer / parser / interface / meaning-maker for them. Complex parsers lead to incorrect mental models, which lead to security vulnerabilities. If you are ever tasked with building a protocol, file format, or API, consider making it only as complex as it must be. We will suffer for a long time with the complex beasts that have already been standardized. Let’s not make more of them. Remember that security, trustworthiness, understanding, and managing complexity are intertwined. Don’t use more complexity than necessary. </rant> P.S. If anyone is curious about Ed25519 I can highly recommend the following article: https://blog.mozilla.org/warner/2011/11/29/ed25519-keys/ https://blog.mozilla.org/warner/2011/11/29/ed25519-keys/
ajrw 5y ago
JWK is a good option for representing public keys, for an Ed25519 key you would have something like this: { "kty":"OKP", "crv":"Ed25519", "x":"oF0a6lgwrJplzfs4RmDUl-NpfEa0Gc8s7IXei9JFRZ0" }
nemo1618 5y ago
I've always found PEM/ASN.1 to be rather ugly. I encode my ed25519 keys as "ed25519:<hex key>".
- deleted 5y ago
  [deleted]
upofadown 5y ago
>...if ed25519 fails, it’ll fail slowly and there’ll be time to migrate to something else. Why would you assume that? The curve in question has been around for only something like 10 years....
wahern 5y ago
Ed25519 isn't quite as simple as people make it out to be, at least in the sense that it's an answer to a problem that's been entirely elided in these discussions, whilst the problems with older alternatives are more familiar. Both RSA and NIST-defined elliptic curves support both signatures and key exchange using the same public key. Not so with Curve25519. Ed25519 is just for signatures and X25519 for key exchange, and the "opaque" 32-byte public keys are actually different. You can convert Ed25519 public keys to X25519 public keys, but for a reverse transformation there are two incompatible methods, and so any protocol where you would like to keep a single public key association for parties--particularly offline protocols where a sender can't ask the recipient to generate an ephemeral key pair for confidential message reception--one is faced with a couple of choices. And the choices made by some existing protocols aren't always obvious. Indeed, the very fact that these differences exist isn't necessarily obvious until you're already knee-deep in the weeds. At this point the solution is more-or-less settled--just publish Ed25519 keys. But it wasn't always that way considering that the Ed25519 signature scheme is much newer than Curve25519. And of course the legacy of this complexity will always be around to some extent. Also left undiscussed is the different modes of Ed25519--regular and prehashed message digestion, a choice that is absent for older schemes. That's a whole 'nother can of worms. Time and usage effect almost every new protocol and format--they don't stay pristine forever. The sheen has already dulled on Curve25519. If you want the fewest possible interoperability headaches Curve448 is the new hotness, if only because Curve448 has been standardized more times than actually deployed. It's also more secure--longer keys[1] and slightly better mathematical properties, AFAIU, of the type that originally made Curve25519 and DJB's SafeCurves initiative appealing. [1] Arguably useful if there's an extended period of time between quantum computing scaling to threaten 128-bit classically equivalent but non-quantum-safe asymmetric schemes, and scaling to threaten ~256-bit equivalent schemes. You're probably better off with RSA in that case, though. Or just not worrying about it and sticking to Curve25519--one of the few times where "but 32 is such a nice, round, convenient string length" is perhaps a legitimate argument.
- tialaramex 5y ago
  The trouble is that if you use my public signing key to do something else it wasn't intended for (here, X25519) I can't participate without also using my private key in this unconventional way. Now, this looks safe which is more than can be said for the equivalent trick in RSA, which looks obviously dangerous and in practice has caused real problems, but it sure would be comforting if people did all the heavy lifting to prove it is safe rather than shrugging and doing it anyway (e.g. 'age') Meanwhile it feels like the lesson we learned aeons ago in cryptography should apply, if we can't prove why it's safe, let's just assume it isn't safe.

69 points by tmfi 5y ago

Why use ASN.1? Just base64 and send. Besides a few bytes at the end, a Tor v3 onion service address is just a base32'd ed25519 public key.

agwa 5y ago

This is unnecessarily complicated. The beautiful thing about Ed25519 keys (private and public) is that they are simply opaque byte strings. Indeed, here is Go's definition of ed25519.PublicKey: type PublicKey []byte So if your application uses exclusively Ed25519, you can just pass around the raw bytes (encoded as base64 as desired). There is no need to muck around with ASN.1. And if you do need to transmit keys in ASN.1 structures, Go's standard library has functions for that: https://golang.org/pkg/crypto/x509/#MarshalPKIXPublicKey https://golang.org/pkg/crypto/x509/#MarshalPKIXPublicKey https://golang.org/pkg/crypto/x509/#ParsePKIXPublicKey https://golang.org/pkg/crypto/x509/#ParsePKIXPublicKey The author has just written worse versions of those functions.

psanford 5y ago
Came here to say that the example code in the docs for ParsePKIXPublicKey seems to be exactly what the author was looking for.
userbinator 5y ago
All keys are just opaque strings of bytes.
- agwa 5y ago
  They actually aren't. RSA keys, for instance, are defined as a pair of integers (modulus and exponent)[1] and the algorithms are specified as taking these integers as input. ECDSA is similar. Therefore, to interchange RSA and ECDSA keys, you need some other specification that tells you how to serialize those integers. It's true that internally Ed25519 keys are also integers, but the encoding/decoding is encapsulated in the Ed25519 algorithm so to users of Ed25519 they are just byte strings. This is quite a bit nicer in practice than the approach taken by RSA and ECDSA. [1] https://tools.ietf.org/html/rfc8017#section-3 https://tools.ietf.org/html/rfc8017#section-3
  - toast0 5y ago
    Much fun[1] can be had because RSA keys are integers. If your key would be encoded with a zero most significant byte, some implementations are fine if that byte is omited, and some will give a nonsensical error instead. [1] Not actually fun, but I got a patch in OpenSSL, so I guess that's neat.
    - kaoD 5y ago
      Adding to the fun: some implementations bark at you if you include it!
- deleted 5y ago
  [deleted]
- jnwatson 5y ago
  If you're going there, all information can be opaque strings of bytes.
runeks 5y ago
> Indeed, here is Go's definition of ed25519.PublicKey […] Wouldn’t you need some sort of length restriction on that? Or are all byte sequences valid Ed25519 keys — e.g. the empty byte sequence and the byte sequence that comprises this[1] cat GIF? https://i.pinimg.com/originals/c3/2b/fa/c32bfa16bcf864e478d3ddfe32440268.gif https://i.pinimg.com/originals/c3/2b/fa/c32bfa16bcf864e478d3...
- lawl 5y ago
  > Wouldn’t you need some sort of length restriction on that? https://github.com/golang/go/blob/master/src/crypto/ed25519/ed25519.go#L203 https://github.com/golang/go/blob/master/src/crypto/ed25519/... It seems it would indeed panic with an invalid size.
- madars 5y ago
  Yup! From https://cr.yp.to/ecdh.html https://cr.yp.to/ecdh.html > How do I validate Curve25519 public keys? > Don't. The Curve25519 function was carefully designed to allow all 32-byte strings as Diffie-Hellman public keys. (minor exceptions follow)
- agwa 5y ago
  Public keys are 32 bytes long, and private keys are 64 bytes long. The Ed25519 functions check to make sure the key is the correct length before using them: https://github.com/golang/go/blob/d4bfe006155ea0d9c2970e22fca5a027125c8a98/src/crypto/ed25519/ed25519.go#L204 https://github.com/golang/go/blob/d4bfe006155ea0d9c2970e22fc... The types would ideally use fixed-length arrays, but at the time the API was designed, Go didn't allow conversions between slices and array pointers, which would have made the API annoying to use: https://github.com/golang/go/issues/24376#issuecomment-372786394 https://github.com/golang/go/issues/24376#issuecomment-37278... Go is now adding support for such conversions, but the Ed25519 API can't be changed due to backwards compatibility guarantees.
timbray 5y ago
Fair enough, thanks, will adopt those. An interesting question is why heavy googling and stackoverflow diving looking for "how to publish ed25519" and related things failed to turn this up.
- nabla9 5y ago
  Call me old fashioned, but I browse and search the documentation first. Starting from the index page. "pkix - Package pkix contains shared, low level structures used for ASN.1 parsing and serialization of X.509 certificates, CRL and OCSP."
  - timbray 5y ago
    Right, what path would have led you to that doc page. I read a lot of doc pages and none of the paths I followed led to that one.
    - nabla9 5y ago
      Just one click away. Go to https://golang.org/ https://golang.org/ click [Packages] Scroll down until you see crypto packages. Now you should see package named pkix and description of the package: pkix - contains shared, low level structures used for ASN.1 parsing and serialization of X.509 certificates, CRL and OCSP.
      - timbray 5y ago
        Call me weird, but I tend to start by searching for what I think I'm looking for. I was looking for "serialize 25519" or something like that. None of "pkix", "ASN.1", "CRL", X.509, o "OCSP" were in my vocabulary when I started.
        nabla9 5y ago
        I see. 1. You needed to learn how the cryptographic serialization happens in practice. Including terminology. 2. You forgot to look again after you had learned all you needed to learn and wrote your own. It's easy to wander from learning to coding.
ComodoHacker 5y ago
>opaque byte strings The point was to have something you (an average Joe) can share on the Internet, and that's definitely not byte string. But the whole PKI thing is indeed very complicated. After such thorough analysis I was expecting some novel and simple solution like memorizable passphrase or QR code to scan.
evanelias 5y ago
As a non-crypto person who coincidentally just spent a couple days mucking with this in Go earlier this week, I absolutely agree with the author's assertion that it's very hard to find complete examples, and even articles that don't assume a decent baseline of domain expertise. fwiw my own thought/research path, after choosing Ed25519, looked something like this: 1. "How do I generate the key pair? I should probably use openssl since I trust that and have used it in the past." After googling, I see I can use `openssl genpkey -algorithm ed25519` for the private key. 2. "OK, that gave me a pem file. How do I deal with that in Go?" From here I found the same blainsmith.com article as OP, which is overall very helpful but unfortunately contains the unnecessary asn1.Unmarshal steps. 3. "Hmm, this looks overly complex." Luckily I took some extra time to look at other search results for "ed25519 pem golang" and found the crypto/x509 functions linked above. In retrospect, if I had just used ed25519.GenerateKey(nil) for step 1, I would have saved a ton of time. But this really wasn't initially clear to me at all. Exactly as the OP said, "Because as I stumbled through the undergrowth figuring this stuff out in the usual way, via Google and StackOverflow, I did not convince myself that the path I was following was the right one." And in this case the OP is a highly respected software engineer with his own wikipedia page, so at least I'm in good company for tripping all over this process :)
- chasil 5y ago
  Since Microsoft strapped OpenSSH into Windows 10, we can assume ssh-keygen is the most ubiquitous ed25519 management tool; anything using OpenSSH format will be widely supported. Does this run just about everywhere? ssh-keygen -t ed25519 -f testkey
  - evanelias 5y ago
    That likely runs everywhere, but in my mind it doesn't solve the core problems here: * The keys are each really just 32 byte sequences. So any tool that serializes them in a more complicated structure is just adding unnecessary complexity, if the calling code only cares about ed25519 keys anyway. * Lack of end-to-end examples in documentation and blog posts.
timbray 5y ago
Just committed changes to use those functions. Didn't remove all the ceremony but saved a few lines of code. Thanks. Credited @agwa in the commit.
FiloSottile 5y ago
Came here to say this. Please don't use PKIX, ASN.1, and Base64-of-DER-without-PEM-headers for Ed25519. Those are all extra complexity from protocols that were misguidedly built with runtime agility, or from a different time in cryptography engineering in which we really felt the need to put dynamic types on everything. [0][1] An Ed25519 public key is a 32 bytes sequence. An Ed25519 private key (called seed in crypto/ed25519 for unfortunate historical standardization reasons) is a 32 byte sequence. That's it. Here they are encoded in Base64. public key: 5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ= private key: shhKyGTvTeLjXGDCjEQgHRA7ps3LRNNfoO5S714kinU= age [2] similarly uses Curve25519 keys encoded with Bech32 to make them easier to copy-paste and read aloud. Look how nice they look compared to PKIX blobs! $ age-keygen # created: 2021-04-23T12:31:14-04:00 # public key: age1gek0nrawzg9fhkrzcmt4ql7au0n6hwflz7lqqc8wwcvefn2vssgsa4uulp AGE-SECRET-KEY-19JFPFAY2DF3HJP5DFGVSY4A4G4YSRHG4ZCJMKNC5MFD9C9ZN5LCSG8VTDL I'll think about how to make the crypto/ed25519 godoc point people in the right direction once Go 1.17 enters feature freeze. [0] https://www.imperialviolet.org/2016/05/16/agility.html https://www.imperialviolet.org/2016/05/16/agility.html [1] https://buttondown.email/cryptography-dispatches/archive/cryptography-dispatches-registries-considered/ https://buttondown.email/cryptography-dispatches/archive/cry... [2] https://github.com/FiloSottile/age https://github.com/FiloSottile/age
- timbray 5y ago
  This makes sense, but if I'm handing these things around probably some of the code [gasp] isn't in Go. So someone should write an RFC for the case where you don't need algorithm agility, and then devs don't need to know or care in the slightest what 25519 keys actually are, they just need to call APIs to serialize & deserialize them.
  - FiloSottile 5y ago
    I'd argue the RFC is already there and it's RFC 8032, which defines "32 bytes string" <-> "public key" and "32 bytes string" <-> "private key" APIs. Then you can use your preferred format to serialize a 32 bytes string if you need a text-safe encoding. (If you don't you're done!) For example, you can use RFC 4648. I am positive every language has Base64 code, and if there's an Ed25519 library out there that can't accept 32 byte strings we should talk to the author because it's broken as it doesn't implement RFC 8032. What else would a serialization RFC say, that is not already said in RFC 8032? Deciding that keys are encoded with Base64 and not with, say, hex seems like a weird thing to force on people.
rauhl 5y ago
The trouble with using just a bag of (potentially Base64-encoded) bytes is that it flirts with cryptographic doom: one relies on users not to make mistakes. For example, is 5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ= a Base64-encoded Ed25519 public key? Sure. Is it a Base64-encoded HMAC-SHA-256 key? Sure! It’s 256 bits, after all. Is it a valid AES-256 key? Yes again. OTOH, (public-key (ed25519 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly not an HMAC private key. For that matter, (secret-key (hmac-sha-256 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly not an Ed25519 public key. And (secret-key (aes-256 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly neither a public Ed25519 key nor a private HMAC-SHA-256 key. Software expecting one and getting one of the other will break. Why does this matter? Because using a public value such as a public key as an HMAC signing key means that anyone can generate a signature, and using a public value such as a public key as a secret encryption key means that anyone can decrypt the encrypted material. Why use the unusual formats I suggested above? Because this: (public-key (ed25519 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly preferable to this: -----BEGIN PUBLIC KEY----- MDYwBQYDK2VwAy0ANXVXN2FuRUdGMW5JakdmcDVwUzJraU4wY24ybUdZa3VTYStUQ0JvRkliUT0 -----END PUBLIC KEY-----
- timbray 5y ago
  The conventional PKIX/PEM is logically equivalent to your s-expression there; I assume you're arguing for using the conventional packing rather than proposing a new s-expression based encoding?
  - rauhl 5y ago
    For your purposes, I think that your choice of PEM is probably the best. As you note, it is logically equivalent to the s-expression (it is typed, not a bag of bytes), and it has the advantage of being a well-known format. If æsthetic concerns are a higher priority than interoperability (say, for a personal hobby project), then I do recommend folks adopt the format I gave an example of. It has some history behind it, dating back to the 1997–1999 timeframe and the IETF’s SPKI Working Group (e.g. https://theworld.com/~cme/examples.txt https://theworld.com/~cme/examples.txt) which resulted in RFCs 2692 & 2693, as well as some great drafts (https://people.csail.mit.edu/rivest/Sexp.txt https://people.csail.mit.edu/rivest/Sexp.txt & https://theworld.com/~cme/spki.txt https://theworld.com/~cme/spki.txt) which never progressed because Reasons™. I still think that the SPKI project got PKI right. Oh well, what might have been!
1vuio0pswjnm7 5y ago
Wouldnt it make more sense to use base32 instead of base64. Also whats wrong with simply using the hex string of the public key. It is not much longer than the base64/32 string. I thought the short length was an intentional part of djb's design. Small public keys, short hex strings. Easy to share.

formerly_proven 5y ago

Why exactly would you do any of this instead of a) just having a field of 32 bytes in the wire protocol (if binary) b) just turning it into a hex string (if text is needed) or base64/85 if you don't want hex for some reason

viraptor 5y ago
> Why exactly would you do any of this Because it's your first adventure in the land of crypto and you have no idea how to do things, which parts you need to learn about and which are not required. It seems trivial for me now, but there was a time I would end up on a similar journey. I don't know of a source which would do a quick walk through all of these levels. And if there is one, I don't know how you'd find it without knowing the right keywords. (Now I'm tempted to write up some "crypto and certificates and public keys and other things explained quickly and without depth so you can ask better questions next" post)
- formerly_proven 5y ago
  Fair enough, if you don't know "the customs" it's probably very easy to end up looking for "public key formats" and fall into the PKIX/X.509/ASN.1 rabbit hole.
  - cordite 5y ago
    It’s actually quite an exciting rabbit hole to explore with not many useful Google search results to educate on. Opinions are my own :)
  - alfiedotwtf 5y ago
    Rabbit hole is a polite way to put it. Antiquated pile of shit is getting closer to the truth
- Aachen 5y ago
  Not the person you replied to, but > Because it's your first adventure in the land of crypto and you have no idea how to do things is rather presumptuous. I do actually know how to do crypto things (I review and test such things for a living) and after reading their comment and skimming the post I'm left wondering the exact same thing.

latortuga 5y ago

There's a typo in the HN title, it should be 25519, not 25119.

timbray 5y ago
Fixed, thanks
dang 5y ago
Fixed, thanks :)

kfreds 5y ago

First of all, I enjoyed reading this article. Second, it inspired me to write the following, with curious beginners as the target audience. It was meant to be about the importance of reducing the computing power of recognizers / parsers / interfaces / meaning-makers, as the author brings up PEM, ASN.1, and X.509. It grew a bit. Enjoy. <rant> In my opinion security exists to facilitate trust, and security has a lot to do with managing complexity. One of the most powerful security technologies we have is math. Especially math that is easy to do in one direction and very difficult to do in the opposite direction. RSA, Ed25519, Diffie-Hellman, and SHA256 all rely on this concept. Another power tool is combining these cryptographic primitives to build protocols that provide e.g. authentication, forward secrecy, confidentiality, and integrity-protection. When using cryptographic primitives directly it is easy to make mistakes. For example, before Ed25519 there were other similar signature schemes that, in addition to a message to be signed, required a number that must not be repeated for different messages, called a nonce. If this number was repeated, an attacker could collect signatures and combine them to get the private key. Ed25519 removes this sharp corner by essentially using nonce=SHA512(message) internally. When designing a protocol like TLS, SSH, Signal, or WireGuard, multiple primitives of different types are combined to get the desired security properties. Cryptographers work hard to combine primitives such that the resulting construction is easy to reason about, and ideally prove that the construction will only break if one of the primitives break. Unfortunately this key insight is often ignored further up the stack. Many protocols, file formats, etc are unnecessarily complex. This makes them harder to reason about, which makes it harder to build a safe recognizer / parser / interface / meaning-maker for them. Complex parsers lead to incorrect mental models, which lead to security vulnerabilities. If you are ever tasked with building a protocol, file format, or API, consider making it only as complex as it must be. We will suffer for a long time with the complex beasts that have already been standardized. Let’s not make more of them. Remember that security, trustworthiness, understanding, and managing complexity are intertwined. Don’t use more complexity than necessary. </rant> P.S. If anyone is curious about Ed25519 I can highly recommend the following article: https://blog.mozilla.org/warner/2011/11/29/ed25519-keys/ https://blog.mozilla.org/warner/2011/11/29/ed25519-keys/

ajrw 5y ago

JWK is a good option for representing public keys, for an Ed25519 key you would have something like this: { "kty":"OKP", "crv":"Ed25519", "x":"oF0a6lgwrJplzfs4RmDUl-NpfEa0Gc8s7IXei9JFRZ0" }

nemo1618 5y ago

I've always found PEM/ASN.1 to be rather ugly. I encode my ed25519 keys as "ed25519:<hex key>".

deleted 5y ago
[deleted]

upofadown 5y ago

>...if ed25519 fails, it’ll fail slowly and there’ll be time to migrate to something else. Why would you assume that? The curve in question has been around for only something like 10 years....

wahern 5y ago

Ed25519 isn't quite as simple as people make it out to be, at least in the sense that it's an answer to a problem that's been entirely elided in these discussions, whilst the problems with older alternatives are more familiar. Both RSA and NIST-defined elliptic curves support both signatures and key exchange using the same public key. Not so with Curve25519. Ed25519 is just for signatures and X25519 for key exchange, and the "opaque" 32-byte public keys are actually different. You can convert Ed25519 public keys to X25519 public keys, but for a reverse transformation there are two incompatible methods, and so any protocol where you would like to keep a single public key association for parties--particularly offline protocols where a sender can't ask the recipient to generate an ephemeral key pair for confidential message reception--one is faced with a couple of choices. And the choices made by some existing protocols aren't always obvious. Indeed, the very fact that these differences exist isn't necessarily obvious until you're already knee-deep in the weeds. At this point the solution is more-or-less settled--just publish Ed25519 keys. But it wasn't always that way considering that the Ed25519 signature scheme is much newer than Curve25519. And of course the legacy of this complexity will always be around to some extent. Also left undiscussed is the different modes of Ed25519--regular and prehashed message digestion, a choice that is absent for older schemes. That's a whole 'nother can of worms. Time and usage effect almost every new protocol and format--they don't stay pristine forever. The sheen has already dulled on Curve25519. If you want the fewest possible interoperability headaches Curve448 is the new hotness, if only because Curve448 has been standardized more times than actually deployed. It's also more secure--longer keys[1] and slightly better mathematical properties, AFAIU, of the type that originally made Curve25519 and DJB's SafeCurves initiative appealing. [1] Arguably useful if there's an extended period of time between quantum computing scaling to threaten 128-bit classically equivalent but non-quantum-safe asymmetric schemes, and scaling to threaten ~256-bit equivalent schemes. You're probably better off with RSA in that case, though. Or just not worrying about it and sticking to Curve25519--one of the few times where "but 32 is such a nice, round, convenient string length" is perhaps a legitimate argument.

tialaramex 5y ago
The trouble is that if you use my public signing key to do something else it wasn't intended for (here, X25519) I can't participate without also using my private key in this unconventional way. Now, this looks safe which is more than can be said for the equivalent trick in RSA, which looks obviously dangerous and in practice has caused real problems, but it sure would be comforting if people did all the heavy lifting to prove it is safe rather than shrugging and doing it anyway (e.g. 'age') Meanwhile it feels like the lesson we learned aeons ago in cryptography should apply, if we can't prove why it's safe, let's just assume it isn't safe.