4 ms·

DER encoded ASN.1 is used for the X.509 end-entity certificate, however generally this follows how CertificateVerify works in TLS 1.3 https://tools.ietf.org/htm

DER encoded ASN.1 is used for the X.509 end-entity certificate, however generally this follows how CertificateVerify works in TLS 1.3 https://tools.ietf.org/html/rfc8446#section-4.4.3 https://tools.ietf.org/html/rfc8446#section-4.4.3

est31 7y ago
Hmm fair enough. Case rested then.
- sudoyear123 7y ago
  generalized asn1 parsing can be super tricky and the industry is generally avoiding asn1 for new protocols. There has been some really interesting work from microsoft on verified parsers for asn1 https://www.usenix.org/system/files/sec19-ramananandro_0.pdf https://www.usenix.org/system/files/sec19-ramananandro_0.pdf
  - est31 7y ago
    Interesting, didn't know that. That trend is a bit sad then as I've been quite enjoying the interactive ASN.1 viewer when developing rcgen, an asn.1 library: https://lapo.it/asn1js/ https://lapo.it/asn1js/
  - wahern 7y ago
    You're supposed use parser generators for ASN.1. The reason ASN.1 is so difficult in TLS and other crypto standards is precisely because ASN.1 messaging is intermixed with ad hoc messaging (as in this case) and implicit state, which means you couldn't even use a parser generator for everything even if you wanted to. The excellent open source ASN.1 compiler, asn1c (http://lionet.info/asn1c/compiler.html http://lionet.info/asn1c/compiler.html), can generate C data structures and a parser and composer for X.509 DER certificates from the formal ASN.1 description. But it's not widely used because, among other reasons, you end up having to write too much ad hoc parsing anyhow, which makes the investment in the parser generator seem not worthwhile. (AFAIU, asn1c is far more popular in the telecom industry, likely because telecom uses fully ASN.1-based messaging.) Of course, if you're not going to use ASN.1 as intended, then the binary encoding (e.g. DER) can be quite tricky to parse using an ad hoc parser, including most parser combinators, mostly because TLV encodings aren't context-free. But I managed to write a full X.509 parser using LPeg. LPeg has an extension for match-time captures which provide a way to invoke a subexpression parameterized on the value of a previous match (e.g. the decoded length context), which can return match success or failure along with a resumption point to the parent expression. See http://lua-users.org/lists/lua-l/2019-04/msg00226.html http://lua-users.org/lists/lua-l/2019-04/msg00226.html and http://www.inf.puc-rio.br/~roberto/lpeg/#matchtime http://www.inf.puc-rio.br/~roberto/lpeg/#matchtime I feel like there's simply no good answer here. The fundamental problem is the tension among 1) strictly specified, formalized protocols (which ASN.1 DER absolutely provides), 2) efficiency in time and space (ASN.1 DER does well, PER takes it to an extreme), 3) the need for forward compatibility so protocols can incrementally evolve (partly technical, partly a social management issue), and of course 4) ease of implementation. Context-free encodings help with #3 and #4, but fail at #2 (e.g. field names aren't necessary, and variable length values require a more complex encoding), and in a security context cause problems with #1 (better to have a failed parse than to successfully parse unknown elements that you ignore).

est31 7y ago

Hmm fair enough. Case rested then.

sudoyear123 7y ago
generalized asn1 parsing can be super tricky and the industry is generally avoiding asn1 for new protocols. There has been some really interesting work from microsoft on verified parsers for asn1 https://www.usenix.org/system/files/sec19-ramananandro_0.pdf https://www.usenix.org/system/files/sec19-ramananandro_0.pdf
- est31 7y ago
  Interesting, didn't know that. That trend is a bit sad then as I've been quite enjoying the interactive ASN.1 viewer when developing rcgen, an asn.1 library: https://lapo.it/asn1js/ https://lapo.it/asn1js/
- wahern 7y ago
  You're supposed use parser generators for ASN.1. The reason ASN.1 is so difficult in TLS and other crypto standards is precisely because ASN.1 messaging is intermixed with ad hoc messaging (as in this case) and implicit state, which means you couldn't even use a parser generator for everything even if you wanted to. The excellent open source ASN.1 compiler, asn1c (http://lionet.info/asn1c/compiler.html http://lionet.info/asn1c/compiler.html), can generate C data structures and a parser and composer for X.509 DER certificates from the formal ASN.1 description. But it's not widely used because, among other reasons, you end up having to write too much ad hoc parsing anyhow, which makes the investment in the parser generator seem not worthwhile. (AFAIU, asn1c is far more popular in the telecom industry, likely because telecom uses fully ASN.1-based messaging.) Of course, if you're not going to use ASN.1 as intended, then the binary encoding (e.g. DER) can be quite tricky to parse using an ad hoc parser, including most parser combinators, mostly because TLV encodings aren't context-free. But I managed to write a full X.509 parser using LPeg. LPeg has an extension for match-time captures which provide a way to invoke a subexpression parameterized on the value of a previous match (e.g. the decoded length context), which can return match success or failure along with a resumption point to the parent expression. See http://lua-users.org/lists/lua-l/2019-04/msg00226.html http://lua-users.org/lists/lua-l/2019-04/msg00226.html and http://www.inf.puc-rio.br/~roberto/lpeg/#matchtime http://www.inf.puc-rio.br/~roberto/lpeg/#matchtime I feel like there's simply no good answer here. The fundamental problem is the tension among 1) strictly specified, formalized protocols (which ASN.1 DER absolutely provides), 2) efficiency in time and space (ASN.1 DER does well, PER takes it to an extreme), 3) the need for forward compatibility so protocols can incrementally evolve (partly technical, partly a social management issue), and of course 4) ease of implementation. Context-free encodings help with #3 and #4, but fail at #2 (e.g. field names aren't necessary, and variable length values require a more complex encoding), and in a security context cause problems with #1 (better to have a failed parse than to successfully parse unknown elements that you ignore).