8 ms·

TLS certificates have at least two internal representations of time

97 points by throw0101a 5y ago

sneak 5y ago
This is both surprising and unsurprising at the same time. I wonder how long until we collectively decide to banish things like ASN.1 to the past like we are doing with C strings and parsing of untrusted network input in memory unsafe languages. ASN.1: unsafe at any speed. See also: https://en.m.wikipedia.org/wiki/X.690 https://en.m.wikipedia.org/wiki/X.690 Everything I read about x509-related serialization reminds me of that rant[1] about the PSD file format. [1]: https://github.com/gco/xee/blob/master/XeePhotoshopLoader.m#L108 https://github.com/gco/xee/blob/master/XeePhotoshopLoader.m#...
- throwawaaarrgh 5y ago
  > and parsing of untrusted network input in memory unsafe languages Are these modern languages? I'm not aware of any modern ones that have a taint mode. Just Rust that has memory checking, Perl that has a simple taint mode, Ruby that has a more robust taint mode, and Jeeves which is like a general taint mode in a DSL in Python.
- dfox 5y ago
  The issue with parsing ASN.1 is not inherent in the specification itself. The issue is about trying to produce DER/BER parser that is hardwired to parse particular instance of the syntax and directly fills some kind of C structs without the intermediate step of producing some kind of (DOM-like) tree of ASN.1 abstract objects. Something like this is usually done for two reasons: efficiency and the idea that such parser can be simpler than full BER parser (as it can ignore constructs that "cannot happen" in the particular syntax). Somewhat predictably, most ASN.1 software that uses this mental model of "ASN.1 is too complex, this is the easiest and fastest way to parse it" has something to do with cryptography, which makes the resulting issues even more critical. Valid criticism of ASN.1 would probably be that the abstract object model is too general on one hand and contains useless historical data types on the other (and the article is particular example of that). But that mostly is not related in any way to complexity and safety of the parsing or the currently popular meme of "unsafe" == bad.
  - mordae 5y ago
    Since ASN.1 is not a proper TLV format (because you can change tag by using [tag] construct), you cannot fully decode it without up-to-date schema. It's much saner to parse it using a validating parser combinator than to go through an AST.
    - pas 5y ago
      ... could you explain this with a bit more detail, please? Asking for those very ubiquitous ASN.1 illiterate friends :)
      - rkeene2 5y ago
        I can go over it -- I should note that I like ASN.1 (as described by X.690). As part of a project where I needed to parse X.509 certificates I wrote a simple ASN.1 -> X.509 parser which does the mapping in place. This can be found at [0]. So while it's possible to decode the structure of an ASN.1 object and many nested things, getting at a meaning requires understanding the schema of the objects being parsed since tags only describe the data type in a limited way. For example, to get the public key from an X.509 certificate you can easily do that -- but it will be an opaque ASN.1 structure unless you know if it's an RSA key or an ECDSA key, which is encoded as data in another ASN.1 OID somewhere, not as PART of the ASN.1 tag. So if you wanted to get the exponent for an X.509 certificate (like x509_to_exponent() does), then you need to know the schema for that type of public key -- here, it's specified in PKCS#1 (because only RSA is supported by this software), a completely unrelated standard. [0] https://cackey.rkeene.org/fossil/artifact/bc9c145e70285c54?ln https://cackey.rkeene.org/fossil/artifact/bc9c145e70285c54?l...
        pas 5y ago
        That makes sense. After all if you have an arbitrary JSON (or XML) you don't know what is that, unless you have a schema, then you can check conformance, etc. And probably given a schema it's easy to give the user (developer) bound methods that reference some value in that "structured blob", and of course it's possible to reuse that method with other blobs. (While having some error handling of course.) But how come people have aversions with ASN1? Is it just stylistic as with XML? (And that it's used as the substrate for a bunch of old harebrained standards and protocols - all with artisanal shitghetti C codecs instead of the aforementioned robust parser + schema?)
- est31 5y ago
  > I wonder how long until we collectively decide to banish things like ASN.1 to the past Indeed, in the security community there is a trend to get rid of ASN.1. E.g. TLS uses a custom ad-hoc format: https://datatracker.ietf.org/doc/html/rfc5246#section-4 https://datatracker.ietf.org/doc/html/rfc5246#section-4 And this is used by e.g. certificate transparency: https://datatracker.ietf.org/doc/html/rfc6962#section-1.2 https://datatracker.ietf.org/doc/html/rfc6962#section-1.2 Maybe it's just a question of tooling, but personally I haven't found a binary format that is as nicely debugable as ASN.1. In many other instances I have to stare at hexdumps while for ASN.1, the lapo.it viewer exists. E.g. I still haven't been able to parse the extra_data of a certificate transparency log entry. In the case of ASN.1, you can dump it to lapo.it. How can you debug data in the ad hoc TLS format? There is literally no information in the binary format, you must know the schema. That being said, it's indeed more difficult to write parsers for ASN.1 formatted data without using ASN.1 libraries. The ad hoc TLS format definitely has an advantage here. But IMO ASN.1 has way too much of a bad name.
  - tankenmate 5y ago
    Have you looked at XDR?
Arnavion 5y ago
Something the author didn't note is that RFC 5280 requires that times before 2050 are encoded as UTCTime and not GeneralizedTime. That also contributes to software being written to only parse UTCTime and not GeneralizedTime, because most extant certs as of today expire before 2050. The longest-lived CA cert in my Linux distro expires in 2046.
- tialaramex 5y ago
  Also, for the root trust store it's reasonable to just ignore the notBefore/ notAfter values so you won't need to parse 2050 dates until other certificates need them. If you don't trust this root any more, remove it from your trust store, and if you don't trust it yet, it won't be in the store. This is a contrast to intermediate and leaf certificates which you are shown by other parties and so will need that notBefore/ notAfter information to make trust decisions. This is why Let's Encrypt expects to still work for people on old Android phones that haven't seen a software update in many years, even after the root via which those phones trust Let's Encrypt "expires" later in 2021. Android doesn't check the expiry date on roots and these phones aren't getting updates so they won't be updated to stop trusting the obsolete root.
throwawaaarrgh 5y ago
"It's true that RFC3280 says that the GeneralizedTime format must be used only for dates beginning in 2050. This is to allow compatibility with existing implementations (like NSS) that do not yet support the format. However, there is no reason for an implementation that supports GeneralizedTime to reject a certificate (or other object) that contains dates before 2050 that are encoded using that format. Dates generated by NSS should use the RFC guideline to ensure interoperability with other RFC3280-conformant implementations." - https://bugzilla.mozilla.org/show_bug.cgi?id=143334 https://bugzilla.mozilla.org/show_bug.cgi?id=143334 (2002) "In the past, Firefox was lenient, NSS still is lenient, but mozilla::pkix is strict. I found a comment in the mozilla::pkix which says the alternative encodings from asn.1 explicitly aren't allowed." - https://bugzilla.mozilla.org/show_bug.cgi?id=1152515 https://bugzilla.mozilla.org/show_bug.cgi?id=1152515 (2015) An interesting comment in there also mentions how QA testing just one function with different dates might require changing the system date, but that would test more than just the one function... I wonder how many pieces of software might miss bugs due to complex tests like this.
kstenerud 5y ago
I got so sick of the broken time formats (ALL of them are broken or deficient, including ISO-8601) that I eventually made my own: one in binary [1], and one in text [2]. Time is one of the most difficult things to get right in software engineering. [1] https://github.com/kstenerud/compact-time/blob/master/compact-time-specification.md#compact-time-format https://github.com/kstenerud/compact-time/blob/master/compac... [2] https://github.com/kstenerud/concise-encoding/blob/master/cte-specification.md#temporal-types https://github.com/kstenerud/concise-encoding/blob/master/ct... and https://github.com/kstenerud/concise-encoding/blob/master/ce-structure.md#temporal-types https://github.com/kstenerud/concise-encoding/blob/master/ce...
- red0point 5y ago
  Relevant xkcd: https://xkcd.com/927/ https://xkcd.com/927/
  - kstenerud 5y ago
    Yes, it's unfortunate that people have latched onto that xkcd comic as their preferred way to dismiss new things out of hand without expending any energy on critical thought. We should start calling them the "927 club".
    - red0point 5y ago
      Instead of feeling personally attacked and resorting to pettiness, maybe you could instead start by explaining why existing standards are bad and why your proposed standard is superior. The xkcd pretty neatly summarized the situation as of now. You developed and self-promote a new standard but do not explain why anyone should care.
      - JauntyHatAngle 5y ago
        I think it's less about the xkcd (I had similar thoughts but hadn't looked at the linked standard so don't know if it's justified), and more that simply linking the xkcd is not really enough discussion to justify it as a neat summary. Linked in context with other response points and it's a worthy reference, but dropped without much further discussion and the onus feels like it remains on you to justify it.
cbsmith 5y ago
I'm just confused about how this kind of stuff becomes a blog post. I mean, it's all right their in the spec & encoding rules.
- cbsmith 5y ago
  s/their/there/
teddyh 5y ago
If you like this, you will probably be interested in this: Everything you Never Wanted to Know about PKI but were Forced to Find Out, by Peter Gutmann: https://www.cs.auckland.ac.nz/~pgut001/pubs/pkitutorial.pdf https://www.cs.auckland.ac.nz/~pgut001/pubs/pkitutorial.pdf Also, this, by the same person: X.509 Style Guide (which talks about the two date formats): “In coming up with the worlds least efficient machine-readable time encoding format, the ISO nevertheless decided to forgo the encoding of centuries, a problem which has been kludged around by redefining the time as UTCTime if the date is 2049 or ealier, and GeneralizedTime if the date is 2050 or later (the original plan was to cut over in 2015, but it was felt that moving it back to 2050 would ensure that the designers were either retired or dead by the time the issue became a serious problem, leaving someone else to take the blame). To decode a date, if it's UTCTime and the year is less than or equal to 49 it's 20xx, if it's UTCTime and the year is equal to or greater than 50 it's 19xx, and if it's GeneralizedTime it's encoded properly (but shouldn't really be used for dates before 2050 because you could run into interoperability problems with existing software). Yuck. To make this even more fun, another spec at one time gave the cutover date as 2050/2051 rather than 2049/2050, and allowed GeneralizedTime to be used before 2050 if you felt you could get away with it. It's likely that a lot of conforming systems will briefly become nonconforming systems in about half a centuries time, […]” — https://www.cs.auckland.ac.nz/~pgut001/pubs/x509guide.txt https://www.cs.auckland.ac.nz/~pgut001/pubs/x509guide.txt
- Joker_vD 5y ago
  Pretty much everything about X.500 is breathtakingly terrifying. How did people even manage to come up with it?
  - teddyh 5y ago
    IIUC, all the old ISO/ITU telecommunication standards were like that. This is why people fled in droves to instead use and implement TCP/IP from the RFCs.
    - Joker_vD 5y ago
      Oh yeah, Gutmann actually explains this history in one of his writings, and then the certificates kinda start making sense. Also, remember the OSI model? It was not supposed to be just a model — and it's a failure of the model anyway, real network protocols just keep piling up application layers on top of each other, the upper ones using the lower ones as their transport layers, — no, there were supposed to be actual separate protocols for each level, and actually there are! Of course, nobody sane ever uses them.
qwertox 5y ago
Is there a public example certificate with an expiry date above 2050 which I could fetch in order to test a script? I would use conn = ssl.create_default_context().wrap_socket(socket.socket(socket.AF_INET), server_hostname=hostname) conn.connect((hostname, 443)) cert = conn.getpeercert() print(cert['notAfter']) to test it.
- svenfaw 5y ago
  This one expires in 2066, for instance: https://censys.io/certificates/3b639a3248914d0d5e3811c080f3136976109073e5d9c04013c9dfa012b2155d/pem https://censys.io/certificates/3b639a3248914d0d5e3811c080f31...
mleonhard 5y ago
ASN.1 doesn't meet the needs of modern society. We should replace it with something simple like JSON and concrete times as 63-bit epoch timestamps. Now is the time to start making this transition, 17 years before the 31-bit epoch timestamp rollover.

97 points by throw0101a 5y ago

This is both surprising and unsurprising at the same time. I wonder how long until we collectively decide to banish things like ASN.1 to the past like we are doing with C strings and parsing of untrusted network input in memory unsafe languages. ASN.1: unsafe at any speed. See also: https://en.m.wikipedia.org/wiki/X.690 https://en.m.wikipedia.org/wiki/X.690 Everything I read about x509-related serialization reminds me of that rant[1] about the PSD file format. [1]: https://github.com/gco/xee/blob/master/XeePhotoshopLoader.m#L108 https://github.com/gco/xee/blob/master/XeePhotoshopLoader.m#...

throwawaaarrgh 5y ago
> and parsing of untrusted network input in memory unsafe languages Are these modern languages? I'm not aware of any modern ones that have a taint mode. Just Rust that has memory checking, Perl that has a simple taint mode, Ruby that has a more robust taint mode, and Jeeves which is like a general taint mode in a DSL in Python.
dfox 5y ago
The issue with parsing ASN.1 is not inherent in the specification itself. The issue is about trying to produce DER/BER parser that is hardwired to parse particular instance of the syntax and directly fills some kind of C structs without the intermediate step of producing some kind of (DOM-like) tree of ASN.1 abstract objects. Something like this is usually done for two reasons: efficiency and the idea that such parser can be simpler than full BER parser (as it can ignore constructs that "cannot happen" in the particular syntax). Somewhat predictably, most ASN.1 software that uses this mental model of "ASN.1 is too complex, this is the easiest and fastest way to parse it" has something to do with cryptography, which makes the resulting issues even more critical. Valid criticism of ASN.1 would probably be that the abstract object model is too general on one hand and contains useless historical data types on the other (and the article is particular example of that). But that mostly is not related in any way to complexity and safety of the parsing or the currently popular meme of "unsafe" == bad.
- mordae 5y ago
  Since ASN.1 is not a proper TLV format (because you can change tag by using [tag] construct), you cannot fully decode it without up-to-date schema. It's much saner to parse it using a validating parser combinator than to go through an AST.
  - pas 5y ago
    ... could you explain this with a bit more detail, please? Asking for those very ubiquitous ASN.1 illiterate friends :)
    - rkeene2 5y ago
      I can go over it -- I should note that I like ASN.1 (as described by X.690). As part of a project where I needed to parse X.509 certificates I wrote a simple ASN.1 -> X.509 parser which does the mapping in place. This can be found at [0]. So while it's possible to decode the structure of an ASN.1 object and many nested things, getting at a meaning requires understanding the schema of the objects being parsed since tags only describe the data type in a limited way. For example, to get the public key from an X.509 certificate you can easily do that -- but it will be an opaque ASN.1 structure unless you know if it's an RSA key or an ECDSA key, which is encoded as data in another ASN.1 OID somewhere, not as PART of the ASN.1 tag. So if you wanted to get the exponent for an X.509 certificate (like x509_to_exponent() does), then you need to know the schema for that type of public key -- here, it's specified in PKCS#1 (because only RSA is supported by this software), a completely unrelated standard. [0] https://cackey.rkeene.org/fossil/artifact/bc9c145e70285c54?ln https://cackey.rkeene.org/fossil/artifact/bc9c145e70285c54?l...
      - pas 5y ago
        That makes sense. After all if you have an arbitrary JSON (or XML) you don't know what is that, unless you have a schema, then you can check conformance, etc. And probably given a schema it's easy to give the user (developer) bound methods that reference some value in that "structured blob", and of course it's possible to reuse that method with other blobs. (While having some error handling of course.) But how come people have aversions with ASN1? Is it just stylistic as with XML? (And that it's used as the substrate for a bunch of old harebrained standards and protocols - all with artisanal shitghetti C codecs instead of the aforementioned robust parser + schema?)
est31 5y ago
> I wonder how long until we collectively decide to banish things like ASN.1 to the past Indeed, in the security community there is a trend to get rid of ASN.1. E.g. TLS uses a custom ad-hoc format: https://datatracker.ietf.org/doc/html/rfc5246#section-4 https://datatracker.ietf.org/doc/html/rfc5246#section-4 And this is used by e.g. certificate transparency: https://datatracker.ietf.org/doc/html/rfc6962#section-1.2 https://datatracker.ietf.org/doc/html/rfc6962#section-1.2 Maybe it's just a question of tooling, but personally I haven't found a binary format that is as nicely debugable as ASN.1. In many other instances I have to stare at hexdumps while for ASN.1, the lapo.it viewer exists. E.g. I still haven't been able to parse the extra_data of a certificate transparency log entry. In the case of ASN.1, you can dump it to lapo.it. How can you debug data in the ad hoc TLS format? There is literally no information in the binary format, you must know the schema. That being said, it's indeed more difficult to write parsers for ASN.1 formatted data without using ASN.1 libraries. The ad hoc TLS format definitely has an advantage here. But IMO ASN.1 has way too much of a bad name.
- tankenmate 5y ago
  Have you looked at XDR?

Arnavion 5y ago

Something the author didn't note is that RFC 5280 requires that times before 2050 are encoded as UTCTime and not GeneralizedTime. That also contributes to software being written to only parse UTCTime and not GeneralizedTime, because most extant certs as of today expire before 2050. The longest-lived CA cert in my Linux distro expires in 2046.

tialaramex 5y ago
Also, for the root trust store it's reasonable to just ignore the notBefore/ notAfter values so you won't need to parse 2050 dates until other certificates need them. If you don't trust this root any more, remove it from your trust store, and if you don't trust it yet, it won't be in the store. This is a contrast to intermediate and leaf certificates which you are shown by other parties and so will need that notBefore/ notAfter information to make trust decisions. This is why Let's Encrypt expects to still work for people on old Android phones that haven't seen a software update in many years, even after the root via which those phones trust Let's Encrypt "expires" later in 2021. Android doesn't check the expiry date on roots and these phones aren't getting updates so they won't be updated to stop trusting the obsolete root.

throwawaaarrgh 5y ago

"It's true that RFC3280 says that the GeneralizedTime format must be used only for dates beginning in 2050. This is to allow compatibility with existing implementations (like NSS) that do not yet support the format. However, there is no reason for an implementation that supports GeneralizedTime to reject a certificate (or other object) that contains dates before 2050 that are encoded using that format. Dates generated by NSS should use the RFC guideline to ensure interoperability with other RFC3280-conformant implementations." - https://bugzilla.mozilla.org/show_bug.cgi?id=143334 https://bugzilla.mozilla.org/show_bug.cgi?id=143334 (2002) "In the past, Firefox was lenient, NSS still is lenient, but mozilla::pkix is strict. I found a comment in the mozilla::pkix which says the alternative encodings from asn.1 explicitly aren't allowed." - https://bugzilla.mozilla.org/show_bug.cgi?id=1152515 https://bugzilla.mozilla.org/show_bug.cgi?id=1152515 (2015) An interesting comment in there also mentions how QA testing just one function with different dates might require changing the system date, but that would test more than just the one function... I wonder how many pieces of software might miss bugs due to complex tests like this.

kstenerud 5y ago

I got so sick of the broken time formats (ALL of them are broken or deficient, including ISO-8601) that I eventually made my own: one in binary [1], and one in text [2]. Time is one of the most difficult things to get right in software engineering. [1] https://github.com/kstenerud/compact-time/blob/master/compact-time-specification.md#compact-time-format https://github.com/kstenerud/compact-time/blob/master/compac... [2] https://github.com/kstenerud/concise-encoding/blob/master/cte-specification.md#temporal-types https://github.com/kstenerud/concise-encoding/blob/master/ct... and https://github.com/kstenerud/concise-encoding/blob/master/ce-structure.md#temporal-types https://github.com/kstenerud/concise-encoding/blob/master/ce...

red0point 5y ago
Relevant xkcd: https://xkcd.com/927/ https://xkcd.com/927/
- kstenerud 5y ago
  Yes, it's unfortunate that people have latched onto that xkcd comic as their preferred way to dismiss new things out of hand without expending any energy on critical thought. We should start calling them the "927 club".
  - red0point 5y ago
    Instead of feeling personally attacked and resorting to pettiness, maybe you could instead start by explaining why existing standards are bad and why your proposed standard is superior. The xkcd pretty neatly summarized the situation as of now. You developed and self-promote a new standard but do not explain why anyone should care.
    - JauntyHatAngle 5y ago
      I think it's less about the xkcd (I had similar thoughts but hadn't looked at the linked standard so don't know if it's justified), and more that simply linking the xkcd is not really enough discussion to justify it as a neat summary. Linked in context with other response points and it's a worthy reference, but dropped without much further discussion and the onus feels like it remains on you to justify it.

cbsmith 5y ago

I'm just confused about how this kind of stuff becomes a blog post. I mean, it's all right their in the spec & encoding rules.

cbsmith 5y ago
s/their/there/

teddyh 5y ago

If you like this, you will probably be interested in this: Everything you Never Wanted to Know about PKI but were Forced to Find Out, by Peter Gutmann: https://www.cs.auckland.ac.nz/~pgut001/pubs/pkitutorial.pdf https://www.cs.auckland.ac.nz/~pgut001/pubs/pkitutorial.pdf Also, this, by the same person: X.509 Style Guide (which talks about the two date formats): “In coming up with the worlds least efficient machine-readable time encoding format, the ISO nevertheless decided to forgo the encoding of centuries, a problem which has been kludged around by redefining the time as UTCTime if the date is 2049 or ealier, and GeneralizedTime if the date is 2050 or later (the original plan was to cut over in 2015, but it was felt that moving it back to 2050 would ensure that the designers were either retired or dead by the time the issue became a serious problem, leaving someone else to take the blame). To decode a date, if it's UTCTime and the year is less than or equal to 49 it's 20xx, if it's UTCTime and the year is equal to or greater than 50 it's 19xx, and if it's GeneralizedTime it's encoded properly (but shouldn't really be used for dates before 2050 because you could run into interoperability problems with existing software). Yuck. To make this even more fun, another spec at one time gave the cutover date as 2050/2051 rather than 2049/2050, and allowed GeneralizedTime to be used before 2050 if you felt you could get away with it. It's likely that a lot of conforming systems will briefly become nonconforming systems in about half a centuries time, […]” — https://www.cs.auckland.ac.nz/~pgut001/pubs/x509guide.txt https://www.cs.auckland.ac.nz/~pgut001/pubs/x509guide.txt

Joker_vD 5y ago
Pretty much everything about X.500 is breathtakingly terrifying. How did people even manage to come up with it?
- teddyh 5y ago
  IIUC, all the old ISO/ITU telecommunication standards were like that. This is why people fled in droves to instead use and implement TCP/IP from the RFCs.
  - Joker_vD 5y ago
    Oh yeah, Gutmann actually explains this history in one of his writings, and then the certificates kinda start making sense. Also, remember the OSI model? It was not supposed to be just a model — and it's a failure of the model anyway, real network protocols just keep piling up application layers on top of each other, the upper ones using the lower ones as their transport layers, — no, there were supposed to be actual separate protocols for each level, and actually there are! Of course, nobody sane ever uses them.

qwertox 5y ago

Is there a public example certificate with an expiry date above 2050 which I could fetch in order to test a script? I would use conn = ssl.create_default_context().wrap_socket(socket.socket(socket.AF_INET), server_hostname=hostname) conn.connect((hostname, 443)) cert = conn.getpeercert() print(cert['notAfter']) to test it.

svenfaw 5y ago
This one expires in 2066, for instance: https://censys.io/certificates/3b639a3248914d0d5e3811c080f3136976109073e5d9c04013c9dfa012b2155d/pem https://censys.io/certificates/3b639a3248914d0d5e3811c080f31...

mleonhard 5y ago

ASN.1 doesn't meet the needs of modern society. We should replace it with something simple like JSON and concrete times as 63-bit epoch timestamps. Now is the time to start making this transition, 17 years before the 31-bit epoch timestamp rollover.