6 ms·

A warning, ASN.1 is disliked by many people in software security. It often features in security vulnerabilities involving TLS and LDAP/Active Directory. They wo

A warning, ASN.1 is disliked by many people in software security. It often features in security vulnerabilities involving TLS and LDAP/Active Directory. They would recommend against adopting ASN.1 for new projects. I get the impression this partially due to a) most implementations being adhoc, of varying completeness, and written in C or C++ (e.g. OpenSSL); and b) sheer complexity of ASN.1 and other standards from that family/era. Sources: following software security researchers/practioners on Twitter, a few podcasts, here, etc.

rdpintqogeogsaa 5y ago
This root cause analysis matches up with my experience. You can absolutely have safe ASN.1 with ASN.1 compilers, but the implementations that star in these vulnerabilities tend to be generic or ad-hoc DER/BER parsers. You'd be hard-pressed to be able to blame ASN.1 with XER (XML Encoding Rules) encoding since then you'd throw an XML parser at the problem, side-stepping the bit twiddling and exchanging it for XML parsing bugs instead. Similarly, if all you have is fixed-length items, OER (Octet Encoding Rules), you essentially get a formally specified struct with a well-defined way to go from there to XML or JSON with JER (JSON Encoding Rules). I imagine we'll see this happen with CBOR (whose ASN.1 equivalent is technically CDDL) a few years down the road.
- riedel 5y ago
  I think the root cause of this is that ASN.1 is deemed uncool and it is difficult to implement a standard compliant FOSS compiler for all the encodings. I was surprised doing research into binary XML and JSON how far you get with ASN.1 . However, the ecosystem seems so discoupled from anything else that there is rarely an implementation path for a quick win beyond mentioned hacky parsers that are both error prone and inefficient or purchasing some obscure commercial complier as I remember. Edit: would be really happy to see some good open source compilers with extensible encoding support being listed here
  - rdpintqogeogsaa 5y ago
    > Edit: would be really happy to see some good open source compilers with extensible encoding support being listed here I think that's the reason of ASN.1 being trapped in a very restrictive space. From Olivier Dubuisson's ASN.1 -- Communication between Heterogeneous Systems, p. 86 et seq. talking about use of ASN.1 by the IETF: > the definition of many macros and macro instances to represent semantic links instead of information object classes and information objects although no ASN.1 compiler properly takes into account the macro concept (on the other hand, no compiler of the public domain does with the information object class concept unfortunately); In essence, ecosystem failure due to a non-existent overlap of the telecom industry (full of commercial and expensive solutions) and the Internet community (working in a standards committee that tracks pre-existing permissionless innovation distributing the standards at no cost, strongly affiliated with free software ideas) caused ASN.1 to fall by the roadside. Now that ASN.1 is considered uncool, the ecosystem certainly won't be coming anymore, so we'll be forced to reinvent ASN.1, probably re-learning every lesson along the way.
    - rusk 5y ago
      > we'll be forced to reinvent ASN.1 I think this already happened with Google Protocol Buffers
    - kortex 5y ago
      > In essence, ecosystem failure due to a non-existent overlap of the telecom industry (full of commercial and expensive solutions) and the Internet community (working in a standards committee that tracks pre-existing permissionless innovation distributing the standards at no cost, strongly affiliated with free software ideas) caused ASN.1 to fall by the roadside. Yep. I think this is the crux of the matter. I don't even think it is an "uncool" thing. In a word, IMHO it's too "closed". Xml is not exactly hip, but I can find high scoring python xml libraries on snyk.io. I know that is not a very scientific metric but it's a good proxy for a bunch of things that are hard to measure. I think even if it were "cool", it would be a struggle to get a culture to nucleate. It has the same vibe as codecs: complex, kinda arcane, too closely associated with closed source, litigious, corporate culture. It doesn't help that it is guilty by association with numerous poorly written libraries. Unless that is what you mean by "uncool". Now that there is cbor, msgpack, cap'n proto, protobuf, thrift, and a bunch others (just in the binary space), asn.1 would have to make a really compelling argument to win out. It would need a lot more than coolness.
rixrax 5y ago
Complete ASN.1 by prof John Larmouth[0] on page 366 contains following epiphany: If you were to wave a magic wand and eliminate from the world all messages that are encodings of ASN.1-defined values, disaster would certainly strike on a scale far beyond any that the most pessimistic have described for possible effects of the Y2K computer bugs. Aircraft would collide, mobile phones would cease to work, virtually all telecoms and network switches would be unmanageable and unmaintainable and would gradually die, electric power distribution systems would cease to work, and ... smart-card-based electronic transactions would fail to complete and your washing machine might fail to work ... and your life would become a misery! -- It's suffice to say that them security people have proven, and unfortunately are likely to keep proving him to have been right on more than one occasion. [0] https://www.oss.com/asn1/resources/books-whitepapers-pubs/larmouth-asn1-book.pdf https://www.oss.com/asn1/resources/books-whitepapers-pubs/la...
- throwaway894345 5y ago
  You could make the same sort of argument about C and yet we know for certain that it is a particularly insecure and error prone language.
rixed 5y ago
What do the security experts recommend?
- emmelaich 5y ago
  XDR perhaps? Less ambitious than ASN1. Used by many systems. https://en.wikipedia.org/wiki/External_Data_Representation https://en.wikipedia.org/wiki/External_Data_Representation
- bawolff 5y ago
  Generally speaking, from a security standpoint, simpler is better. It of course depends on your requirements, but if you just need a structured data exchange format, JSON is an obvious choice. But again, it depends on what your requirements are.
  - 5e92cb50239222b 5y ago
    JSON is not a simple format. I don't feel confident enough to recommend anything in particular, but I believe one of the well-designed binary formats would be a much better choice. I'll just leave this here. https://news.ycombinator.com/item?id=28826600 https://news.ycombinator.com/item?id=28826600 https://seriot.ch/projects/parsing_json.html https://seriot.ch/projects/parsing_json.html
  - CodesInChaos 5y ago
    IMO Json is more complex than the popular DER/BER encoding of ASN.1 and suffers from an ambiguous specification.
  - pantalaimon 5y ago
    I'd rather use CBOR instead of JSON when doing anything outside the browser that doesn't need to be human editable
    - formerly_proven 5y ago
      IMHO: CBOR is a worse encoding than msgpack, despite being a copy-paste clone of msgpack: Indefinite lengths, weird fp formats (decimal floats?), iirc CBOR also "improved" msgpack's int encoding by making it annoying to de- and encode, the whole tagging system is superfluous etc. Just because something has an RFC number doesn't make it good.
nullc 5y ago
> a) most implementations being adhoc, of varying completeness, While attempting to implement clean auditable code which would accept all signatures openssl accepted and reject all that it rejected years ago I fell down a rabbit hole of trying to discover if there was any complete and correct open source implementations of BER. I audited dozens of them and was not able to find a single one. They all implement different subsets and have various flaws in the more weird/useless ways of encoding things. This effort eventually result in an openssl CVE, and we never got a consistent implementation: the exact set of messages accepted was far too irregular and dependent on the implementation. (OpenSSL eventually 'fixed' the issue by restricting the set of accepted inputs ... in an uncoordinated manner guaranteed to create a vulnerability for any system where consistent validation is security critical.) ASN.1 isn't unique in this though. Almost all other complex parsed formats with multiple implementations have troublesome inconsistencies too. The gap between "make it work" and "make it always work correctly" is too big. People make code that is sufficient for the purposes they care about and then it gets deployed into places where it's not sufficient but good enough to look like it is for a little while.
userbinator 5y ago
It often features in security vulnerabilities As the saying goes, "don't shoot the messenger". There's more quantity than quality when it comes to ASN.1 implementations, especially free ones, and that's largely where the impression is coming from. DER in particular is not hard to parse, but people don't seem to bother with thinking carefully about the edge cases and such.
johnisgood 5y ago
I am using ASN.1 because Erlang has a pretty awesome implementation of it. The documentation[1][2] is great too. [1] https://www.erlang.org/doc/apps/asn1/asn1.pdf https://www.erlang.org/doc/apps/asn1/asn1.pdf [2] https://www.erlang.org/doc/apps/asn1/asn1_getting_started.html https://www.erlang.org/doc/apps/asn1/asn1_getting_started.ht...

rdpintqogeogsaa 5y ago

This root cause analysis matches up with my experience. You can absolutely have safe ASN.1 with ASN.1 compilers, but the implementations that star in these vulnerabilities tend to be generic or ad-hoc DER/BER parsers. You'd be hard-pressed to be able to blame ASN.1 with XER (XML Encoding Rules) encoding since then you'd throw an XML parser at the problem, side-stepping the bit twiddling and exchanging it for XML parsing bugs instead. Similarly, if all you have is fixed-length items, OER (Octet Encoding Rules), you essentially get a formally specified struct with a well-defined way to go from there to XML or JSON with JER (JSON Encoding Rules). I imagine we'll see this happen with CBOR (whose ASN.1 equivalent is technically CDDL) a few years down the road.

riedel 5y ago
I think the root cause of this is that ASN.1 is deemed uncool and it is difficult to implement a standard compliant FOSS compiler for all the encodings. I was surprised doing research into binary XML and JSON how far you get with ASN.1 . However, the ecosystem seems so discoupled from anything else that there is rarely an implementation path for a quick win beyond mentioned hacky parsers that are both error prone and inefficient or purchasing some obscure commercial complier as I remember. Edit: would be really happy to see some good open source compilers with extensible encoding support being listed here
- rdpintqogeogsaa 5y ago
  > Edit: would be really happy to see some good open source compilers with extensible encoding support being listed here I think that's the reason of ASN.1 being trapped in a very restrictive space. From Olivier Dubuisson's ASN.1 -- Communication between Heterogeneous Systems, p. 86 et seq. talking about use of ASN.1 by the IETF: > the definition of many macros and macro instances to represent semantic links instead of information object classes and information objects although no ASN.1 compiler properly takes into account the macro concept (on the other hand, no compiler of the public domain does with the information object class concept unfortunately); In essence, ecosystem failure due to a non-existent overlap of the telecom industry (full of commercial and expensive solutions) and the Internet community (working in a standards committee that tracks pre-existing permissionless innovation distributing the standards at no cost, strongly affiliated with free software ideas) caused ASN.1 to fall by the roadside. Now that ASN.1 is considered uncool, the ecosystem certainly won't be coming anymore, so we'll be forced to reinvent ASN.1, probably re-learning every lesson along the way.
  - rusk 5y ago
    > we'll be forced to reinvent ASN.1 I think this already happened with Google Protocol Buffers
  - kortex 5y ago
    > In essence, ecosystem failure due to a non-existent overlap of the telecom industry (full of commercial and expensive solutions) and the Internet community (working in a standards committee that tracks pre-existing permissionless innovation distributing the standards at no cost, strongly affiliated with free software ideas) caused ASN.1 to fall by the roadside. Yep. I think this is the crux of the matter. I don't even think it is an "uncool" thing. In a word, IMHO it's too "closed". Xml is not exactly hip, but I can find high scoring python xml libraries on snyk.io. I know that is not a very scientific metric but it's a good proxy for a bunch of things that are hard to measure. I think even if it were "cool", it would be a struggle to get a culture to nucleate. It has the same vibe as codecs: complex, kinda arcane, too closely associated with closed source, litigious, corporate culture. It doesn't help that it is guilty by association with numerous poorly written libraries. Unless that is what you mean by "uncool". Now that there is cbor, msgpack, cap'n proto, protobuf, thrift, and a bunch others (just in the binary space), asn.1 would have to make a really compelling argument to win out. It would need a lot more than coolness.

rixrax 5y ago

Complete ASN.1 by prof John Larmouth[0] on page 366 contains following epiphany: If you were to wave a magic wand and eliminate from the world all messages that are encodings of ASN.1-defined values, disaster would certainly strike on a scale far beyond any that the most pessimistic have described for possible effects of the Y2K computer bugs. Aircraft would collide, mobile phones would cease to work, virtually all telecoms and network switches would be unmanageable and unmaintainable and would gradually die, electric power distribution systems would cease to work, and ... smart-card-based electronic transactions would fail to complete and your washing machine might fail to work ... and your life would become a misery! -- It's suffice to say that them security people have proven, and unfortunately are likely to keep proving him to have been right on more than one occasion. [0] https://www.oss.com/asn1/resources/books-whitepapers-pubs/larmouth-asn1-book.pdf https://www.oss.com/asn1/resources/books-whitepapers-pubs/la...

throwaway894345 5y ago
You could make the same sort of argument about C and yet we know for certain that it is a particularly insecure and error prone language.

rixed 5y ago

What do the security experts recommend?

emmelaich 5y ago
XDR perhaps? Less ambitious than ASN1. Used by many systems. https://en.wikipedia.org/wiki/External_Data_Representation https://en.wikipedia.org/wiki/External_Data_Representation
bawolff 5y ago
Generally speaking, from a security standpoint, simpler is better. It of course depends on your requirements, but if you just need a structured data exchange format, JSON is an obvious choice. But again, it depends on what your requirements are.
- 5e92cb50239222b 5y ago
  JSON is not a simple format. I don't feel confident enough to recommend anything in particular, but I believe one of the well-designed binary formats would be a much better choice. I'll just leave this here. https://news.ycombinator.com/item?id=28826600 https://news.ycombinator.com/item?id=28826600 https://seriot.ch/projects/parsing_json.html https://seriot.ch/projects/parsing_json.html
- CodesInChaos 5y ago
  IMO Json is more complex than the popular DER/BER encoding of ASN.1 and suffers from an ambiguous specification.
- pantalaimon 5y ago
  I'd rather use CBOR instead of JSON when doing anything outside the browser that doesn't need to be human editable
  - formerly_proven 5y ago
    IMHO: CBOR is a worse encoding than msgpack, despite being a copy-paste clone of msgpack: Indefinite lengths, weird fp formats (decimal floats?), iirc CBOR also "improved" msgpack's int encoding by making it annoying to de- and encode, the whole tagging system is superfluous etc. Just because something has an RFC number doesn't make it good.

nullc 5y ago

> a) most implementations being adhoc, of varying completeness, While attempting to implement clean auditable code which would accept all signatures openssl accepted and reject all that it rejected years ago I fell down a rabbit hole of trying to discover if there was any complete and correct open source implementations of BER. I audited dozens of them and was not able to find a single one. They all implement different subsets and have various flaws in the more weird/useless ways of encoding things. This effort eventually result in an openssl CVE, and we never got a consistent implementation: the exact set of messages accepted was far too irregular and dependent on the implementation. (OpenSSL eventually 'fixed' the issue by restricting the set of accepted inputs ... in an uncoordinated manner guaranteed to create a vulnerability for any system where consistent validation is security critical.) ASN.1 isn't unique in this though. Almost all other complex parsed formats with multiple implementations have troublesome inconsistencies too. The gap between "make it work" and "make it always work correctly" is too big. People make code that is sufficient for the purposes they care about and then it gets deployed into places where it's not sufficient but good enough to look like it is for a little while.

userbinator 5y ago

It often features in security vulnerabilities As the saying goes, "don't shoot the messenger". There's more quantity than quality when it comes to ASN.1 implementations, especially free ones, and that's largely where the impression is coming from. DER in particular is not hard to parse, but people don't seem to bother with thinking carefully about the edge cases and such.

johnisgood 5y ago

I am using ASN.1 because Erlang has a pretty awesome implementation of it. The documentation[1][2] is great too. [1] https://www.erlang.org/doc/apps/asn1/asn1.pdf https://www.erlang.org/doc/apps/asn1/asn1.pdf [2] https://www.erlang.org/doc/apps/asn1/asn1_getting_started.html https://www.erlang.org/doc/apps/asn1/asn1_getting_started.ht...