6 ms·

Can the veterans of the 90s SSL Wars explain the issues with ASN1/DER/BER? Looking it up today, it seems like a pretty smart and extensive serialization system,

Can the veterans of the 90s SSL Wars explain the issues with ASN1/DER/BER? Looking it up today, it seems like a pretty smart and extensive serialization system, and I have to wonder why new systems like Google Protobufs chose to reinvent the wheel. Conversely, how have modern systems avoided the pitfalls (if any) of ASN1/DER/BER?

jwalton 5y ago
When I first saw protobufs, I wondered exactly the same thing. There’s an “XER” if you want a human-readable XML encoding, too.
carapace 5y ago
Personally, I think that people just like to reinvent things. I don't want to sound shitty (or have kentonv show up again to scold me for it) but I get the feeling that, a lot of the time, it's just that simple. https://news.ycombinator.com/item?id=20725550 https://news.ycombinator.com/item?id=20725550
- juanbyrge 5y ago
  To me that is a specious argument. It's like asking why Python was invented when Cobol could suffice. The dozens of ASN.1 specs are absolutely hideous and entrenched in obsolete telecom jargon. If the sole goal Protobuf was to avoid having Google engineers be required to refer to the dozens of ASN.1 specs when disagreements or confusions arose, then it would have been 100% worth it for just that reason.
  - carapace 5y ago
    First, let me confess that I don't have enough experience with ASN.1 or Protobufs to have an informed opinion. The supporting argument for the "because it's there" hypothesis for why people reinvent things (in IT) is that they do it so often. Even if all the newer message/serialization systems are better than ASN.1, they're not all better than each other, eh? Why so many? Same goes for chat systems, programming languages, etc.
    - cryptonector 5y ago
      There has been a lot more new stuff in the world of programming languages, even recently, than there has been in the world of data schemata and encoding rules. That said, most of the innovation in programming language theory has been around Haskell and related languages, and it has not justified languages like Golang or Python. DSLs in general are justified regardless of whether they are innovative in terms of programming language theory.
  - cryptonector 5y ago
    The ASN.1 specs are beautiful. They are beautifully written, better than anything the IETF produces because the ITU-T is an expensive standards development organization that can afford to have people who only do this sort of thing. The ASN.1 specs are very readable. Much easier to read than many important RFCs.
kevin_thibedeau 5y ago
10 different string encodings is one problem.
- AceJohnny2 5y ago
  Can one use UTF8? The 90s were rough on text encoding, but it seems pretty settled now.
  - memling 5y ago
    > Can one use UTF8? For new standards, yes. But ASN.1 was first specified in the '80s, and backwards compatibility is a thing. So really it depends on what you're doing: if you can start with a subset of ASN.1, which I think is done in MDER[0] and OER[1], you have a bit more freedom. But if you're working in legacy formats and standards that operate internationally, you could run into problems. [0]: https://www.iso.org/standard/66717.html https://www.iso.org/standard/66717.html [1]: see among others https://www.ntcip.org/document-numbers-and-status/ https://www.ntcip.org/document-numbers-and-status/
    - cryptonector 5y ago
      Kerberos implementations generally just-send-whatever in IA5String fields. That means Windows sends UTF-8, and MIT Kerberos and Heimdal send whatever the user's locale uses. Windows doesn't normalize or anything. It works in that a) it interops when using ASCII names, b) it interops when using non-ASCII names in UTF-8 locales on Unix. It violates the spec, but it works.
- cryptonector 5y ago
  Stick to UTF8String. ASN.1 predates Unicode.
  - yrro 5y ago
    Or IA5SString if you know, ahead of time, that you only need ASCII.
breser 5y ago
I know of at least one problem with ASN.1. The string encodings other than UTF-8 are terrible. Most of the string encodings are very limited and weird subsets of ASCII that nobody actually uses anymore. ASN.1 itself doesn't define the encodings and just refers to other standards. The problem with this is probably most notable with the T.61 encoding which changed over the years and since ASN.1 references other standards nobody is quite sure exactly what you have to support to have T.61 actually work right. Within X.509 certificates though nobody bothers to actually implement T.61 and just uses the T.61 flag for ISO-8859-1. There are a bunch of gory details around this mess in this (now quite old) write-up here: https://www.cs.auckland.ac.nz/~pgut001/pubs/x509guide.txt https://www.cs.auckland.ac.nz/~pgut001/pubs/x509guide.txt Since that write up I believe UTF-8 is pretty much the expectation for character encoding for X.509. I documented some of the quirks around 6 years ago when I took an existing X.509 parser and improved it for use in certificate trust management in Subversion: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_subr/x509parse.c?revision=1875971&view=markup#l863 http://svn.apache.org/viewvc/subversion/trunk/subversion/lib... Basically ASN.1 wasn't well defined and it only works well when people agreed to only use certain features or to interpret things in a particular way when ambiguous. It's also notoriously difficult to parse well. It's very easy to have bugs in your parser, even if you're implementing a subset of it that's needed for X.509. Especially if you're doing so in a non-memory safe language. I can't speak for why Google invented Protobufs, but I can't imagine anyone sane picking up ASN.1 for anything modern and deciding that this is what they want to use.
- AceJohnny2 5y ago
  > I can't imagine anyone sane picking up ASN.1 for anything modern and deciding that this is what they want to use. Part of my curiosity stems from Apple using it as part of their bootable file-format: https://www.theiphonewiki.com/wiki/IMG4_File_Format https://www.theiphonewiki.com/wiki/IMG4_File_Format But as you say, I have to assume they're using it in a very constrained way.
  - memling 5y ago
    > Part of my curiosity stems from Apple using it as part of their bootable file-format: https://www.theiphonewiki.com/wiki/IMG4_File_Format https://www.theiphonewiki.com/wiki/IMG4_File_Format I could only speculate, but I wonder if part of the reason is that DER is completely unambiguous and therefore suitable for cryptographic services. It's also very easy to decode without a specification (TLV format). Apple are almost certainly using ASN.1 compilers for their mobile devices and security layers (even if they ship FOSS implementations, I'd be surprised if they aren't checking their work with commercial compilers), so there's overlap there. Rolling your own format in that case could be unnecessary and another failure point that could be rolled into a single unit.
stevep98 5y ago
I used the Netscape/Mozilla NSS library quite a bit, and one problem I found with it, is that all of the DER encoding/decoding was written by hand. They should have generated all that boilerplate from the ASN.1 modules written in the specs (later, RFC 2459, but at the time, a hodge-podge of scattered specs). Hand-coding works okay when the data is what you expect. But when you throw mal-formed certificates at it, you have to catch all the edge cases. Having generated code would have enabled much more edge cases to be covered.
- jmspring 5y ago
  Those libraries were originally written in the early/mid 90s. Don’t recall much in the way of code generation tools that would take those specs and generate the code at the time. Spent a bunch of time working with and adding to those libraries.
abbot2 5y ago
ASN.1 is extremely complicated and hard to implement correctly. All ASN.1 implementations I've seen are either specialized (know how to work only with a very specific message), or slow, buggy and expose equally complicated APIs. Modern systems like protobufs tend to use much simpler encodings & specs which are easier to understand and implement correctly.
wbl 5y ago
Oh where to begin? ASN.1 really demands code generation. Unfortunately lots of nonconforming stuff has to be dealt with. The concept of encoding rules and the module tagging scheme make for a pretty big number of possible representations. The language semantics of ASN.1 don't really map to anything well, particularly around default fields and structures that can vary. Newer systems don't have encoding rules and pick a semantics that matches a target language much more closely.
- wbl 5y ago
  Also expect to pay to read the spec.
  - memling 5y ago
    ASN.1 standards are free: https://www.itu.int/rec/t-rec-x.680/en https://www.itu.int/rec/t-rec-x.680/en Many, though not all, specifications that use ASN.1 are also freely available. I've been out of telecom for awhile, so I don't know the status of the newer standards, but when I was working in the business GSM MAP and MMS were the only proprietary ones that were an issue.
    - userbinator 5y ago
      GSM standards are also freely available --- look at 3gpp.org or etsi.org --- the biggest problem is finding which ones actually contain what you're looking for.
  - cryptonector 5y ago
    The ITU-T ASN.1 specs have been free for a very long time now. They used to be non-free, and that was a big problem with ASN.1, but that was decades ago.
- AceJohnny2 5y ago
  > ASN.1 really demands code generation. On this specific point: isn't this also the case for other high-performance serialisers? Google ProtoBufs, Apache Thrift, any protocol through Rust's SerDes...
bri3d 5y ago
ASN.1 was too broad. There is immense value in a more constrained specification that does not include so many hazardous serialization types and antiquated string formats. Now, should Protobufs or Thrift simply have been constrained versions of ASN.1? I think there is a view of software engineering where this would have been an ideal outcome, but almost universally when we see too-big standards, they are declared "dangerous" and avoided like the plague before they are downscoped.
- cryptonector 5y ago
  ASN.1 in 1984 was not too broad. It was too simple, and it was too targeted to tag-length-value encoding rules (which are stupid -- TLV is a crutch that is only maybe useful when you lack a compiler, which early on was the case). ASN.1 today is as broad as it needed to evolve to be because its users needed it.
  - oblio 5y ago
    There is value in throwing away cruft, especially cruft that comes from the IT Middle Ages (before we decided to drop any non 8 bit word sizes, before UTF-8 became the almost universal string encoding, etc.).
    - cryptonector 5y ago
      Maybe, but ASN.1 is a Chesterton's fence. Before you throw it away and reinvent it badly, acquaint yourself with it. And you might notice that ASN.1 has a long history, but it's thoroughly modern today, and much more so than many alternatives to ASN.1 that have been created even in recent times.
      - bri3d 5y ago
        I agree with this, and I think that overall the Chesterton's Fence principle should be applied more in software engineering. What's hard is finding a set of "thoroughly modern" ASN.1 implementations that work together, and trusting that they will do so. The name is overloaded by the years of revisions and cruft.
eismcc 5y ago
I owned the Microsoft ASN1 library for a while around 2005. It was a maintenance nightmare and I spent a lot of time fixing static analysis derived issues. That said, I always found the standard quite interesting with different encodings based on the degree of prior shared info or format. My assumption is that not-invented-here is part of the why it’s not used.
- cryptonector 5y ago
  I own Heimdal's ASN.1 compiler. It's a pleasure.
jupp0r 5y ago
No veteran of the 90s SSL wars, but I once upon the time was tasked with fixing security bugs in a custom protocol backend server which used ASN.1 for purposes that one would probably use protobuf nowadays. The quality of existing open source libraries to parse ASN.1 leaves a lot to be desired.
cryptonector 5y ago
There is NO problem with ASN.1 itself except a bit of ugliness. There are SERIOUS problems with DER/BER/CER and with all tag-length-value schemes -- this includes protobufs! ASN.1 is just syntax and semantics. There are encoding rules that produce textual representations (GSER), XML (XER), JSON (JER), there's XDR-style encoding rules (PER and OER, but with 1-octet units instead of 4-octet units, plus efficient representation of optional fields). In fact, you can make ASN.1 encoding rules that are based on NDR and XDR and which work for all of IDL and XDR and that subset of ASN.1 that is covered by the semantics of IDL and XDR, and you can extend that to cover all of ASN.1 if you want. I should know these things, as I maintain an ASN.1 compiler and I intend to eventually teach it to do XDR and NDR. Really, there's nothing about data schemas that you can express in JSON, CBOR, IDL, XDR, S-expressions, or any schema language you want, that you can't express in ASN.1, or, if there is, it's got to be a pretty niche feature and easily added to ASN.1 anyways. Even functions (RPCs) can be expressed in ASN.1 with some conventions, and routinely are, because it's really just a request/response protocol. But every year someone invents a new thing because of how stupid, tired, and old ASN.1 is (or, rather, they perceive it to be). Or because of how complex ASN.1 is and how there's a paucity of tools, so then they: reinvent the wheel (often badly), a wheel for which instantly there is a paucity of tools.
rixed 5y ago
Have spent a few years during the late 90s/early 2000s in an industry running on ASN.1, coming from the web. I was initially surprised by how enamoured most of my coworkers were with ASN.1 and its tools, but it grew on me too: the pleasure of interacting only with a protocol specifications regardless of the implementation language/intricacies of the remote party, the guaranty that there could be no invalid messages received or emitted, the automatic generation of tests and tools, eventually balanced out the inconvenience of not being able to readily read data on the wire (it was before every human-readable protocols gets encrypted) and the inconvenience of not being able to start coding upfront. It was like going from runtime type checking to static type checking: initially inconvenient, but paying dividends after a short while. So why did this tech disappeared if it was ultimately better than the later alternatives (textual protocols, shema-less serializers, and eventually protobuf which reinstated some form of efficient encoding and type checking). As it uncannily frequently occurs with technological evolution, the reason is probably not to be found within its technical issues (which basically all boil down to: designed by committee). ASN.1 was just a bit too inconvenient, the free tools to generate code were just not quite good and robust enough, and the approach of starting with designing your types and protocols and putting in place your code production tool-chain before being able to ship anything was at odd with the mood of the day, which was to let the junior cheap dev fire off his code editor during the coffee break of the first design planning meeting to build the first half-backed prototype that would be already sold to the customer by the time he hits :wq. To move fast and break things, ASN.1 got in the way. So did formal specifications in general, code analyzing tools, even basic type checking, all of them thrown out the window during the same period for the extra weight, extra time-to-market and extra cost of hiring. Text protocols out competing saner alternatives because they are initially simpler (SIP vs H.323 anyone?), schema-less data formats predominating almost entirely because you can start hacking quicker, etc. are all attributable to that cultural rather than technical trend I believe. Now it seems the industry is slowly recovering from these excesses. Maybe because of the damage that has done, but more likely because of the end of cheap hardware progresses, encryption everywhere and massive data volumes (that's what made Google come up with better protocols than HTTP and better formats than human readable text, after all).

jwalton 5y ago

When I first saw protobufs, I wondered exactly the same thing. There’s an “XER” if you want a human-readable XML encoding, too.

carapace 5y ago

Personally, I think that people just like to reinvent things. I don't want to sound shitty (or have kentonv show up again to scold me for it) but I get the feeling that, a lot of the time, it's just that simple. https://news.ycombinator.com/item?id=20725550 https://news.ycombinator.com/item?id=20725550

juanbyrge 5y ago
To me that is a specious argument. It's like asking why Python was invented when Cobol could suffice. The dozens of ASN.1 specs are absolutely hideous and entrenched in obsolete telecom jargon. If the sole goal Protobuf was to avoid having Google engineers be required to refer to the dozens of ASN.1 specs when disagreements or confusions arose, then it would have been 100% worth it for just that reason.
- carapace 5y ago
  First, let me confess that I don't have enough experience with ASN.1 or Protobufs to have an informed opinion. The supporting argument for the "because it's there" hypothesis for why people reinvent things (in IT) is that they do it so often. Even if all the newer message/serialization systems are better than ASN.1, they're not all better than each other, eh? Why so many? Same goes for chat systems, programming languages, etc.
  - cryptonector 5y ago
    There has been a lot more new stuff in the world of programming languages, even recently, than there has been in the world of data schemata and encoding rules. That said, most of the innovation in programming language theory has been around Haskell and related languages, and it has not justified languages like Golang or Python. DSLs in general are justified regardless of whether they are innovative in terms of programming language theory.
- cryptonector 5y ago
  The ASN.1 specs are beautiful. They are beautifully written, better than anything the IETF produces because the ITU-T is an expensive standards development organization that can afford to have people who only do this sort of thing. The ASN.1 specs are very readable. Much easier to read than many important RFCs.

kevin_thibedeau 5y ago

10 different string encodings is one problem.

AceJohnny2 5y ago
Can one use UTF8? The 90s were rough on text encoding, but it seems pretty settled now.
- memling 5y ago
  > Can one use UTF8? For new standards, yes. But ASN.1 was first specified in the '80s, and backwards compatibility is a thing. So really it depends on what you're doing: if you can start with a subset of ASN.1, which I think is done in MDER[0] and OER[1], you have a bit more freedom. But if you're working in legacy formats and standards that operate internationally, you could run into problems. [0]: https://www.iso.org/standard/66717.html https://www.iso.org/standard/66717.html [1]: see among others https://www.ntcip.org/document-numbers-and-status/ https://www.ntcip.org/document-numbers-and-status/
  - cryptonector 5y ago
    Kerberos implementations generally just-send-whatever in IA5String fields. That means Windows sends UTF-8, and MIT Kerberos and Heimdal send whatever the user's locale uses. Windows doesn't normalize or anything. It works in that a) it interops when using ASCII names, b) it interops when using non-ASCII names in UTF-8 locales on Unix. It violates the spec, but it works.
cryptonector 5y ago
Stick to UTF8String. ASN.1 predates Unicode.
- yrro 5y ago
  Or IA5SString if you know, ahead of time, that you only need ASCII.

breser 5y ago

I know of at least one problem with ASN.1. The string encodings other than UTF-8 are terrible. Most of the string encodings are very limited and weird subsets of ASCII that nobody actually uses anymore. ASN.1 itself doesn't define the encodings and just refers to other standards. The problem with this is probably most notable with the T.61 encoding which changed over the years and since ASN.1 references other standards nobody is quite sure exactly what you have to support to have T.61 actually work right. Within X.509 certificates though nobody bothers to actually implement T.61 and just uses the T.61 flag for ISO-8859-1. There are a bunch of gory details around this mess in this (now quite old) write-up here: https://www.cs.auckland.ac.nz/~pgut001/pubs/x509guide.txt https://www.cs.auckland.ac.nz/~pgut001/pubs/x509guide.txt Since that write up I believe UTF-8 is pretty much the expectation for character encoding for X.509. I documented some of the quirks around 6 years ago when I took an existing X.509 parser and improved it for use in certificate trust management in Subversion: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_subr/x509parse.c?revision=1875971&view=markup#l863 http://svn.apache.org/viewvc/subversion/trunk/subversion/lib... Basically ASN.1 wasn't well defined and it only works well when people agreed to only use certain features or to interpret things in a particular way when ambiguous. It's also notoriously difficult to parse well. It's very easy to have bugs in your parser, even if you're implementing a subset of it that's needed for X.509. Especially if you're doing so in a non-memory safe language. I can't speak for why Google invented Protobufs, but I can't imagine anyone sane picking up ASN.1 for anything modern and deciding that this is what they want to use.

AceJohnny2 5y ago
> I can't imagine anyone sane picking up ASN.1 for anything modern and deciding that this is what they want to use. Part of my curiosity stems from Apple using it as part of their bootable file-format: https://www.theiphonewiki.com/wiki/IMG4_File_Format https://www.theiphonewiki.com/wiki/IMG4_File_Format But as you say, I have to assume they're using it in a very constrained way.
- memling 5y ago
  > Part of my curiosity stems from Apple using it as part of their bootable file-format: https://www.theiphonewiki.com/wiki/IMG4_File_Format https://www.theiphonewiki.com/wiki/IMG4_File_Format I could only speculate, but I wonder if part of the reason is that DER is completely unambiguous and therefore suitable for cryptographic services. It's also very easy to decode without a specification (TLV format). Apple are almost certainly using ASN.1 compilers for their mobile devices and security layers (even if they ship FOSS implementations, I'd be surprised if they aren't checking their work with commercial compilers), so there's overlap there. Rolling your own format in that case could be unnecessary and another failure point that could be rolled into a single unit.

stevep98 5y ago

I used the Netscape/Mozilla NSS library quite a bit, and one problem I found with it, is that all of the DER encoding/decoding was written by hand. They should have generated all that boilerplate from the ASN.1 modules written in the specs (later, RFC 2459, but at the time, a hodge-podge of scattered specs). Hand-coding works okay when the data is what you expect. But when you throw mal-formed certificates at it, you have to catch all the edge cases. Having generated code would have enabled much more edge cases to be covered.

jmspring 5y ago
Those libraries were originally written in the early/mid 90s. Don’t recall much in the way of code generation tools that would take those specs and generate the code at the time. Spent a bunch of time working with and adding to those libraries.

abbot2 5y ago

ASN.1 is extremely complicated and hard to implement correctly. All ASN.1 implementations I've seen are either specialized (know how to work only with a very specific message), or slow, buggy and expose equally complicated APIs. Modern systems like protobufs tend to use much simpler encodings & specs which are easier to understand and implement correctly.

wbl 5y ago

Oh where to begin? ASN.1 really demands code generation. Unfortunately lots of nonconforming stuff has to be dealt with. The concept of encoding rules and the module tagging scheme make for a pretty big number of possible representations. The language semantics of ASN.1 don't really map to anything well, particularly around default fields and structures that can vary. Newer systems don't have encoding rules and pick a semantics that matches a target language much more closely.

wbl 5y ago
Also expect to pay to read the spec.
- memling 5y ago
  ASN.1 standards are free: https://www.itu.int/rec/t-rec-x.680/en https://www.itu.int/rec/t-rec-x.680/en Many, though not all, specifications that use ASN.1 are also freely available. I've been out of telecom for awhile, so I don't know the status of the newer standards, but when I was working in the business GSM MAP and MMS were the only proprietary ones that were an issue.
  - userbinator 5y ago
    GSM standards are also freely available --- look at 3gpp.org or etsi.org --- the biggest problem is finding which ones actually contain what you're looking for.
- cryptonector 5y ago
  The ITU-T ASN.1 specs have been free for a very long time now. They used to be non-free, and that was a big problem with ASN.1, but that was decades ago.
AceJohnny2 5y ago
> ASN.1 really demands code generation. On this specific point: isn't this also the case for other high-performance serialisers? Google ProtoBufs, Apache Thrift, any protocol through Rust's SerDes...

bri3d 5y ago

ASN.1 was too broad. There is immense value in a more constrained specification that does not include so many hazardous serialization types and antiquated string formats. Now, should Protobufs or Thrift simply have been constrained versions of ASN.1? I think there is a view of software engineering where this would have been an ideal outcome, but almost universally when we see too-big standards, they are declared "dangerous" and avoided like the plague before they are downscoped.

cryptonector 5y ago
ASN.1 in 1984 was not too broad. It was too simple, and it was too targeted to tag-length-value encoding rules (which are stupid -- TLV is a crutch that is only maybe useful when you lack a compiler, which early on was the case). ASN.1 today is as broad as it needed to evolve to be because its users needed it.
- oblio 5y ago
  There is value in throwing away cruft, especially cruft that comes from the IT Middle Ages (before we decided to drop any non 8 bit word sizes, before UTF-8 became the almost universal string encoding, etc.).
  - cryptonector 5y ago
    Maybe, but ASN.1 is a Chesterton's fence. Before you throw it away and reinvent it badly, acquaint yourself with it. And you might notice that ASN.1 has a long history, but it's thoroughly modern today, and much more so than many alternatives to ASN.1 that have been created even in recent times.
    - bri3d 5y ago
      I agree with this, and I think that overall the Chesterton's Fence principle should be applied more in software engineering. What's hard is finding a set of "thoroughly modern" ASN.1 implementations that work together, and trusting that they will do so. The name is overloaded by the years of revisions and cruft.

eismcc 5y ago

I owned the Microsoft ASN1 library for a while around 2005. It was a maintenance nightmare and I spent a lot of time fixing static analysis derived issues. That said, I always found the standard quite interesting with different encodings based on the degree of prior shared info or format. My assumption is that not-invented-here is part of the why it’s not used.

cryptonector 5y ago
I own Heimdal's ASN.1 compiler. It's a pleasure.

jupp0r 5y ago

No veteran of the 90s SSL wars, but I once upon the time was tasked with fixing security bugs in a custom protocol backend server which used ASN.1 for purposes that one would probably use protobuf nowadays. The quality of existing open source libraries to parse ASN.1 leaves a lot to be desired.

cryptonector 5y ago

There is NO problem with ASN.1 itself except a bit of ugliness. There are SERIOUS problems with DER/BER/CER and with all tag-length-value schemes -- this includes protobufs! ASN.1 is just syntax and semantics. There are encoding rules that produce textual representations (GSER), XML (XER), JSON (JER), there's XDR-style encoding rules (PER and OER, but with 1-octet units instead of 4-octet units, plus efficient representation of optional fields). In fact, you can make ASN.1 encoding rules that are based on NDR and XDR and which work for all of IDL and XDR and that subset of ASN.1 that is covered by the semantics of IDL and XDR, and you can extend that to cover all of ASN.1 if you want. I should know these things, as I maintain an ASN.1 compiler and I intend to eventually teach it to do XDR and NDR. Really, there's nothing about data schemas that you can express in JSON, CBOR, IDL, XDR, S-expressions, or any schema language you want, that you can't express in ASN.1, or, if there is, it's got to be a pretty niche feature and easily added to ASN.1 anyways. Even functions (RPCs) can be expressed in ASN.1 with some conventions, and routinely are, because it's really just a request/response protocol. But every year someone invents a new thing because of how stupid, tired, and old ASN.1 is (or, rather, they perceive it to be). Or because of how complex ASN.1 is and how there's a paucity of tools, so then they: reinvent the wheel (often badly), a wheel for which instantly there is a paucity of tools.

rixed 5y ago

Have spent a few years during the late 90s/early 2000s in an industry running on ASN.1, coming from the web. I was initially surprised by how enamoured most of my coworkers were with ASN.1 and its tools, but it grew on me too: the pleasure of interacting only with a protocol specifications regardless of the implementation language/intricacies of the remote party, the guaranty that there could be no invalid messages received or emitted, the automatic generation of tests and tools, eventually balanced out the inconvenience of not being able to readily read data on the wire (it was before every human-readable protocols gets encrypted) and the inconvenience of not being able to start coding upfront. It was like going from runtime type checking to static type checking: initially inconvenient, but paying dividends after a short while. So why did this tech disappeared if it was ultimately better than the later alternatives (textual protocols, shema-less serializers, and eventually protobuf which reinstated some form of efficient encoding and type checking). As it uncannily frequently occurs with technological evolution, the reason is probably not to be found within its technical issues (which basically all boil down to: designed by committee). ASN.1 was just a bit too inconvenient, the free tools to generate code were just not quite good and robust enough, and the approach of starting with designing your types and protocols and putting in place your code production tool-chain before being able to ship anything was at odd with the mood of the day, which was to let the junior cheap dev fire off his code editor during the coffee break of the first design planning meeting to build the first half-backed prototype that would be already sold to the customer by the time he hits :wq. To move fast and break things, ASN.1 got in the way. So did formal specifications in general, code analyzing tools, even basic type checking, all of them thrown out the window during the same period for the extra weight, extra time-to-market and extra cost of hiring. Text protocols out competing saner alternatives because they are initially simpler (SIP vs H.323 anyone?), schema-less data formats predominating almost entirely because you can start hacking quicker, etc. are all attributable to that cultural rather than technical trend I believe. Now it seems the industry is slowly recovering from these excesses. Maybe because of the damage that has done, but more likely because of the end of cheap hardware progresses, encryption everywhere and massive data volumes (that's what made Google come up with better protocols than HTTP and better formats than human readable text, after all).