5 ms·

> You might have heard of similar such abstract syntax notations used for interface definitions such as Google Protocol Buffers, or Facebook’s Apache Thrift, bu

> You might have heard of similar such abstract syntax notations used for interface definitions such as Google Protocol Buffers, or Facebook’s Apache Thrift, but those languages have not been managed by a standardization organization, so the owning corporations could (in theory) make breaking changes or change the license or even remove the language definitions overnight. Is this really the main difference between ASN.1 and Google protobufs, that one is managed by a private corporation and the other by a standardization organization? Can they otherwise be used "interchangably" in designing interfaces, a la two different programming languages (with different syntax of course)?

jwalton 5y ago
In terms of tooling, there’s excellent tooling for ASN.1 for C and C++ and maybe some other languages. There’s excellent tooling for protobufs for a handful of languages too, but they’re different sets, so in practice what languages you want to use would likely come into play.
- breser 5y ago
  How excellent the ASN.1 tooling is depends on which subset of ASN.1 you're using. Some of the tooling supports one iteration of ASN.1 or the other. To the degree that the IETF had to write a document on how to deal with this since some of the standards use the older ASN.1 and some use the newer ASN.1: https://tools.ietf.org/id/draft-ietf-pkix-asn1-translation-03.html https://tools.ietf.org/id/draft-ietf-pkix-asn1-translation-0... Interoperability with ASN.1 is very fragile at best.
  - cryptonector 5y ago
    We have tons of interoperable PKIX implementations (OpenSSL and derivatives, NSS, OpenJDK's, GnuTLS, wolfSSL, Heimdal, and many many more), and a bunch of interoperable Kerberos implementations (MIT Kerberos, Heimdal, Windows / AD, OpenJDK's, the IBM Java's, GNU Shishi, there's a python implementation).
  - cryptonector 5y ago
    BTW, that I-D is now RFC 6025 [0]. There's also RFC 5912 [1], which adds x.681/x.682/x.683 constraints to PKIX modules. I use this to great effect in Heimdal[2]. One function call can decode everything in a certificate, and a second can pretty print it in JSON; one command can pretty-print a certificate in all its glory in JSON. [0] https://datatracker.ietf.org/doc/html/rfc6025 [1] https://datatracker.ietf.org/doc/html/rfc5912 [2] https://github.com/heimdal/heimdal https://github.com/heimdal/heimdal/tree/master/lib/asn1
- memling 5y ago
  > In terms of tooling, there’s excellent tooling for ASN.1 for C and C++ and maybe some other languages. There’s excellent tooling for protobufs for a handful of languages too, but they’re different sets, so in practice what languages you want to use would likely come into play. In my experience, tooling is actually very good for most commonly-used languages, including C/C++, C#, Java, Python, and maybe even Go. And, of course, erlang. The real challenge is, I think, that you cannot find good free tooling, and the barrier to entry for Joe Developer is fairly high (in the thousands of dollars).
  - dikei 5y ago
    I concur. Furthermore, the pricing if often opaque: you know the tools are expensive when they always want you to contact sales for a quote.
memling 5y ago
> Is this really the main difference between ASN.1 and Google protobufs, that one is managed by a private corporation and the other by a standardization organization? Can they otherwise be used "interchangably" in designing interfaces, a la two different programming languages (with different syntax of course)? No, the two are not interoperable and probably won't be made that way. Protobuf has undergone changes that challenge its backwards-compatibility (e.g., with item presence). ASN.1 supports multiple encoding rules, and while it's possible that someone could map ASN.1 syntax to protobuf encodings, it would only support a subset of ASN.1 because protobuf doesn't support length or value constraints (among other ASN.1 features). ASN.1 does have a little-used standard called Encoding Control Notation[0] that in principle supports the construction of novel encodings. But I have never seen a compiler, commercial or otherwise, that supports it. It requires a certain expressiveness in your parser that's hard to do right, although I've wondered if LISP or Racket could take it on. [0]: https://www.itu.int/rec/T-REC-X.692-202102-I https://www.itu.int/rec/T-REC-X.692-202102-I
bri3d 5y ago
ASN.1 struggles because the word "ASN.1" can name a lot of different implementations with different nuances, and a "complete" ASN.1 implementation is a massive and hazardous undertaking which has left many with a sour taste. Meanwhile, ProtoBufs and Thrift work off of more constrained and well-versioned interfaces. Honestly, ASN.1 with semantic versioning at the protocol level would probably have been as robust and useful as Protobufs. If ASN.1 had been forked into "ASN.1 3.0 without 10 hazardous and awful 1980s text encodings," it could even be fairly palatable today. Whether the overly expansive nature of ASN.1 is a product of the committee / standards organization design or the timeframe in which it originated is certainly an interesting philosophical question.
- jmspring 5y ago
  ASN.1 is a specification. DER/BER were methods to encode that specification.
- jnwatson 5y ago
  I used a subset of ASN.1 for a project, and it worked quite well. ASN.1 versioning in particular is a work of art.
- cryptonector 5y ago
  > Meanwhile, ProtoBufs and Thrift work off of more constrained and well-versioned interfaces. Not so. Protocol buffers is just a TLV encoding, which is bad (see elsewhere in this thread) -- it's just a cut-down ASN.1 and variation on BER, so what. ASN.1 can "well-version" everything just as well as anything else.
  - bri3d 5y ago
    If I have two "proto3" implementations using the same definitions, I trust they work together, generally speaking. If I have two ASN.1 BER implementations, I sadly can't really trust they work together, because I don't know what parts of "ASN.1" each one implemented.
cryptonector 5y ago
Protocol buffers is a tag-length-value encoding. It's got all the problems that DER and CER have. It's what happens when people decide to reinvent a wheel they don't understand.
- oblio 5y ago
  You should write a blogpost at this point. You can write more about these problems and it would have higher visibility.
  - cryptonector 5y ago
    Good point. I might, yes.
- feffe 5y ago
  What are the issues with TLV? I guess one could be that it's difficult to modify messages. On the other hand skipping parts of a message is efficient.
  - cryptonector 5y ago
    https://news.ycombinator.com/item?id=29937046 https://news.ycombinator.com/item?id=29937046

jwalton 5y ago

In terms of tooling, there’s excellent tooling for ASN.1 for C and C++ and maybe some other languages. There’s excellent tooling for protobufs for a handful of languages too, but they’re different sets, so in practice what languages you want to use would likely come into play.

breser 5y ago
How excellent the ASN.1 tooling is depends on which subset of ASN.1 you're using. Some of the tooling supports one iteration of ASN.1 or the other. To the degree that the IETF had to write a document on how to deal with this since some of the standards use the older ASN.1 and some use the newer ASN.1: https://tools.ietf.org/id/draft-ietf-pkix-asn1-translation-03.html https://tools.ietf.org/id/draft-ietf-pkix-asn1-translation-0... Interoperability with ASN.1 is very fragile at best.
- cryptonector 5y ago
  We have tons of interoperable PKIX implementations (OpenSSL and derivatives, NSS, OpenJDK's, GnuTLS, wolfSSL, Heimdal, and many many more), and a bunch of interoperable Kerberos implementations (MIT Kerberos, Heimdal, Windows / AD, OpenJDK's, the IBM Java's, GNU Shishi, there's a python implementation).
- cryptonector 5y ago
  BTW, that I-D is now RFC 6025 [0]. There's also RFC 5912 [1], which adds x.681/x.682/x.683 constraints to PKIX modules. I use this to great effect in Heimdal[2]. One function call can decode everything in a certificate, and a second can pretty print it in JSON; one command can pretty-print a certificate in all its glory in JSON. [0] https://datatracker.ietf.org/doc/html/rfc6025 [1] https://datatracker.ietf.org/doc/html/rfc5912 [2] https://github.com/heimdal/heimdal https://github.com/heimdal/heimdal/tree/master/lib/asn1
memling 5y ago
> In terms of tooling, there’s excellent tooling for ASN.1 for C and C++ and maybe some other languages. There’s excellent tooling for protobufs for a handful of languages too, but they’re different sets, so in practice what languages you want to use would likely come into play. In my experience, tooling is actually very good for most commonly-used languages, including C/C++, C#, Java, Python, and maybe even Go. And, of course, erlang. The real challenge is, I think, that you cannot find good free tooling, and the barrier to entry for Joe Developer is fairly high (in the thousands of dollars).
- dikei 5y ago
  I concur. Furthermore, the pricing if often opaque: you know the tools are expensive when they always want you to contact sales for a quote.

memling 5y ago

> Is this really the main difference between ASN.1 and Google protobufs, that one is managed by a private corporation and the other by a standardization organization? Can they otherwise be used "interchangably" in designing interfaces, a la two different programming languages (with different syntax of course)? No, the two are not interoperable and probably won't be made that way. Protobuf has undergone changes that challenge its backwards-compatibility (e.g., with item presence). ASN.1 supports multiple encoding rules, and while it's possible that someone could map ASN.1 syntax to protobuf encodings, it would only support a subset of ASN.1 because protobuf doesn't support length or value constraints (among other ASN.1 features). ASN.1 does have a little-used standard called Encoding Control Notation[0] that in principle supports the construction of novel encodings. But I have never seen a compiler, commercial or otherwise, that supports it. It requires a certain expressiveness in your parser that's hard to do right, although I've wondered if LISP or Racket could take it on. [0]: https://www.itu.int/rec/T-REC-X.692-202102-I https://www.itu.int/rec/T-REC-X.692-202102-I

bri3d 5y ago

ASN.1 struggles because the word "ASN.1" can name a lot of different implementations with different nuances, and a "complete" ASN.1 implementation is a massive and hazardous undertaking which has left many with a sour taste. Meanwhile, ProtoBufs and Thrift work off of more constrained and well-versioned interfaces. Honestly, ASN.1 with semantic versioning at the protocol level would probably have been as robust and useful as Protobufs. If ASN.1 had been forked into "ASN.1 3.0 without 10 hazardous and awful 1980s text encodings," it could even be fairly palatable today. Whether the overly expansive nature of ASN.1 is a product of the committee / standards organization design or the timeframe in which it originated is certainly an interesting philosophical question.

jmspring 5y ago
ASN.1 is a specification. DER/BER were methods to encode that specification.
jnwatson 5y ago
I used a subset of ASN.1 for a project, and it worked quite well. ASN.1 versioning in particular is a work of art.
cryptonector 5y ago
> Meanwhile, ProtoBufs and Thrift work off of more constrained and well-versioned interfaces. Not so. Protocol buffers is just a TLV encoding, which is bad (see elsewhere in this thread) -- it's just a cut-down ASN.1 and variation on BER, so what. ASN.1 can "well-version" everything just as well as anything else.
- bri3d 5y ago
  If I have two "proto3" implementations using the same definitions, I trust they work together, generally speaking. If I have two ASN.1 BER implementations, I sadly can't really trust they work together, because I don't know what parts of "ASN.1" each one implemented.

cryptonector 5y ago

Protocol buffers is a tag-length-value encoding. It's got all the problems that DER and CER have. It's what happens when people decide to reinvent a wheel they don't understand.

oblio 5y ago
You should write a blogpost at this point. You can write more about these problems and it would have higher visibility.
- cryptonector 5y ago
  Good point. I might, yes.
feffe 5y ago
What are the issues with TLV? I guess one could be that it's difficult to modify messages. On the other hand skipping parts of a message is efficient.
- cryptonector 5y ago
  https://news.ycombinator.com/item?id=29937046 https://news.ycombinator.com/item?id=29937046