4 ms·

Oh where to begin? ASN.1 really demands code generation. Unfortunately lots of nonconforming stuff has to be dealt with. The concept of encoding rules and the

by wbl 5y ago

Oh where to begin? ASN.1 really demands code generation. Unfortunately lots of nonconforming stuff has to be dealt with. The concept of encoding rules and the module tagging scheme make for a pretty big number of possible representations. The language semantics of ASN.1 don't really map to anything well, particularly around default fields and structures that can vary. Newer systems don't have encoding rules and pick a semantics that matches a target language much more closely.

wbl 5y ago
Also expect to pay to read the spec.
- memling 5y ago
  ASN.1 standards are free: https://www.itu.int/rec/t-rec-x.680/en https://www.itu.int/rec/t-rec-x.680/en Many, though not all, specifications that use ASN.1 are also freely available. I've been out of telecom for awhile, so I don't know the status of the newer standards, but when I was working in the business GSM MAP and MMS were the only proprietary ones that were an issue.
  - userbinator 5y ago
    GSM standards are also freely available --- look at 3gpp.org or etsi.org --- the biggest problem is finding which ones actually contain what you're looking for.
- cryptonector 5y ago
  The ITU-T ASN.1 specs have been free for a very long time now. They used to be non-free, and that was a big problem with ASN.1, but that was decades ago.
AceJohnny2 5y ago
> ASN.1 really demands code generation. On this specific point: isn't this also the case for other high-performance serialisers? Google ProtoBufs, Apache Thrift, any protocol through Rust's SerDes...
- xyzzy_plugh 5y ago
  Not really. You can trivially encode or decode protobuf or thrift at runtime, given a message specification, and this isn't uncommon in the wild. It's just that you usually expect messages which are well-defined at build time, so why not generate code?
- wbl 5y ago
  There isn't in the wild nonconformant data you also need to live with out there for most of them. The combination is unholy.
- cryptonector 5y ago
  No, it's not. There is no reasonable syntax/IDL/schema/whatever you want to call it for which you wouldn't have a choice of implementing by code generation or by bytecode generation. How is that not obvious? It would be like saying that "the problem with LISP is that it has to be interpreted", or that "the problem with C is that it can only be compiled to object code", when both such statements are clearly incorrect because of real-life counter-examples. But there is something special to ASN.1. Instead of seeing that there's nothing new under the Sun when it comes to data encoding and schemata, and that there hasn't been anything new in that field really since S-expressions, ASN.1 has engendered a special hatred that blinds everyone to things that they would grant as obvious in other cases.
cryptonector 5y ago
> ASN.1 really demands code generation. Nope, nyet, bzzt. Proofs by counter-example: - OpenLDAP has a printf/scanf-like approach to BER encoding - Heimdal has an ASN.1 compiler that generates code, yes, but also alternatively generates bytecode that gets interpreted at run-time. > The language semantics of ASN.1 don't really map to anything well, particularly around default fields and structures that can vary. You are ill-informed. Proof by counter-example: - there are ASN.1 encoding rules that produce natural XML (XER) and JSON (JER) - "default fields" are supported (the relevant keyword is `DEFAULT`, naturally) - "structures that can vary" -- if you mean unions, it's got that (the relevant keyword is `CHOICE`), and if you mean "extensions", it's got extensibility markers (that effectively are alike a CHOICE of an octet string of unknown stuff, or else the extensions known at module compile time.
- wbl 5y ago
  I have worked on code that took the OpenLDAP approach. It sucked, guiding to partial parsing and processing. The rest of your question misunderstands the nature of semantics I'm talking about. It's not that we can't make XML or JSON it's that programming languages often don't have types that map naturally to all of ASN.1 default not nil doesn't work in Go for example.
  - cryptonector 5y ago
    Oh, I agree. I don't like the printf/scanf-like approach to BER encoding. In fact, it's awful. The point I was making is that code generation is not the only option for ASN.1 or any encoding. Also, ASN.1 types map very well onto C (surprise): - OCTET STRING -> struct with pointer and length in bytes - BIT STRING -> struct with pointer and length in bits - INTEGER (constrained) -> some stdint.h integer type - INTEGER (unconstrained) -> struct with pointer to array of uint64_t, array element count, and boolean to indicate if signed or unsigned - REAL -> double or some arbitrary precision real library's type - most string types -> pointer to array of char, or counted byte string type - SEQUENCE OF and SET OF -> struct with pointer to array and count of elements - SEQUENCE and SET -> struct - CHOICE -> struct with discriminant enum and union of alternatives - tags -> ignore - OPTIONAL -> pointer - DEFAULT -> nothing special - NULL -> int (whatever) - BOOLEAN -> unsigned int, bool, maybe a bitfield of unsigned integer type so that all booleans can be compressed, etc. - OBJECT IDENTIFIER and RELATIVE OBJECT IDENTIFIER -> struct with pointer to DER encoding, and length in bytes - extensibility markers -> [hard to make this pithy, but it can be handled just fine] That covers like 99% of it. Suffice it to say that there's a very natural mapping of most of ASN.1 onto C. Things like classes and object sets aren't types but can guide the tooling to provide automatic encoding and decoding through open types (typed holes). BTW, `SET` is silly. `SET OF` is only of interest if you have arrays where order doesn't matter and you want a canonical encoding, but since one should not depend on canonical encodings, `SET OF` is also silly. IMO both should be deprecated (they can't be removed, but hey).

by wbl 5y ago

wbl 5y ago

Also expect to pay to read the spec.

memling 5y ago
ASN.1 standards are free: https://www.itu.int/rec/t-rec-x.680/en https://www.itu.int/rec/t-rec-x.680/en Many, though not all, specifications that use ASN.1 are also freely available. I've been out of telecom for awhile, so I don't know the status of the newer standards, but when I was working in the business GSM MAP and MMS were the only proprietary ones that were an issue.
- userbinator 5y ago
  GSM standards are also freely available --- look at 3gpp.org or etsi.org --- the biggest problem is finding which ones actually contain what you're looking for.
cryptonector 5y ago
The ITU-T ASN.1 specs have been free for a very long time now. They used to be non-free, and that was a big problem with ASN.1, but that was decades ago.

AceJohnny2 5y ago

> ASN.1 really demands code generation. On this specific point: isn't this also the case for other high-performance serialisers? Google ProtoBufs, Apache Thrift, any protocol through Rust's SerDes...

xyzzy_plugh 5y ago
Not really. You can trivially encode or decode protobuf or thrift at runtime, given a message specification, and this isn't uncommon in the wild. It's just that you usually expect messages which are well-defined at build time, so why not generate code?
wbl 5y ago
There isn't in the wild nonconformant data you also need to live with out there for most of them. The combination is unholy.
cryptonector 5y ago
No, it's not. There is no reasonable syntax/IDL/schema/whatever you want to call it for which you wouldn't have a choice of implementing by code generation or by bytecode generation. How is that not obvious? It would be like saying that "the problem with LISP is that it has to be interpreted", or that "the problem with C is that it can only be compiled to object code", when both such statements are clearly incorrect because of real-life counter-examples. But there is something special to ASN.1. Instead of seeing that there's nothing new under the Sun when it comes to data encoding and schemata, and that there hasn't been anything new in that field really since S-expressions, ASN.1 has engendered a special hatred that blinds everyone to things that they would grant as obvious in other cases.

cryptonector 5y ago

> ASN.1 really demands code generation. Nope, nyet, bzzt. Proofs by counter-example: - OpenLDAP has a printf/scanf-like approach to BER encoding - Heimdal has an ASN.1 compiler that generates code, yes, but also alternatively generates bytecode that gets interpreted at run-time. > The language semantics of ASN.1 don't really map to anything well, particularly around default fields and structures that can vary. You are ill-informed. Proof by counter-example: - there are ASN.1 encoding rules that produce natural XML (XER) and JSON (JER) - "default fields" are supported (the relevant keyword is `DEFAULT`, naturally) - "structures that can vary" -- if you mean unions, it's got that (the relevant keyword is `CHOICE`), and if you mean "extensions", it's got extensibility markers (that effectively are alike a CHOICE of an octet string of unknown stuff, or else the extensions known at module compile time.

wbl 5y ago
I have worked on code that took the OpenLDAP approach. It sucked, guiding to partial parsing and processing. The rest of your question misunderstands the nature of semantics I'm talking about. It's not that we can't make XML or JSON it's that programming languages often don't have types that map naturally to all of ASN.1 default not nil doesn't work in Go for example.
- cryptonector 5y ago
  Oh, I agree. I don't like the printf/scanf-like approach to BER encoding. In fact, it's awful. The point I was making is that code generation is not the only option for ASN.1 or any encoding. Also, ASN.1 types map very well onto C (surprise): - OCTET STRING -> struct with pointer and length in bytes - BIT STRING -> struct with pointer and length in bits - INTEGER (constrained) -> some stdint.h integer type - INTEGER (unconstrained) -> struct with pointer to array of uint64_t, array element count, and boolean to indicate if signed or unsigned - REAL -> double or some arbitrary precision real library's type - most string types -> pointer to array of char, or counted byte string type - SEQUENCE OF and SET OF -> struct with pointer to array and count of elements - SEQUENCE and SET -> struct - CHOICE -> struct with discriminant enum and union of alternatives - tags -> ignore - OPTIONAL -> pointer - DEFAULT -> nothing special - NULL -> int (whatever) - BOOLEAN -> unsigned int, bool, maybe a bitfield of unsigned integer type so that all booleans can be compressed, etc. - OBJECT IDENTIFIER and RELATIVE OBJECT IDENTIFIER -> struct with pointer to DER encoding, and length in bytes - extensibility markers -> [hard to make this pithy, but it can be handled just fine] That covers like 99% of it. Suffice it to say that there's a very natural mapping of most of ASN.1 onto C. Things like classes and object sets aren't types but can guide the tooling to provide automatic encoding and decoding through open types (typed holes). BTW, `SET` is silly. `SET OF` is only of interest if you have arrays where order doesn't matter and you want a canonical encoding, but since one should not depend on canonical encodings, `SET OF` is also silly. IMO both should be deprecated (they can't be removed, but hey).