7 ms·

ASN.1 has support for ISO 2022 as well as ASCII and Unicode (ASCII is a subset of Unicode as well as a subset of ISO 2022). (My own nonstandard extensions add a

ASN.1 has support for ISO 2022 as well as ASCII and Unicode (ASCII is a subset of Unicode as well as a subset of ISO 2022). (My own nonstandard extensions add a few more (such as TRON character code and packed BCD), and the standard unrestricted character string type can be used if you really need arbitrary character sets.) (Unicode is not a very good character set, anyways.) Also, DER allows to indicate the type of data within the file (unless you are using implicit types). Protobuf has only a limited case of this (you cannot always identify the types), and it requires different framing for different types. However, DER uses the same framing for all types, and strings are not inherently limited to 2GB by the file format. Furthermore, there are other non-scalar types as well. In any of these cases, you do not have to use all of the types (nor do you need to implement all of the types); you only need to use the types that are applicable for your use. I will continue to use ASN.1; Protobuf is not good enough in my opinion.

cryptonector 10mo ago
To be fair, if you don't need to support anything other than Unicode, then this is not an advantage, and over time we're all going to need non-Unicode less and less. That said I'm a big fan of ASN.1 (see my comment history).
- morshu9001 10mo ago
  I'm still confused how these ISO 2022 strings even work, and the ASN1 docs discourage using the UniversalString and GraphicString types. All these different string types are intimidating if I just want unicode/ascii, and even if I were using an obscure encoding, I'd use generic bytes instead of wanting asn1 to care about it.
  - cryptonector 10mo ago
    GeneralString relies on control characters to "load" character sets into the C0 and C1 registers. This is madness -- specifically it's pre-Unicode madness, but before Unicode it made sense.
    - morshu9001 10mo ago
      Oh gosh. Fair enough that this exists and something uses it, but I'd absolutely want to handle that on the ends only, not get asn1 involved in parsing it.
      - cryptonector 10mo ago
        Oh GeneralString is madness. It's pre-Unicode madness. It exists because Unicode didn't exist in 1984, but people still wanted to be able to exchange text in multiple scripts, which necessitated being able to "switch codesets" in the middle. It's... yeah, it's.. it's nuts. I've _not_ implemented GeneralString, and practically no one needs to even when specs say to. E.g., in Kerberos the strings are GeneralString, but all the implementations just-send-8 and do not attempt to interpret any codeset switching escapes.
      - zzo38computer 10mo ago
        ASN.1 does not necessarily need to get involved in parsing the values; for some applications doing so is unnecessary anyways (this is true for many fields of many types and not only this one, though). ASN.1 will need to be involved in parsing the framing; whether or not it is involved in parsing the values depends on whether the application requires it for that specific value (for example, it is commonly not necessary to parse OIDs (you can usually just treat them as opaque data which can be compared for equality (or looked up in a table), although sometimes it is useful to display them), although some implementations insist on doing so anyways).
        cryptonector 10mo ago
        Correct, ASN.1 does not tell you how to implement all its semantics. You can totally have tooling that exposes a GeneralString's blob payload to the application and lets the app handle the codeset switching aspects of GeneralString. I want to object that anyone who really has to handle multi-codeset GeneralString values would want a better library but... ...what would that be if not a converter to/from Unicode? But what if one really wants an array/list of {codeset, string} pairs? At that point open-coding support for those escapes is probably just as well since that one might have the only application in the world that wants that!! :laugh:
  - zzo38computer 10mo ago
    > I'm still confused how these ISO 2022 strings even work There is C0, G0, C1, and G1 sets (C0 and C1 are control characters and G0 and G1 are graphic characters), and escape sequences are used to select the C or G set for bytes with or without the high bit set. Graphic string does not allow control characters and General string does allow control characters. You probably do not need all control characters; your schema should probably restrict which control characters are allowed in each context (although the ASN.1 schema format does not seem to have any way to do this). This way, you will only handle the control characters which are appropriate for your use. This is messy, although canonical form simplifies it by adding some restrictions (this is one of the reasons why DER is better than BER, in my opinion). TRON code is better and is much simpler than the working of ISO 2022. (Unicode has a different kind of mess; although decoding is simpler, actually handling the decoded characters in text is its own big mess for many reasons. Unicode is a stateful character set, even though the encoding is stateless; TRON code is the other way around (and with a significantly simpler stateful encoding than ISO 2022).) > the ASN1 docs discourage using the UniversalString and GraphicString types UniversalString is UTF-32BE and GraphicString is ISO 2022 without control characters. By knowing what they are, you should know in which circumstances they should be considered useful or not useful; I think that they should not be discouraged in general (although usually if you want Unicode, you would use UTF-8 rather than UTF-32, there are some circumstances where you might want to use UTF-32, such as if the data or program is already UTF-32 for other reasons). (The data type which probably should be avoided is the UTC time type, which is not Y2K compliant.) > All these different string types are intimidating if I just want unicode/ascii If you only want ASCII, use the IA5 type (or Visible if you do not want control characters); if you only want Unicode, use the UTF-8 string type (or Universal if you want UTF-32 instead for some reason). ("IA5" is another name for ASCII that as far as I can tell hardly anyone other than ITU uses.) However, Unicode is not a very good character set, and they should not force or expect you to use it. As I had mentioned before, you do not need to use or implement all of the ASN.1 data types; only use the ones appropriate for your application (so, if you do not like most of the types, then don't use those types). I also made up some additional nonstandard ASN.1 types (called ASN.1X), which also might be useful for some applications; you are not required to use or implement these either.
    - 10mo ago

cryptonector 10mo ago

To be fair, if you don't need to support anything other than Unicode, then this is not an advantage, and over time we're all going to need non-Unicode less and less. That said I'm a big fan of ASN.1 (see my comment history).

morshu9001 10mo ago
I'm still confused how these ISO 2022 strings even work, and the ASN1 docs discourage using the UniversalString and GraphicString types. All these different string types are intimidating if I just want unicode/ascii, and even if I were using an obscure encoding, I'd use generic bytes instead of wanting asn1 to care about it.
- cryptonector 10mo ago
  GeneralString relies on control characters to "load" character sets into the C0 and C1 registers. This is madness -- specifically it's pre-Unicode madness, but before Unicode it made sense.
  - morshu9001 10mo ago
    Oh gosh. Fair enough that this exists and something uses it, but I'd absolutely want to handle that on the ends only, not get asn1 involved in parsing it.
    - cryptonector 10mo ago
      Oh GeneralString is madness. It's pre-Unicode madness. It exists because Unicode didn't exist in 1984, but people still wanted to be able to exchange text in multiple scripts, which necessitated being able to "switch codesets" in the middle. It's... yeah, it's.. it's nuts. I've _not_ implemented GeneralString, and practically no one needs to even when specs say to. E.g., in Kerberos the strings are GeneralString, but all the implementations just-send-8 and do not attempt to interpret any codeset switching escapes.
    - zzo38computer 10mo ago
      ASN.1 does not necessarily need to get involved in parsing the values; for some applications doing so is unnecessary anyways (this is true for many fields of many types and not only this one, though). ASN.1 will need to be involved in parsing the framing; whether or not it is involved in parsing the values depends on whether the application requires it for that specific value (for example, it is commonly not necessary to parse OIDs (you can usually just treat them as opaque data which can be compared for equality (or looked up in a table), although sometimes it is useful to display them), although some implementations insist on doing so anyways).
      - cryptonector 10mo ago
        Correct, ASN.1 does not tell you how to implement all its semantics. You can totally have tooling that exposes a GeneralString's blob payload to the application and lets the app handle the codeset switching aspects of GeneralString. I want to object that anyone who really has to handle multi-codeset GeneralString values would want a better library but... ...what would that be if not a converter to/from Unicode? But what if one really wants an array/list of {codeset, string} pairs? At that point open-coding support for those escapes is probably just as well since that one might have the only application in the world that wants that!! :laugh:
- zzo38computer 10mo ago
  > I'm still confused how these ISO 2022 strings even work There is C0, G0, C1, and G1 sets (C0 and C1 are control characters and G0 and G1 are graphic characters), and escape sequences are used to select the C or G set for bytes with or without the high bit set. Graphic string does not allow control characters and General string does allow control characters. You probably do not need all control characters; your schema should probably restrict which control characters are allowed in each context (although the ASN.1 schema format does not seem to have any way to do this). This way, you will only handle the control characters which are appropriate for your use. This is messy, although canonical form simplifies it by adding some restrictions (this is one of the reasons why DER is better than BER, in my opinion). TRON code is better and is much simpler than the working of ISO 2022. (Unicode has a different kind of mess; although decoding is simpler, actually handling the decoded characters in text is its own big mess for many reasons. Unicode is a stateful character set, even though the encoding is stateless; TRON code is the other way around (and with a significantly simpler stateful encoding than ISO 2022).) > the ASN1 docs discourage using the UniversalString and GraphicString types UniversalString is UTF-32BE and GraphicString is ISO 2022 without control characters. By knowing what they are, you should know in which circumstances they should be considered useful or not useful; I think that they should not be discouraged in general (although usually if you want Unicode, you would use UTF-8 rather than UTF-32, there are some circumstances where you might want to use UTF-32, such as if the data or program is already UTF-32 for other reasons). (The data type which probably should be avoided is the UTC time type, which is not Y2K compliant.) > All these different string types are intimidating if I just want unicode/ascii If you only want ASCII, use the IA5 type (or Visible if you do not want control characters); if you only want Unicode, use the UTF-8 string type (or Universal if you want UTF-32 instead for some reason). ("IA5" is another name for ASCII that as far as I can tell hardly anyone other than ITU uses.) However, Unicode is not a very good character set, and they should not force or expect you to use it. As I had mentioned before, you do not need to use or implement all of the ASN.1 data types; only use the ones appropriate for your application (so, if you do not like most of the types, then don't use those types). I also made up some additional nonstandard ASN.1 types (called ASN.1X), which also might be useful for some applications; you are not required to use or implement these either.
  - 10mo ago