5 ms·
Mandatory comment about ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. Yes, it's a bit ugly but if you stick to the DER enc
by brabel 11mo ago
Mandatory comment about ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility.
Yes, it's a bit ugly but if you stick to the DER encoding it's really not worse than Protbuf at all. Check out the Wikipedia example:
https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_DER https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_DER
Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves.
- bloppe 11mo agoWhat makes it too complex in your opinion?
- zzo38computer 11mo agoI also think ASN.1 DER is better (there are other formats, but in my opinion, DER is the only good one, because BER is too messy). I use it in some of my stuff, and when I can, my new designs also use ASN.1 DER rather than using JSON and Protobuf etc. (Some types are missing from standard ASN.1 but I made up a variant called "ASN.1X" which adds some additional types such as key/value list and some others. With the key/value list type added, it is now a superset of the data model of JSON, so you can convert JSON to ASN.1X DER.) (I wrote a implementation of DER encoding/decoding in C, which is public domain and FOSS.)
- morshu9001 11mo agoASN.1 is way overengineered to the point of making it hard to support. You don't need inheritance for example.
- zzo38computer 11mo agoit is not necessary to use or to implement all of the data types and other features of ASN.1; you can implement only the features that you are using. Since DER uses the same framing for all data types, it is possible to skip past any fields that you do not care about (although in some cases you will still need to check its type, to determine whether or not an optional field is present; fortunately the type can be checked easily, even if it is not a type you implement).
- morshu9001 11mo agoYes but I don't want to worry about what parts of the spec are implemented on each end. If you removed all the unnecessary stuff and formed a new standard, it'd basically be protobuf.
- zzo38computer 11mo agoI do not agree. Which parts are necessary depends on the application; there is not one good way to do for everyone (and Protobuf is too limited). You will need to implement the parts specific to your schema/application on each end, and if the format does not have the data types that you want then you must add them in a more messy way (especially when using JSON).
- morshu9001 11mo agoIn what ASN1 application is protobuf spec too limited? I've used protobuf for tons of different things, it's always felt right. Though I understand certain encodings of ASN1 can have better performance for specific things.
- 11mo ago
- pphysch 11mo ago> ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. After working heavily with SNMP across a wide variety of OEMs, this flexibility becomes a downside. Or SNMP/MIBs were specified at the wrong abstraction level, where the ASN.1 flexibility gives mfgs too much power to do insane and unconventional things.
- morshu9001 11mo agoYeah same, ASN.1 was a nightmare when I was dealing with LTE
- cryptonector 11mo agoI've been working on and with Kerberos and PKIX for decades. I don't find ASN.1 to be a problem as long as you have good tooling or are willing to build it. The specs are a pleasure to read -- clear, concise, precise, and approachable (once you have a mental model for it anyways). Of course, I am an ASN.1 compiler maintainer, but hey, I had to become one because the compiler I was using was awesome but not good enough, so I made it good enough. I'm curious what made it a nightmare for you.
- whatevaa 11mo agoYou just said it. You had to become compiler maintainer to make it good enough.
- morshu9001 11mo agoThis was the main reason. The asn.1 language has a ton of unnecessary features that make it harder to implement, but the stuff I dealt with was using those features so I couldn't just ignore it. I didn't write a compiler but did hack around some asn1c outputted code to make it faster for our use case. And had to use asn1c in the first place because there was no complete Rust asn1 compiler at the time, though I tried DIY'ing it and gave up. I also remember it being complicated to use, but it's been too long to recall why exactly, probably the feature bloat. Once I used proto3, I realized it's all you need.
- dgan 11mo agoI honestly looked up for a encoder/decoder for python/c++ application, and couldnt find anything usable; i guess i would need to contact the purchase department for a license (?), while with protobuf i can make the decision myself & all alone
- strongpigeon 11mo ago> Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves. Yeah. I do remember a lot of workloads at Google where most of the CPU time was spent serializing/deserializing protos.
- ses1984 11mo agoI feel like most high throughput distributed systems eventually reach a point where some part of it is constrained by de/serialization. Not much is faster than protobuf except for zero copy formats.
- kragen 11mo agoBut zero-copy formats like FlatBuffers or Cap'n Proto can be much faster. Like, faster by an arbitrarily large factor, for data at rest.
- theamk 11mo agoASN.1 has too much stuff. The moment you write "I made ASN.1 decoder/encoder", someone will throw TeletexString or BMPString at it. Or inheritance, as morshu9001 sad. So at this point: - You can support all those features, and your ASM.1 library will be horribly bloated and over-engineered. - You can support your favorite subset, but then you cannot say it's ASN.1 anymore. It will be "ASN.brabel", which only has one implementation (yours). And who wants that? (unless you are Google and have immense developer influence... But in this case, why not design things from scratch, since we are making all-new protocol anyway?)
- themafia 11mo ago> someone will throw TeletexString or BMPString ASCII with escapes and UCS-2. > horribly bloated and over-engineered. It's no more or less complicated than XML, JSON or CSV. Which is why you can use ASN.1 to serialize to and from all these formats. ASN.1 provides you an additional level of schema above these. It simply allows you to describe your problem. I find ASN.1 far more sane and useful than something like JSON Schema which is just as "bloated and over-engineered." It turns out describing data is not a simple problem.
- kragen 11mo agoASN.1 is far, far more complicated than JSON or any particular flavor of CSV, in part because it does provide an extra level of schema that those other formats don't.
- theamk 11mo agoNope, TeletexString is ITU T.61, a.k.a codepage 1036. So Backspace (0x08) is OK, but Tab (0x09) is not. What, your implementation does not include CP1036 to Unicode translation table? Sorry, it's no longer ASN.1, it's now ASN.themafia. Oh, it does? Then how about xmlhstring vs hstring, are you handing difference properly? What about REAL type? Does your asn.1 library include support for arbitrary-precision floating point numbers, both base 2 and 10? And no, you cannot use "double", I am sure there is an application out there which uses base 10 reals, or has 128 bits of mantissa. ASN.1 is full of overengineered, ancient things like those, and the worst part - once you actually start using it to interoperate with other software, there is a good chance you'll see them. If you want something that people actually implement fully, choose something else.
- adastra22 11mo agoASN.1 is a nightmare, and I would never use it for a greenfield project.
- cryptonector 11mo agoI... wouldn't use it for a greenfield project either unless I got good at porting Luke Howard's Swift ASN.1 stack to whatever language I might be using that isn't C. For C I'd just use Heimdal's awesome ASN.1 compiler and be done. Even then I would be tempted to use flatbuffers instead, or else I'd have to go implement OER (a bunch of work I don't really care to do). The problem with ASN.1 -- the only real problem with ASN.1, is lack of excellent tooling.
- cryptonector 11mo agoThank you! Now I don't have to be the one saying this. Props if you use OER over DER. But since OP needs available tooling they might as well go to flatbuffers, which is much better than PB much like OER is much better than DER.
- IshKebab 11mo agoI don't see how you can seriously criticise Protobuf's very simple encoding scheme as being too complex while recommending ASN.1!! Totally mad.