5 ms·
Whenever I read about protocol buffers, it reminds me of Microsoft's OLE libraries. IIRC, MS Word's file format is (or was) a bunch of OLE streams.
by criddell 6y ago
Whenever I read about protocol buffers, it reminds me of Microsoft's OLE libraries. IIRC, MS Word's file format is (or was) a bunch of OLE streams.
- pjmlp 6y agoNow it just gets serialized as a bunch of XML files. However COM is pretty much alive and is the foundation of UWP.
- tasogare 6y agoPretty much alive and UWP, chose one.
- mumblemumble 6y agoYou may be thinking of the Compound File Binary Format[1], which is used by OLE applications. OLE itself is an API mechanism, not a serialization format. So when an application that speaks OLE finds a chunk in a compound file that needs to be handled by another application, it would use OLE to go ask that application for help. Compound formats, in and of themselves, aren't really all that special. One could make a case that this is a common use case for fairly prosaic technologies like XML and MIME, too, for example. [1]: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-cfb/53989ce4-7b05-4f8d-829b-d08d6148375b https://docs.microsoft.com/en-us/openspecs/windows_protocols...
- pantulis 6y agoI reminds me of CORBA's IDL.
- dekhn 6y agoAnd Sun XDR.
- cryptonector 6y agoMS-RPC NDR IDL, ONC/RPC XDR, ASN.1, and many, many others. There aren't too many variations. You can have "self-describing" formats, which is generally short for tag-length-value (TLV) formats where the tag identifies the type of the encoded value, and the value is an octet string of the given length encoding a value of the given type tag. This includes ASN.1's BER/DER/CER. Protobufs is a TLV encoding. Or you can have a more "binary" format with no tags and lengths elided for fields with fixed-sized encodings or fields whose variable-length value encodings are self-describing as to length (e.g., various varint formats). This includes NDR, XDR, ASN.1's PER and OER. Flatbuffers is a variation on this theme. Or you can have textual encodings like XML and JSON. There's not a lot of other options. Each has its pros and cons, but, really, the packed encoding type (NDR, XDR, PER, OER, flatbuffers) is generally superior. That's because TLV schemes are redundant and therefore bloated, and because you really need to have the schema at hand (or have code generated from the schema, which is a different way of having the schema at hand) no matter what, so "self-describing" just isn't terribly useful. If you've ever seen the output of "asn1print" type programs you'll understand that you still need to know the schema in your head to understand the output, so might as well have a dumper/pretty-printer that can use a schema to really pretty-print... Textual encodings generally end up getting binary / optimized variants. E.g., XML has FastInfoSet (which is essentially a transcoding to ASN.1 with PER!). JSON has many binary formats, including several "JSONB" schemes (e.g., PostgreSQL's), and CBOR. This is because at the end of the day, textual formats are hard to parse -- they really suck. No one needs to build a SIMD parser for NDR/XDR/PER/OER/flatbuffers/CBOR/JSONB, but JSON is so hard to parse quickly that SIMD parsers for it exist. Besides the many standards, I've seen quite a few ad-hoc schemes, such as Lustre's RPC format (basically flatbuffers, but without an IDL compiler and code generator), many Perl5 DataDumper-type modules, Lisp S-expressions (the grand-daddy of data encoding formats!), and many others. In a few years we might see a new round of bad reinventions of this wheel. This happens because inexperienced people find that scheme X "sucks" or there's no tooling for scheme X in the hot new programming language yet, and rather than read the specs for X and implement their own tooling they go off and make up a new scheme as they implement it. ASN.1 gets a lot of hate, but mostly the issue is with the TLV-type encoding rules, but there are many more encoding rules than just BER (and the DER and CER variants of BER): there's binary-style non-TLV ERs like PER and OER, there's XER (XML!), JER (JSON!), and even GSER (a textual encoding just for LDAP), and anyone can create new encoding rules if they really want to. Another reason ASN.1 gets hated on is lack of tooling, but every time anyone invents a new scheme... they have to write new tooling anyways -- what a waste. Note: I'm not saying we should only be using ASN.1 or anything of the sort. I'm saying we need to really understand what came before so we don't make the same mistakes again. Most importantly we need to understand that the T and often the L in TLV are just a lame crutch that bloats data on the wire and makes it easy to hand-roll bug-riddled codecs.