3 ms·
> The reason most formats don't length-prefix everything is because it makes it costly to encode in both time and space. Yeah this is true, except for BSON bec
by IshKebab 7y ago
> The reason most formats don't length-prefix everything is because it makes it costly to encode in both time and space.
Yeah this is true, except for BSON because it uses fixed-size length prefixes, so you can just go back and fill them in later. Presumably that's why they used fixed-size lengths. The downsides are it is less space efficient and limited to 2GB.
In any case Amazon make the very good point that formats are read more often than they are written. It makes sense to optimise for the read case.
> Embedded processors tended to be big-endian, like older PowerPC and older ARM.
Nobody uses PowerPC anymore, and ARM hasn't been big endian for ages. Also MessagePack isn't designed for embedded systems and it still uses big endian. I don't think that's the reason. I suspect it's from a misguided belief that "network byte order" still matters.
And I totally agree, a schema-based format makes way more sense for my use case - changing is difficult though.