4 ms·

For C structs, start with a uint32_t 4 char code identifier and a size field. Then you can extend them as long as you only ever append, which is a constraint o

For C structs, start with a uint32_t 4 char code identifier and a size field. Then you can extend them as long as you only ever append, which is a constraint on protobufs anyways. This is how most media containers have worked since the Amiga days. Meanwhile, their encoding/decoding is infinitely faster than most other serialization formats, allowing you to serialize data back and forth to disk or as IPC faster than anything else.

pjc50 7y ago
.. as long as you never change endianness, structure packing, word alignment requirements, and always use types with explicit sizes. And you have to keep the data in the structure itself, so as soon as you hit a pointer you need to write explicit serialisation.
- CoolGuySteve 7y ago
  > .. as long as you never change endianness, structure packing, word alignment requirements, and always use types with explicit sizes. #pragma pack(1) and byteswapping solve all this and neither are more work than specifying some intermediate format and calling some external tool in your build system. In C++ you can make types that swap automatically if required. > And you have to keep the data in the structure itself, so as soon as you hit a pointer you need to write explicit serialisation. I have yet to see a serialization format that does this well and quickly. Just make a new message type for it that follows the first.
- makapuf 7y ago
  Or juste use riff which useful and provide just enough to implement wav, bmp and other file formats (so you'll find heaps of examples(.
- Matthias247 7y ago
  Indeed. Using plain structs one can't just add a single field with a reasonable default and still be backward compatible - which is a very very common case in API design. You could add completely new revision of the API. But having to maintain it - typically additionally to the old revision since both sides might not updateable in atomic fashion - is a lot of extra work. Serialization which allows to add and remove fields in a backward compatible fashion is a mandatory requirement for any bigger sized project for me. If you need super high performance maybe use Flatbuffers or Cap'N'Proto instead of Protobuf/JSON/CBOR - but I really wouldn't recommend to go for plain structs.

pjc50 7y ago

.. as long as you never change endianness, structure packing, word alignment requirements, and always use types with explicit sizes. And you have to keep the data in the structure itself, so as soon as you hit a pointer you need to write explicit serialisation.

CoolGuySteve 7y ago
> .. as long as you never change endianness, structure packing, word alignment requirements, and always use types with explicit sizes. #pragma pack(1) and byteswapping solve all this and neither are more work than specifying some intermediate format and calling some external tool in your build system. In C++ you can make types that swap automatically if required. > And you have to keep the data in the structure itself, so as soon as you hit a pointer you need to write explicit serialisation. I have yet to see a serialization format that does this well and quickly. Just make a new message type for it that follows the first.
makapuf 7y ago
Or juste use riff which useful and provide just enough to implement wav, bmp and other file formats (so you'll find heaps of examples(.
Matthias247 7y ago
Indeed. Using plain structs one can't just add a single field with a reasonable default and still be backward compatible - which is a very very common case in API design. You could add completely new revision of the API. But having to maintain it - typically additionally to the old revision since both sides might not updateable in atomic fashion - is a lot of extra work. Serialization which allows to add and remove fields in a backward compatible fashion is a mandatory requirement for any bigger sized project for me. If you need super high performance maybe use Flatbuffers or Cap'N'Proto instead of Protobuf/JSON/CBOR - but I really wouldn't recommend to go for plain structs.