4 ms·
It's 2026 and I'm still defining my own messaging and wire protocols. Plain C structs that fit in a UDP datagram that you can reinterpret_cast from is still be
by mgaunard 6mo ago
It's 2026 and I'm still defining my own messaging and wire protocols.
Plain C structs that fit in a UDP datagram that you can reinterpret_cast from is still best. You can still provide schemas and UUIDs for that, and dynamically transcode to JSON or whatever.
- benterix 6mo agoIf you decide to use UDP, do you ignore the transmission errors or write the handling layer on your own?
- mgaunard 6mo agoI handle it in different ways by topic. For topics which are sending the state of something, a gap naturally self-recovers so long as you keep sending the state even if it doesn't change. For message buses that need to be incremental, you need to have a separate snapshot system to recover state. That's usually pretty rare outside of things like order books (I work in low-latency trading). For requests/response, I find it's better to tell the requester their request was not received rather than transparently re-send it, since by the time you re-send it it might be stale already. So what I do at the protocol level is just have ack logic, but no retransmit. Also it's datagram-oriented rather than byte-oriented, so overall much nicer guarantees than TCP (so long as all your messages fit in one UDP payload).
- pjc50 6mo agoProvided that: - you agree never to care about endianness (can probably get away with this now) - you don't want to represent anything complicated or variable length, including strings
- codedokode 6mo agoYou can have strings by using relative pointers ("string starts 123 bytes before this").
- mgaunard 6mo agoYou can also just use an array which sets a max capacity, and either use a null-terminator or a separate size field. In practice you probably want to have both, and choose what's most practical based on the message.
- bluGill 6mo agoUntil you have to work with big and little endian systems. There are other weirdness about how different computers represent things as well. utf-8 / ucs-16 strings (or other code pages). Not all floats are ieee-754. Still when you can ignore all those issues what you did is really easy and often works.
- codedokode 6mo agoI disagree. Big endian is long dead and not worth worrying about. And code pages too. What is more important, is dealing with schema changes, when you add new fields to requests and responses.
- bluGill 6mo agoThere are niches where those matter. but yes schema changes is most likely to get you today
- codedokode 6mo agoWhat you use is perfect for short-range communication (application and child process talking over shared memory), but not good for long-range communication (over Internet) because you can have old client talking to new version of a server, so you will have to add version numbers and have the code to parse outdated formats. But protobuf has compatibility built in and you do not need to write anything to support outdated clients. Also, protobuf uses solutions like varints to compress data to use less network traffic. So it is obviously made for long-range communication, and you probably do not have that and send 7 zeros for every small number. TL;DR protobuf has version compatibility and compact number encoding.
- mgaunard 6mo agoI already said you can UUIDs and schemas, and even dynamic conversion between mismatched schemas. Doing plain C structs doesn't prevent any of this.
- codedokode 6mo agoIt requires extra effort to write conversion algorithm for older data structure version.
- mgaunard 6mo agoThe converter is generated automatically based on the differences between the two schemas. Takes zero effort other than CPU cycles.