7 ms·
> didn’t find any standard for separating protobuf messages The fact that protobufs are not self-delimiting is an endless source of frustration, but I know of
by NavinF 3y ago
> didn’t find any standard for separating protobuf messages
The fact that protobufs are not self-delimiting is an endless source of frustration, but I know of 2 standards for doing this:
- SerializeDelimited* is part of the protobuf library: https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/util/delimited_message_util.cc https://github.com/protocolbuffers/protobuf/blob/main/src/go...
- Riegeli is "a file format for storing a sequence of string records, typically serialized protocol buffers. It supports dense compression, fast decoding, seeking, detection and optional skipping of data corruption, filtering of proto message fields for even faster decoding, and parallel encoding": https://github.com/google/riegeli https://github.com/google/riegeli
- harvie 3y agohttps://msgpack.org/ https://msgpack.org/
- vore 3y agoHuh? This is an entirely different serialization format altogether.
- twic 3y agoCreate a MessagePack message containing a byte array, put the protobuf in that. Of course, the enterprise solution would be to hex encode it and put it in a CDATA section in an XML document.
- formerly_proven 3y agoJust stick this in front of your frames: 81 A2 70 62 C6 <32 bit be length> <protobuf data>
- KMag 3y agoYou got me. What's that format / magic number? At first I thought it was the zlib header for incompressible data, but that's not right. Is that just the MAC address of some NIC you own?
- aidenn0 3y agoIt's msgpack: 0x8n A map containing "n" key/value pairs -> 0x81 introduces a single key/value pair b101XXXXX : A string where XXXXX is the string length -> 0xA2 introudces a 2 byte string 0xC6 followed by a 32-bit value N introduces an array of bytes of size N 0x70 0x62 "pb" So it roughly corresponds to {"pb": <protobuf data> }
- paulddraper 3y agoPoster is being cute.
- retSava 3y agoDoesn't have a Brainf*k implementation, disqualified. /s