3 ms·

I don’t really get the advantage of this over either json (for human readability), protobuf (for type safety and schemas at the parsing layer), etc. it seems li

by lukevp 2y ago

I don’t really get the advantage of this over either json (for human readability), protobuf (for type safety and schemas at the parsing layer), etc. it seems like it mostly has disadvantages and would be difficult to implement efficiently because there are not consistent characters to tokenize.

jitl 2y ago
It’s binary and there’s no code generating step like with Protobuf. Protobuf is hugely complicated, I can see using this for IPC between processes I own. I’m not sure what you mean about tokenization, but printf format strings are pretty well understood.
- klodolph 2y ago
  I read the page and missed the part where it’s binary. I had to read it a second time to find that information. The reason why I missed it the first time is because this information is introduced in the second half of a paragraph, where the first half is about a different topic. This is a bad way to divide paragraphs.
  - deleted 2y ago
    [deleted]
  - MobiusHorizons 2y ago
    The paragraph: > The encoding/decoding is done with printf/scanf like functions with a format string and a variable number of arguments. However, no actual string formatting is done, the payload is a binary representation of arguments. Seems to encapsulate the complexity of having a string formatting interface to a binary protocol reasonably clearly, but is not crafted to withstand speed reading or skipping over sentences.
packetlost 2y ago
It's far, far simpler, than Protobuf, more efficient than JSON, and uses existing infrastructure that many people will be at least loosely familiar with. I think it's a pretty good idea!
- shadowgovt 2y ago
  Key things it lacks that protobuf offers: - bindings to other languages: I'm seeing C/C++ and Python supported. I could, hypothetically, use anything with a C-native interface to wrap that C library and bind to other languages, but that's work I'll have to do. And that binding is going to be as easy-to-use as a wrapper around a C-native library. - versioning (or some other way to deal with drift between serialization and deserialization). In practice, protocol buffers are often used to do both long-term storage serialization and short-term on-wire transmission. Both use cases always eventually involve two services that cannot be kept in lockstep sync on releases (either because you have to read older files or because you needed to release your backend before your frontend, do a rolling release, whatever). This is simpler than protobuf, which is cool (and I'd recommend looking over its implementation to figure out how to write something like this), but if you put it into implementation in a non-toy distributed-system project it will not scale. It may have other applications though; I can definitely see it as the backbone for a simple videogame messaging protocol. Still, no harm in writing a thing and putting it out there; I'm happy it exists even if I won't use it.
  - packetlost 2y ago
    > but if you put it into implementation in a non-toy distributed-system project it will not scale Plenty of distributed systems don't have the requirements you outline or need to scale. Distributed systems are not necessarily large, cross organizational lines, or require rolling releases. Sometimes the tradeoff in complexity by, for example, introducing a dependency on protoc et. al., is not worth it. I've built several systems that this would've been a perfectly fine choice for. I'd also point out that because it's so simple, adding implementations in new languages should be fairly straightforward, but I think this really shines in C codebases where complexity must be mostly dealt with by hand (for better or worse).
  - deleted 2y ago
    [deleted]
dhagrow 2y ago
Seems like a better comparison would be any schemaless binary format, like msgpack, bson, or the many others. This adds some basic type validation, which you would have to do with a separate library, like msgspec if you're using Python.
0xbadcafebee 2y ago
It's intended to be a barebones IPC between 2 processes. It's not for Google to use for 100-gigabit control planes, or for web development. (But fwiw, if you did want to inspect this traffic as human-readable, it would be insanely easy to create either a packet sniffer for this or add support to tcpdump/wireguard. It's just dumping a printf format code and some raw data onto the wire, so you just pick up a new frame, read the format code, snip the data from the frame, pass it to printf() with the format code and the data. It's like 12 lines of code, no library needed other than libc. As a hacker I find this way more exciting than using a bloated standard to pass utf-8 strings)

by lukevp 2y ago

jitl 2y ago

It’s binary and there’s no code generating step like with Protobuf. Protobuf is hugely complicated, I can see using this for IPC between processes I own. I’m not sure what you mean about tokenization, but printf format strings are pretty well understood.

klodolph 2y ago
I read the page and missed the part where it’s binary. I had to read it a second time to find that information. The reason why I missed it the first time is because this information is introduced in the second half of a paragraph, where the first half is about a different topic. This is a bad way to divide paragraphs.
- deleted 2y ago
  [deleted]
- MobiusHorizons 2y ago
  The paragraph: > The encoding/decoding is done with printf/scanf like functions with a format string and a variable number of arguments. However, no actual string formatting is done, the payload is a binary representation of arguments. Seems to encapsulate the complexity of having a string formatting interface to a binary protocol reasonably clearly, but is not crafted to withstand speed reading or skipping over sentences.

packetlost 2y ago

It's far, far simpler, than Protobuf, more efficient than JSON, and uses existing infrastructure that many people will be at least loosely familiar with. I think it's a pretty good idea!

shadowgovt 2y ago
Key things it lacks that protobuf offers: - bindings to other languages: I'm seeing C/C++ and Python supported. I could, hypothetically, use anything with a C-native interface to wrap that C library and bind to other languages, but that's work I'll have to do. And that binding is going to be as easy-to-use as a wrapper around a C-native library. - versioning (or some other way to deal with drift between serialization and deserialization). In practice, protocol buffers are often used to do both long-term storage serialization and short-term on-wire transmission. Both use cases always eventually involve two services that cannot be kept in lockstep sync on releases (either because you have to read older files or because you needed to release your backend before your frontend, do a rolling release, whatever). This is simpler than protobuf, which is cool (and I'd recommend looking over its implementation to figure out how to write something like this), but if you put it into implementation in a non-toy distributed-system project it will not scale. It may have other applications though; I can definitely see it as the backbone for a simple videogame messaging protocol. Still, no harm in writing a thing and putting it out there; I'm happy it exists even if I won't use it.
- packetlost 2y ago
  > but if you put it into implementation in a non-toy distributed-system project it will not scale Plenty of distributed systems don't have the requirements you outline or need to scale. Distributed systems are not necessarily large, cross organizational lines, or require rolling releases. Sometimes the tradeoff in complexity by, for example, introducing a dependency on protoc et. al., is not worth it. I've built several systems that this would've been a perfectly fine choice for. I'd also point out that because it's so simple, adding implementations in new languages should be fairly straightforward, but I think this really shines in C codebases where complexity must be mostly dealt with by hand (for better or worse).
- deleted 2y ago
  [deleted]

dhagrow 2y ago

Seems like a better comparison would be any schemaless binary format, like msgpack, bson, or the many others. This adds some basic type validation, which you would have to do with a separate library, like msgspec if you're using Python.

0xbadcafebee 2y ago

It's intended to be a barebones IPC between 2 processes. It's not for Google to use for 100-gigabit control planes, or for web development. (But fwiw, if you did want to inspect this traffic as human-readable, it would be insanely easy to create either a packet sniffer for this or add support to tcpdump/wireguard. It's just dumping a printf format code and some raw data onto the wire, so you just pick up a new frame, read the format code, snip the data from the frame, pass it to printf() with the format code and the data. It's like 12 lines of code, no library needed other than libc. As a hacker I find this way more exciting than using a bloated standard to pass utf-8 strings)