3 ms·

I’ve written hundreds of parsers at this point, for many kinds of protocols - http, mqtt, canbus, midi, various strange binary stuff, cli’s, just lots and lots

I’ve written hundreds of parsers at this point, for many kinds of protocols - http, mqtt, canbus, midi, various strange binary stuff, cli’s, just lots and lots of parsing. I’ve found that text protocols are usually easier to parse, depending on the situation. Some binary protocols are easy because you can just cast the read buffer to a struct, but even that can fail when doing a partial read (which is the norm more then the exception in embedded spaces). And even in the best of cases you usually need to do a lot of post-processing to make sure that the data you read is correct. In the worst of cases you’re bitshifting and you have conditionals everywhere and you’re just not going to have a fun time. Canbus and midi are a lot easier to write parsers for then e.g. protobuf and mqtt. For text, stuff like http, redis, and those types of protocols are easy, but the text parsing world has json, which is the absolute worst of pains.

IshKebab 6y ago
Canbus is binary isn't it? And Protobuf is very simple. In any case the problem is that it's easy to write a parser for text data that works for the data you have. It's extremely difficult to write a parser that exactly matches the spec and gets all the quoting/charset/etc. stuff right.
- friend-monoid 6y ago
  Late reply here but yes, canbus is binary, so is midi. They are both pretty simple. When I say that Protobuf is harder then canbus what I mean is that you have to deal with e.g. variable length integers (“varints”) and key offsets. And yes, that’s very true, full spec parsing is a pain, but that’s also a very rare special case. “In the real world” you usually don’t really care about full compatibility. If someone can find and send your printer data to get it to print securely and your customers can browse and login to your web page and your app, and you can send your Kafka messages to your order service, then everyone is happy, and none of those things require a full spec parser for anything.

IshKebab 6y ago

Canbus is binary isn't it? And Protobuf is very simple. In any case the problem is that it's easy to write a parser for text data that works for the data you have. It's extremely difficult to write a parser that exactly matches the spec and gets all the quoting/charset/etc. stuff right.

friend-monoid 6y ago
Late reply here but yes, canbus is binary, so is midi. They are both pretty simple. When I say that Protobuf is harder then canbus what I mean is that you have to deal with e.g. variable length integers (“varints”) and key offsets. And yes, that’s very true, full spec parsing is a pain, but that’s also a very rare special case. “In the real world” you usually don’t really care about full compatibility. If someone can find and send your printer data to get it to print securely and your customers can browse and login to your web page and your app, and you can send your Kafka messages to your order service, then everyone is happy, and none of those things require a full spec parser for anything.