6 ms·

And capitalisation issues, and encodings… give me a binary protocol to parse any time. They’re normally comparatively well-defined, whereas text protocols are s

And capitalisation issues, and encodings… give me a binary protocol to parse any time. They’re normally comparatively well-defined, whereas text protocols are seldom properly defined and so have undefined behaviour left, right and centre, which inevitably leads to security bugs—or even if they are well-defined, they’re probably done in a way that makes your language’s string type unsuitable, and makes parsing more complicated.

touisteur 5y ago
Just use recordflux for binary parser proved absent of runtime errors... ;-)
- einpoklum 5y ago
  Can you expand on that a little?
  - michaelmior 5y ago
    RecordFlux[0] is a DSL written in Ada for specifying messages in a binary protocol. Code for parsing these messages is then generated automatically with a number of useful properties automatically proven including that no runtime errors will occur. [0] https://github.com/Componolit/RecordFlux https://github.com/Componolit/RecordFlux
pjc50 5y ago
People say this, but then there's ASN.1 which has had critical security bugs in: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=asn.1 https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=asn.1 I will agree that exhaustively defining text protocols is extremely hard, starting from character set / encoding and getting worse from there.
- lmilcin 5y ago
  ASN.1 is not a binary protocol. It is a language to describe messages. Typically you create message description which is then compiled to code that can serialize/deserialize messages in BER-TLV or PER-TLV. I know because I wrote a complete parser/serializer for BER-TLV. It is simple protocol and any security issue is in the parser/serializer and not the protocol itself. That for simple reason that the protocol is nothing more than a format to serialize/deserialize the data.
  - lmz 5y ago
    True, but they certainly don't make it too easy either with the indefinite length constructs.
- admax88q 5y ago
  I guess if you're going to pick the absolute worst example of a binary protocol.
  - lmilcin 5y ago
    That is absolutely not fair. BER-TLV is really nice protocol. I have worked with it for couple of years when I worked on an EMV application. It uses BER-TLV to communicate with the credit card but it is also very convenient format for all sorts of other uses and I would use it wherever I could. Think of it as Json but in binary form. It is not complicated and I would not even bother parsing the messages -- I could interpret hex dumps of them by sight very easily.
scruple 5y ago
> give me a binary protocol to parse any time I don't disagree in principle but I've come across a handful of very poorly documentated binary protocols in my years and that is an extremely painful thing to deal with compared to text-based protocols.
- chrismorgan 5y ago
  Ugh, flashbacks. Now that you’ve mentioned that I feel the burning need to change my wording to add the condition “fairly described”. I have also come across very poorly documented binary protocols!
- jrockway 5y ago
  I think it's pretty easy to make poor use of HTTP to the same end. Imagine you're dumping traffic between a client and a server and the exchange is: GET /asdfasdfasdfdsaf X-Asdf-Asdf: 83e7234 HTTP/1.0 202 X-83e7233: 1 X-83f730b: 4 It's text, but you still have no idea what's going on. Overall, I think it's kind of a wash. It's basically equally easy to take a documented text or binary protocol and write a working parser. Neither format solves any intrinsic problems -- malicious input has to be accounted for, you have to write fuzz tests, you have to deal with broken producers. It's a wash. People like text because they can type it into telnet and see something work, which is kind of cool, but probably not a productive use of time. I can type HTTP messages, but use curl anyway. (SMTP was always my favorite. "HELO there / MAIL FROM foo / RCPT TO bar / email body / ." Felt like a real conversation going on. Still not sure how to send an email that consists of a single dot on its own line though.)
  - taejo 5y ago
    > Still not sure how to send an email that consists of a single dot on its own line though. A line starting with a period is escaped by adding an extra period; the receiving side removes the first character of a line if it is a period.
  - knome 5y ago
    >Still not sure how to send an email that consists of a single dot on its own line though. It seems you prefix any line of data starting with a period with an additional period, to therefore distinguish it from the end of mail period. https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.2 https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.2
the_arun 5y ago
>>* The author disclaims copyright to this source code. In place of >>* a legal notice, here is a blessing: >>* >>* May you do good and not evil. >>* May you find forgiveness for yourself and forgive others. >>* May you share freely, never taking more than you give. Above all, this note at the beginning of the source code impressed me.

touisteur 5y ago

Just use recordflux for binary parser proved absent of runtime errors... ;-)

einpoklum 5y ago
Can you expand on that a little?
- michaelmior 5y ago
  RecordFlux[0] is a DSL written in Ada for specifying messages in a binary protocol. Code for parsing these messages is then generated automatically with a number of useful properties automatically proven including that no runtime errors will occur. [0] https://github.com/Componolit/RecordFlux https://github.com/Componolit/RecordFlux

pjc50 5y ago

People say this, but then there's ASN.1 which has had critical security bugs in: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=asn.1 https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=asn.1 I will agree that exhaustively defining text protocols is extremely hard, starting from character set / encoding and getting worse from there.

lmilcin 5y ago
ASN.1 is not a binary protocol. It is a language to describe messages. Typically you create message description which is then compiled to code that can serialize/deserialize messages in BER-TLV or PER-TLV. I know because I wrote a complete parser/serializer for BER-TLV. It is simple protocol and any security issue is in the parser/serializer and not the protocol itself. That for simple reason that the protocol is nothing more than a format to serialize/deserialize the data.
- lmz 5y ago
  True, but they certainly don't make it too easy either with the indefinite length constructs.
admax88q 5y ago
I guess if you're going to pick the absolute worst example of a binary protocol.
- lmilcin 5y ago
  That is absolutely not fair. BER-TLV is really nice protocol. I have worked with it for couple of years when I worked on an EMV application. It uses BER-TLV to communicate with the credit card but it is also very convenient format for all sorts of other uses and I would use it wherever I could. Think of it as Json but in binary form. It is not complicated and I would not even bother parsing the messages -- I could interpret hex dumps of them by sight very easily.

scruple 5y ago

> give me a binary protocol to parse any time I don't disagree in principle but I've come across a handful of very poorly documentated binary protocols in my years and that is an extremely painful thing to deal with compared to text-based protocols.

chrismorgan 5y ago
Ugh, flashbacks. Now that you’ve mentioned that I feel the burning need to change my wording to add the condition “fairly described”. I have also come across very poorly documented binary protocols!
jrockway 5y ago
I think it's pretty easy to make poor use of HTTP to the same end. Imagine you're dumping traffic between a client and a server and the exchange is: GET /asdfasdfasdfdsaf X-Asdf-Asdf: 83e7234 HTTP/1.0 202 X-83e7233: 1 X-83f730b: 4 It's text, but you still have no idea what's going on. Overall, I think it's kind of a wash. It's basically equally easy to take a documented text or binary protocol and write a working parser. Neither format solves any intrinsic problems -- malicious input has to be accounted for, you have to write fuzz tests, you have to deal with broken producers. It's a wash. People like text because they can type it into telnet and see something work, which is kind of cool, but probably not a productive use of time. I can type HTTP messages, but use curl anyway. (SMTP was always my favorite. "HELO there / MAIL FROM foo / RCPT TO bar / email body / ." Felt like a real conversation going on. Still not sure how to send an email that consists of a single dot on its own line though.)
- taejo 5y ago
  > Still not sure how to send an email that consists of a single dot on its own line though. A line starting with a period is escaped by adding an extra period; the receiving side removes the first character of a line if it is a period.
- knome 5y ago
  >Still not sure how to send an email that consists of a single dot on its own line though. It seems you prefix any line of data starting with a period with an additional period, to therefore distinguish it from the end of mail period. https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.2 https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.2

the_arun 5y ago

>>* The author disclaims copyright to this source code. In place of >>* a legal notice, here is a blessing: >>* >>* May you do good and not evil. >>* May you find forgiveness for yourself and forgive others. >>* May you share freely, never taking more than you give. Above all, this note at the beginning of the source code impressed me.