5 ms·

I genuinely dislike CBOR. Formats which require me to calculate the length of the message before sending it lead to bad library design which often leads to eas

I genuinely dislike CBOR. Formats which require me to calculate the length of the message before sending it lead to bad library design which often leads to easily compromised code. Add in an "indefinite length" option and you've got potentially unbounded client memory usage to watch out for. As if that wasn't enough you get extensible tags so the meaning of any message is entirely dependent on the context it was sent in. It gives you a lot of decent protocol wire design and then flatly ignores everything we've learned about these types of designs in the last 3 decades. Be on the lookout for client libraries to slowly add in all of these checks as the vulnerabilities are discovered in them.

Netch 2y ago
> I genuinely dislike CBOR. Formats which require me to calculate the length of the message before sending it CBOR is not. No full message length. Only primitive type values. Compound type values require length _in items_ (not bytes) but indefinite encoding is possible if allowed ("deterministic" constraint to message forbids it). So technically this doesnʼt differ from e.g. JSON where you have to wait until final "}" or "]" for a long compound item, and have to install an artificial limit. > Add in an "indefinite length" option and you've got potentially unbounded client memory usage to watch out for. > As if that wasn't enough you get extensible tags so the meaning of any message is entirely dependent on the context it was sent in. So does any such a protocol. JSON, XML, any ASN.1, CBOR, whatever. If there are no extensible tags, there are field names - or they will be added by customer using what is allowed. Iʼve seen this in ASN.1 sequence-of pairs of name+value, directly emulating JSON-like dictionary. To limit it is not how security issues are handled.
rkeene2 2y ago
FWIW, ASN.1 BER supports an indefinite length encoding (though DER does not; ASN.1 JSON and ASN.1 XML do as well)
- akira2501 2y ago
  ASN suffers from some of the other problems regardless of the encoding. I was thinking about this more and I think FTP actually had a lot of the right ideas here. You want two channels. One for fast interactive command messaging and a second one coordinate especially just for bulk transfers arranged over the command channel. The design flaw in FTP of putting these on two separate ports put it immediately in conflict with firewall best practices so I think it went mostly unnoticed that it fundamentally is a good arrangement. What you want is one channel, sequenced packets, with a defined maximum message length that is negotiated at startup and never changes during the life of the channel. This should probably never be more than 65k. There should be a known length packet type, and an unknown length packet type, with any attempt to send more than the negotiated maximum triggering an error and disconnect. If you do need to send more than the negotiated amount you should open a new connection, in a bulk transfer mode, that after the initial handshake, has no protocol and is merely an associated stream of bytes that are wholly uninterpreted by the middleware layer other than to help you associate it with the sequenced packet that requested its initiation. You'd actually be able to use TCP (with DSCP even), mostly avoid head of line blocking and multiplexer latencies, and have a reasonable security guarantee in the sequenced packet mode, and never have a protocol which pretends that 4GB strings in the middle of a packet are necessary or even a good idea to "support." The downfall of this is that it would be much harder to implement it on a serverless architecture and would be nearly as complicated as a protocol as WebSocket ends up being. It might be worth playing with as a concept anyways.
  - rkeene2 2y ago
    I think the protocol on which PDUs are exchanged on is a bit orthogonal to the PDU protocol itself here, but the split-mode networking protocols (FTP, IRC DCC, mosh, SIP, etc) always do end up getting less used because establishing two different connections is a bigger burden and there really needs to be a great advantage to doing so, and often require other considerations (better connection tracking, TURN/ICE servers, etc). For a localhost/UNIX domain protocol, it might work since most of these considerations are significantly reduced. As to DSCP, it might be useful across controlled networks but in general, in my experience it's not really ever honored broadly. Back on the original topic, when I was building this for my Linux distribution I ended up just using basic JSON with TLS (for client certificate authentication) though since having authentication for remote management was a goal, and once I was already having to perform a TLS negotiation then the PDU consideration for performance wasn't something to really spend too much time on.
  - IgorPartola 2y ago
    You would need a pretty intense QoS system built for this if you are using something like TCP for this using two different connections. Underneath it would still use IPv4/6 packets and you only have so much bandwidth. The bulk transfer channel might stall the transfer of control packets temporarily. You would need to tightly control and multiplex the messages yourself, needing to do this in one connection or using something like TCP priority flag. Personally I just think that TCP is a shitty protocol for building applications. In almost every use case either UDP or SCTP are a better choice. With raw UDP datagrams you aren’t guaranteed delivery but it’s great for the kind of telemetry where the last message in is what matters. You can also build quite flexible stuff on top of it. SCTP gives you in sequence reliable datagrams and congestion control. This means you don’t have to devise message length communication into your application layer protocol and can rely on your transport. It also has multiplexing built right in. If it had a built in checksum it would truly be ideal. Not sure if something like secure communication really belongs at this level but if it had that I double we would ever use anything else. From SCTP’s Wikipedia page: > SCTP applications submit data for transmission in messages (groups of bytes) to the SCTP transport layer. SCTP places messages and control information into separate chunks (data chunks and control chunks), each identified by a chunk header. The protocol can fragment a message into multiple data chunks, but each data chunk contains data from only one user message. SCTP bundles the chunks into SCTP packets. The SCTP packet, which is submitted to the Internet Protocol, consists of a packet header, SCTP control chunks (when necessary), followed by SCTP data chunks (when available).
gjvc 2y ago
"lead to bad library design" how?
- akira2501 2y ago
  More complex buffer management in non-garbage collected languages. man 3 cmsg For a glimpse into the darker corners of hell. At least you get to grip it there. In GCed languages you have to hope the library designer gave you reasonable controls for timeouts, aborting large data transfers, and releasing buffers efficiently in a bursty network environment. See all the controls any HTTP server gives you for this. "Maximum header size", "Maximum header(s) size", "Maximum Body Size", "Request Header Timeout", "Total Request Timeout." None of those were added by default or by tasteful library authors. They were all added to answer to specific emergent security vulnerabilities that were discovered and then were demanded by the users.
XlA5vEKsMISoIln 2y ago
JSON's everything is "indefinite length". Also its implementations are wildly inconsistent [1] (is anyone going to read the code to figure out which is the blessed systemd parser?). Also it doesn't have integers. A lot of things with JSON will deteriorate into stringifying anyway, for example dates. [1]: https://seriot.ch/projects/parsing_json.html https://seriot.ch/projects/parsing_json.html
trissylegs 2y ago
CBOR does have indefinite length arrays, maps, byte strings and text strings : https://www.rfc-editor.org/rfc/rfc8949.html#name-indefinite-lengths-for-some https://www.rfc-editor.org/rfc/rfc8949.html#name-indefinite-...
Retr0id 2y ago
I'm a fan of DAG-CBOR, which is a deterministic subset of CBOR with most of the weird bits excluded (like indefinite lengths, multiple floating point formats, etc.), but it can still represent any sensible JSON object.

Netch 2y ago

> I genuinely dislike CBOR. Formats which require me to calculate the length of the message before sending it CBOR is not. No full message length. Only primitive type values. Compound type values require length _in items_ (not bytes) but indefinite encoding is possible if allowed ("deterministic" constraint to message forbids it). So technically this doesnʼt differ from e.g. JSON where you have to wait until final "}" or "]" for a long compound item, and have to install an artificial limit. > Add in an "indefinite length" option and you've got potentially unbounded client memory usage to watch out for. > As if that wasn't enough you get extensible tags so the meaning of any message is entirely dependent on the context it was sent in. So does any such a protocol. JSON, XML, any ASN.1, CBOR, whatever. If there are no extensible tags, there are field names - or they will be added by customer using what is allowed. Iʼve seen this in ASN.1 sequence-of pairs of name+value, directly emulating JSON-like dictionary. To limit it is not how security issues are handled.

rkeene2 2y ago

FWIW, ASN.1 BER supports an indefinite length encoding (though DER does not; ASN.1 JSON and ASN.1 XML do as well)

akira2501 2y ago
ASN suffers from some of the other problems regardless of the encoding. I was thinking about this more and I think FTP actually had a lot of the right ideas here. You want two channels. One for fast interactive command messaging and a second one coordinate especially just for bulk transfers arranged over the command channel. The design flaw in FTP of putting these on two separate ports put it immediately in conflict with firewall best practices so I think it went mostly unnoticed that it fundamentally is a good arrangement. What you want is one channel, sequenced packets, with a defined maximum message length that is negotiated at startup and never changes during the life of the channel. This should probably never be more than 65k. There should be a known length packet type, and an unknown length packet type, with any attempt to send more than the negotiated maximum triggering an error and disconnect. If you do need to send more than the negotiated amount you should open a new connection, in a bulk transfer mode, that after the initial handshake, has no protocol and is merely an associated stream of bytes that are wholly uninterpreted by the middleware layer other than to help you associate it with the sequenced packet that requested its initiation. You'd actually be able to use TCP (with DSCP even), mostly avoid head of line blocking and multiplexer latencies, and have a reasonable security guarantee in the sequenced packet mode, and never have a protocol which pretends that 4GB strings in the middle of a packet are necessary or even a good idea to "support." The downfall of this is that it would be much harder to implement it on a serverless architecture and would be nearly as complicated as a protocol as WebSocket ends up being. It might be worth playing with as a concept anyways.
- rkeene2 2y ago
  I think the protocol on which PDUs are exchanged on is a bit orthogonal to the PDU protocol itself here, but the split-mode networking protocols (FTP, IRC DCC, mosh, SIP, etc) always do end up getting less used because establishing two different connections is a bigger burden and there really needs to be a great advantage to doing so, and often require other considerations (better connection tracking, TURN/ICE servers, etc). For a localhost/UNIX domain protocol, it might work since most of these considerations are significantly reduced. As to DSCP, it might be useful across controlled networks but in general, in my experience it's not really ever honored broadly. Back on the original topic, when I was building this for my Linux distribution I ended up just using basic JSON with TLS (for client certificate authentication) though since having authentication for remote management was a goal, and once I was already having to perform a TLS negotiation then the PDU consideration for performance wasn't something to really spend too much time on.
- IgorPartola 2y ago
  You would need a pretty intense QoS system built for this if you are using something like TCP for this using two different connections. Underneath it would still use IPv4/6 packets and you only have so much bandwidth. The bulk transfer channel might stall the transfer of control packets temporarily. You would need to tightly control and multiplex the messages yourself, needing to do this in one connection or using something like TCP priority flag. Personally I just think that TCP is a shitty protocol for building applications. In almost every use case either UDP or SCTP are a better choice. With raw UDP datagrams you aren’t guaranteed delivery but it’s great for the kind of telemetry where the last message in is what matters. You can also build quite flexible stuff on top of it. SCTP gives you in sequence reliable datagrams and congestion control. This means you don’t have to devise message length communication into your application layer protocol and can rely on your transport. It also has multiplexing built right in. If it had a built in checksum it would truly be ideal. Not sure if something like secure communication really belongs at this level but if it had that I double we would ever use anything else. From SCTP’s Wikipedia page: > SCTP applications submit data for transmission in messages (groups of bytes) to the SCTP transport layer. SCTP places messages and control information into separate chunks (data chunks and control chunks), each identified by a chunk header. The protocol can fragment a message into multiple data chunks, but each data chunk contains data from only one user message. SCTP bundles the chunks into SCTP packets. The SCTP packet, which is submitted to the Internet Protocol, consists of a packet header, SCTP control chunks (when necessary), followed by SCTP data chunks (when available).

gjvc 2y ago

"lead to bad library design" how?

akira2501 2y ago
More complex buffer management in non-garbage collected languages. man 3 cmsg For a glimpse into the darker corners of hell. At least you get to grip it there. In GCed languages you have to hope the library designer gave you reasonable controls for timeouts, aborting large data transfers, and releasing buffers efficiently in a bursty network environment. See all the controls any HTTP server gives you for this. "Maximum header size", "Maximum header(s) size", "Maximum Body Size", "Request Header Timeout", "Total Request Timeout." None of those were added by default or by tasteful library authors. They were all added to answer to specific emergent security vulnerabilities that were discovered and then were demanded by the users.

XlA5vEKsMISoIln 2y ago

JSON's everything is "indefinite length". Also its implementations are wildly inconsistent [1] (is anyone going to read the code to figure out which is the blessed systemd parser?). Also it doesn't have integers. A lot of things with JSON will deteriorate into stringifying anyway, for example dates. [1]: https://seriot.ch/projects/parsing_json.html https://seriot.ch/projects/parsing_json.html

trissylegs 2y ago

CBOR does have indefinite length arrays, maps, byte strings and text strings : https://www.rfc-editor.org/rfc/rfc8949.html#name-indefinite-lengths-for-some https://www.rfc-editor.org/rfc/rfc8949.html#name-indefinite-...

Retr0id 2y ago

I'm a fan of DAG-CBOR, which is a deterministic subset of CBOR with most of the weird bits excluded (like indefinite lengths, multiple floating point formats, etc.), but it can still represent any sensible JSON object.