3 ms·
You've basically got two ways of handling this: Using a delimiter to mark end of packet. For text protocols we usually use "\n" to be the delimiter. This usual
by rkangel 3y ago
You've basically got two ways of handling this:
Using a delimiter to mark end of packet. For text protocols we usually use "\n" to be the delimiter. This usually requires some escaping in the packet to make it unambiguous. Two standard protocols for this are SLIP and HDLC.
Length encoding - like what you did.
The downside of the delimiter approach is that it changes the length of the packet - when you are escaping one byte becomes two and that's sometimes a pain if you're doing it in place in memory (less of a problem if you're streaming byte by byte). The big advantage is that it allows for resynchronisation - if you lose a single byte from your stream, or your length byte ever gets corrupted then you're permanently out of sync - the receiver will never again know where the start or end of a packet is. With the delimiter approach, you just lose one packet. So if you're ever doing this for a UART or network stream or something, always do the delimiter approach!
- hiimkeks 3y ago> So if you're ever doing this for a UART or network stream or something, always do the delimiter approach! I'm confused. I mean for UART, sure. But network streams is usually sent over a protocol that recovers lost data. Am I reckless for sending length-prefixed data chunks over TCP?
- rkangel 3y agoNo, you're right that this approach is ok for TCP streams. I shouldn't have thrown that in there without clarification that I meant something more generic. I actually use SLIP for packetisation over TCP anyway, because then syncing for any other logging or diagnostic thing that joins halfway through is easy. Basically I find it to just be a more robust system.
- mannyv 3y ago"I actually use SLIP for packetisation over TCP anyway" Whoa, that's an acronym I haven't heard in a long time. SLIP/TCP seems redundant. Your serial encoding IP then sending it over TCP/IP? Is this a tunnel?
- rkangel 3y agoTCP is a stream with no packet boundaries. If you want to separate that stream into a series of messages then you need to do packetisation. The SLIP delimiter approach is a pretty good one. You're right that it was originally for segmenting a serial link to send separate IP packets (so you could theoretically have SLIP over TCP over IP over SLIP), but it works just as well in other contexts.
- NoahKAndrews 3y agoA TCP network stream will guarantee no missed bytes via behind-the-scenes retransmissions, no?
- NavinF 3y agoYes, that's why it's very common to send length-prefixed messages over TCP. (The TCP checksum is very weak at only 16 bits, but there's usually another layer like TLS that gives you integrity for free so nobody cares)