4 ms·

In my experience, the biggest issue with TCP is that it assumes that the traffic is a single continuous byte stream, whereas the majority of applications send m

In my experience, the biggest issue with TCP is that it assumes that the traffic is a single continuous byte stream, whereas the majority of applications send messages. Vint mentioned the lack of encryption as a mistake, but even on private networks, the 16-bit TCP checksums are too weak to protect against bit-flips. Many orgs noticed that enforcing encryption reduces "random" crashes and errors, because it also enforces a strong checksum, typically 128 or 256 bit. However, even a 32-bit checksum would have sufficed for ordinary use. Almost all "real" protocols built on top of TCP eventually end up reinventing the same wheels: message boundaries, logical sub-streams with priorities, and strong checksums or encryption. Examples include Citrix's ICA, HTTP 3.0, gRPC, etc...

zubnix 4y ago
Http 3.0 is UDP, ICA these days is also UDP.
- jayd16 4y ago
  Yes, but by necessity and only after http/2 attempted everything listed over TCP.
- klabb3 4y ago
  Technically true, but parent’s point still stands (even if it should have said “on top of IP”). Byte streams are not ideal to deal with for protocols. As soon as you have encryption, like tls, you have discrete messages anyway (records) and the stream is largely an illusion. That said, I still like TCP as a compromise between simplicity and versatility. It’s been holding up incredibly well for a huge variety of use cases.
jandrese 4y ago
IMHO TCP made the correct call here. TCP is a transport, not an application protocol. You are supposed to run your protocol over top of it, including things like blocking, substreams, etc... It's easy to add features to a simple transport, it's not so easy to work around features when you don't want them, plus they end up complicating the stack and become a possible source of bugs. Even "simple" TCP proved to be quite a challenge to implement back in the 90s, with many many buggy stacks out on the Internet. By being just a stream of bytes you are free to do whatever you want with the protocol. Sure that may mean implementing your own form of blocking but that's better than being stuck in a protocol where you are forced to work around the message oriented features even though you are just streaming data. The checksum does hail from an earlier era where 16 bit machines were commonplace and the total amount of data sent is minimal. Also, there is some assumption that the underlying transport is going to have its own checksums. You will note that IPv6 ditches the checksum entirely.
- klabb3 4y ago
  > By being just a stream of bytes you are free to do whatever you want with the protocol. Not entirely. Head-of-line blocking is an inherent problem with TCP so you can’t have real priority “streams” like you can with eg QUIC. Another “missing feature” is datagrams, ie simply opting out of the retransmission part of tcp. That said, I still agree with the general statement.
  - foobarian 4y ago
    Didn't they build that weird "emergency packet" feature to skip to the head of line? IIRC it was never well supported in various system stacks..
    - sanxiyn 4y ago
      Yes, TCP header had 16 bits allocated to "Urgent Pointer". It's (unfortunately) still there for compatibility, but RFC 9293, which updated RFC 793 after more than 40 years, has the following to say about it: > As a result of implementation differences and middlebox interactions, new applications SHOULD NOT employ the TCP urgent mechanism. However, TCP implementations MUST still include support for the urgent mechanism.
      - LegionMammal978 4y ago
        As it happens, that feature is used in the FTP protocol (RFC 959) to abort the current file transfer (or to immediately print a status message), since some old implementations would have to stop listening on the control connection while a transfer was in progress and could only be woken up with an interrupt. At least vsftpd supports it, I haven't looked at any other implementations. The main difficulty is that the libc API for reading urgent data is somewhat painful, and most higher-level TCP libraries haven't bothered with it for obvious reasons. Of course, basic ftp:// clients (the only kind that are really used today) would probably rather just close the control connection, which has the same effect.
- jiggawatts 4y ago
throw0101b 4y ago
> In my experience, the biggest issue with TCP is that it assumes that the traffic is a single continuous byte stream, whereas the majority of applications send messages. TCP and IP used to be one thing, and they split off TCP in what we now call Layer 4 (where UDP, and others, also exist). So they did think ahead in separating things to a certain extent. The fact that we have a whole bunch of middle-boxes that don't allow DCCP, SCTP, etc, is hardly the fault of Cerf et al.

zubnix 4y ago

Http 3.0 is UDP, ICA these days is also UDP.

jayd16 4y ago
Yes, but by necessity and only after http/2 attempted everything listed over TCP.
klabb3 4y ago
Technically true, but parent’s point still stands (even if it should have said “on top of IP”). Byte streams are not ideal to deal with for protocols. As soon as you have encryption, like tls, you have discrete messages anyway (records) and the stream is largely an illusion. That said, I still like TCP as a compromise between simplicity and versatility. It’s been holding up incredibly well for a huge variety of use cases.

jandrese 4y ago

IMHO TCP made the correct call here. TCP is a transport, not an application protocol. You are supposed to run your protocol over top of it, including things like blocking, substreams, etc... It's easy to add features to a simple transport, it's not so easy to work around features when you don't want them, plus they end up complicating the stack and become a possible source of bugs. Even "simple" TCP proved to be quite a challenge to implement back in the 90s, with many many buggy stacks out on the Internet. By being just a stream of bytes you are free to do whatever you want with the protocol. Sure that may mean implementing your own form of blocking but that's better than being stuck in a protocol where you are forced to work around the message oriented features even though you are just streaming data. The checksum does hail from an earlier era where 16 bit machines were commonplace and the total amount of data sent is minimal. Also, there is some assumption that the underlying transport is going to have its own checksums. You will note that IPv6 ditches the checksum entirely.

klabb3 4y ago
> By being just a stream of bytes you are free to do whatever you want with the protocol. Not entirely. Head-of-line blocking is an inherent problem with TCP so you can’t have real priority “streams” like you can with eg QUIC. Another “missing feature” is datagrams, ie simply opting out of the retransmission part of tcp. That said, I still agree with the general statement.
- foobarian 4y ago
  Didn't they build that weird "emergency packet" feature to skip to the head of line? IIRC it was never well supported in various system stacks..
  - sanxiyn 4y ago
    Yes, TCP header had 16 bits allocated to "Urgent Pointer". It's (unfortunately) still there for compatibility, but RFC 9293, which updated RFC 793 after more than 40 years, has the following to say about it: > As a result of implementation differences and middlebox interactions, new applications SHOULD NOT employ the TCP urgent mechanism. However, TCP implementations MUST still include support for the urgent mechanism.
    - LegionMammal978 4y ago
      As it happens, that feature is used in the FTP protocol (RFC 959) to abort the current file transfer (or to immediately print a status message), since some old implementations would have to stop listening on the control connection while a transfer was in progress and could only be woken up with an interrupt. At least vsftpd supports it, I haven't looked at any other implementations. The main difficulty is that the libc API for reading urgent data is somewhat painful, and most higher-level TCP libraries haven't bothered with it for obvious reasons. Of course, basic ftp:// clients (the only kind that are really used today) would probably rather just close the control connection, which has the same effect.
jiggawatts 4y ago

throw0101b 4y ago

> In my experience, the biggest issue with TCP is that it assumes that the traffic is a single continuous byte stream, whereas the majority of applications send messages. TCP and IP used to be one thing, and they split off TCP in what we now call Layer 4 (where UDP, and others, also exist). So they did think ahead in separating things to a certain extent. The fact that we have a whole bunch of middle-boxes that don't allow DCCP, SCTP, etc, is hardly the fault of Cerf et al.