4 ms·

I have a distant memory of Alan Cox saying something back in the day along the lines of "there are so many things left undocumented in the TCP/IP RFCs that if y

I have a distant memory of Alan Cox saying something back in the day along the lines of "there are so many things left undocumented in the TCP/IP RFCs that if you just implement it straight from the spec your stack won't work at all." But I can't find any reference to this now. Does anyone else remember this? Is this true? Can you interoperate with the Internet by just implementing the spec?

nabla9 7y ago
TCP/IP is relatively stable ecology where stacks in routers, applications etc. have evolved to survive with each other and intentionally malicious actors by developing coping mechanisms. Introducing a clean implementation of TCP/IP stack is like introducing a new species. It has no immune system against others and others don't know its 'signature behavior' either. You can probably get interoperability at least part of the time. You may have deteriorated throughput and more broken connections with some stack and not with others. But you have introduced a new species. If your brand new stack has a tiny bug or new kind of misconfiguration and you start spreading it fast, the hell can break loose and you may ruin the day of many people before they find you and cut you off.
maltalex 7y ago
> Is this true? Can you interoperate with the Internet by just implementing the spec? I've never implemented the entire stack. Few people have. But from the protocols I've implemented, that's only a slight exaggeration. As soon as a popular implementation has a bug or decides not to behave according to spec, everyone has to adapt. You can typically use the spec to get 95% of the way to full interoperability.
- andy_ppp 7y ago
  And presumably 95% of the work is in the final 5%?
jsnell 7y ago
No, it's not true. You can implement something that can load a webpage from 99.9% of the world's web servers from a couple of RFCs in a day. But this ease of implementation means that for basically any issue where the specs give any degree of freedom, somebody will have deviated from the norm. (I've implemented TCP from scratch three times, one of those implementations basically ran the traffic of entire countries.) The real problem is that every step toward 100% gets progressively harder to find and debug. See e.g. [0] for a middle box that would mangle connections if it received two identical SYN packets, or [1] for a way in which almost all servers anyone currently runs are accidentally resilient to a certain kind of connection-killing packet corruption, but S3 isn't. [0] https://www.snellman.net/blog/archive/2014-11-11-tcp-is-harder-than-it-looks.html https://www.snellman.net/blog/archive/2014-11-11-tcp-is-hard... [1] https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/ https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/
- eatonphil 7y ago
  > You can implement something that can load a webpage from 99.9% of the world's web servers from a couple of RFCs in a day. As an alternative perspective... while the RFCs are great, it's taken me weeks (maybe months now) to hack together a not-yet-functional TCP/IP stack. I've never dug below the application layer before. I'm still working through it. I cannot even say if the RFCs are sufficient, but I'll take your word.
toast0 7y ago
I don't think it's quite true of TCP/IP that you wouldn't work at all, although you do have to be careful, because there are a lot of RFCs, and it's not always clear which ones are important. Also, one of the RFCs has wrong functions for calculating or adjusting checksums. I think there's also some convention on tcp option ordering that may be important but not well documented. Either way, I would keep a couple other implementations close -- if not to peek at their code ocassionaly, at least to inspect their output.
whitequark_ 7y ago
I designed smoltcp (and wrote most of the code currently in it). The original TCP/IP RFC (RFC 793) contains several ambiguous requirements, and as a result they do not specify a well-defined system. There are also some outright incorrect statements. There are a few follow up RFCs (e.g. RFC 1122) that clarify these issues, and there are more RFCs (e.g. RFC 7414) that describe the TCP features that you should avoid using. By using this collection of TCP/IP RFCs that grew over the years, it is indeed possible to implement a stack from first principles and have it interoperate with other existing stacks without much trouble. (At least so long as you don't put the same bugs in your test suite as you do in your stack... which you will.) However, being able to transmit some bytes reliably, and having a high-performance stack that works well in real world conditions are different. You might be able to do the former from RFCs, but the latter absolutely requires a nontrivial amount of tribal knowledge that you have to collect crumb by crumb, and often quite painfully, too. Smoltcp is somewhere halfway between. It's pretty reliable, but I am sure there is much to be improved in its operation in adverse conditions, with obscure peers, and so on.

nabla9 7y ago

TCP/IP is relatively stable ecology where stacks in routers, applications etc. have evolved to survive with each other and intentionally malicious actors by developing coping mechanisms. Introducing a clean implementation of TCP/IP stack is like introducing a new species. It has no immune system against others and others don't know its 'signature behavior' either. You can probably get interoperability at least part of the time. You may have deteriorated throughput and more broken connections with some stack and not with others. But you have introduced a new species. If your brand new stack has a tiny bug or new kind of misconfiguration and you start spreading it fast, the hell can break loose and you may ruin the day of many people before they find you and cut you off.

maltalex 7y ago

> Is this true? Can you interoperate with the Internet by just implementing the spec? I've never implemented the entire stack. Few people have. But from the protocols I've implemented, that's only a slight exaggeration. As soon as a popular implementation has a bug or decides not to behave according to spec, everyone has to adapt. You can typically use the spec to get 95% of the way to full interoperability.

andy_ppp 7y ago
And presumably 95% of the work is in the final 5%?

jsnell 7y ago

No, it's not true. You can implement something that can load a webpage from 99.9% of the world's web servers from a couple of RFCs in a day. But this ease of implementation means that for basically any issue where the specs give any degree of freedom, somebody will have deviated from the norm. (I've implemented TCP from scratch three times, one of those implementations basically ran the traffic of entire countries.) The real problem is that every step toward 100% gets progressively harder to find and debug. See e.g. [0] for a middle box that would mangle connections if it received two identical SYN packets, or [1] for a way in which almost all servers anyone currently runs are accidentally resilient to a certain kind of connection-killing packet corruption, but S3 isn't. [0] https://www.snellman.net/blog/archive/2014-11-11-tcp-is-harder-than-it-looks.html https://www.snellman.net/blog/archive/2014-11-11-tcp-is-hard... [1] https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/ https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/

eatonphil 7y ago
> You can implement something that can load a webpage from 99.9% of the world's web servers from a couple of RFCs in a day. As an alternative perspective... while the RFCs are great, it's taken me weeks (maybe months now) to hack together a not-yet-functional TCP/IP stack. I've never dug below the application layer before. I'm still working through it. I cannot even say if the RFCs are sufficient, but I'll take your word.

toast0 7y ago

I don't think it's quite true of TCP/IP that you wouldn't work at all, although you do have to be careful, because there are a lot of RFCs, and it's not always clear which ones are important. Also, one of the RFCs has wrong functions for calculating or adjusting checksums. I think there's also some convention on tcp option ordering that may be important but not well documented. Either way, I would keep a couple other implementations close -- if not to peek at their code ocassionaly, at least to inspect their output.

whitequark_ 7y ago

I designed smoltcp (and wrote most of the code currently in it). The original TCP/IP RFC (RFC 793) contains several ambiguous requirements, and as a result they do not specify a well-defined system. There are also some outright incorrect statements. There are a few follow up RFCs (e.g. RFC 1122) that clarify these issues, and there are more RFCs (e.g. RFC 7414) that describe the TCP features that you should avoid using. By using this collection of TCP/IP RFCs that grew over the years, it is indeed possible to implement a stack from first principles and have it interoperate with other existing stacks without much trouble. (At least so long as you don't put the same bugs in your test suite as you do in your stack... which you will.) However, being able to transmit some bytes reliably, and having a high-performance stack that works well in real world conditions are different. You might be able to do the former from RFCs, but the latter absolutely requires a nontrivial amount of tribal knowledge that you have to collect crumb by crumb, and often quite painfully, too. Smoltcp is somewhere halfway between. It's pretty reliable, but I am sure there is much to be improved in its operation in adverse conditions, with obscure peers, and so on.