5 ms·

Understand a little about TCP (2015)

65 points by ashitlerferad 6y ago

fosk 6y ago
Worth nothing that TCP_NODELAY is on by default on NGINX[1]. [1] http://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nodelay http://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_...
- heyoni 6y ago
  Should you keep it on then if you’re using SSL?
- im3w1l 6y ago
  Yeah this seems like some low-level detail that should really be outsourced to whatever library / framework / server you are using.
29athrowaway 6y ago
Transfer Control Protocol (TCP) provides solutions for many problems affecting data transmission. The most important aspect, reliability, is addressed via acknowledgments and retransmissions, and shapes the "TCP state machine" (insert diagram here). Congestion control is another one. One mechanism of congestion control in TCP is the Nagle's algorithm. https://en.wikipedia.org/wiki/Nagle%27s_algorithm https://en.wikipedia.org/wiki/Nagle%27s_algorithm Disabling Nagle's algorithm is usually done via the TCP_NODELAY flag. If you want to learn more about networking I recommend: - http://williamstallings.com/DataComm/ http://williamstallings.com/DataComm/ - https://beej.us/guide/bgnet/ https://beej.us/guide/bgnet/
SomaticPirate 6y ago
I’m always curious how someone gains this lower level knowledge. It seems there isn’t much substitute to actually implementing a reference yourself or keeping an ear to ground for blogs like this. Any other good resources to learn low-level (Linux) networking? I would love to know what conntrack actually does rather than just raising the limit in my applications when I see a special log line.
- jcrawfordor 6y ago
  This free textbook is a pretty good one on general computer networking: http://intronetworks.cs.luc.edu/current2/html/ http://intronetworks.cs.luc.edu/current2/html/ I find it hard to get a good single source on Linux networking as the situation tends to change fast enough to make big efforts into documenting it out of date just as they're finished. You can learn a lot by messing around at home though if you have a couple spare machines. These days it might be a better choice to focus on software-defined networking, but so much of SDN is proprietary that it's hard to feel great about learning it.
- harry8 6y ago
  The late Richard W. Stevens wrote the classic texts. TCP/IP illustrated and Unix Network Programming. I have old editions and haven't seen recent ones. There's murmourings that the updated editions are not as good, the writers emplyed to updated them don't have Steven's talents, can't confirm. Advanced Programming in the Unix Environment by the same author is also worthwhile IMHO. And absolutely, use the books to write your own webserver (or irc client or ...) and your own TCP/IP stack if you can find the time and motiviation you won't regret it. It's liberating to do it half-assed, doesn't have to be good at all. Improve it as much as you care to. edit: Julia Evans is inspiring. See how she goes about it. We should something like what she does when things crop up we don't fully understand. Don't have to write them up like she does, but why not? Feynman technique, if you can explain it simply to the person you were before you learned it you understand it.
  - mr_toad 6y ago
    > Unix Network Programming. If you want to learn to send working TCP/IP packets across the internet with just plain hand crafted C code and no networking frameworks or libraries this is a must read.
- toast0 6y ago
  Read the books and the blogs (as time permits), look at tcpdump and wireshark when clients do weird stuff; then try to figure out why, and how you can fix it, when you can only control one side of the connection. Try a bunch of stuff. Conntrack is pretty simple though; if you've configured your firewall rules for low performance, Xtables keeps track of each of the active connections, and there's a limit. If you hit the limit, it won't track anymore connections and you're stuck. Try to configure stateless firewall rules instead :)
dosshell 6y ago
What does the author mean with "TCP packages"? How is it different from the IP packet?
- kabes 6y ago
  Tcp is a transport layer protocol on top of ip. So the tcp packet with stuff like sequence and port numbers in its header is encapsulated as the data part of an ip packet, which in turn is send via lower level packets like ethernet frames.
- danwills 6y ago
  I think they are different layers with TCP/UDP sitting on top of the IP layer: https://en.wikipedia.org/wiki/File%3AUDP_encapsulation.svg https://en.wikipedia.org/wiki/File%3AUDP_encapsulation.svg From: https://en.wikipedia.org/wiki/Internet_protocol_suite?wprov=sfla1 https://en.wikipedia.org/wiki/Internet_protocol_suite?wprov=...
helsinkiandrew 6y ago
Nagle’s Syndrome. This came up in a HN post a few days ago and seemingly every week. There should be posters made and widely distributed saying if you experience Unexpected 40ms delays in your application read up on the Nagle algorithm and TCP_NODELAY
- tleb_ 6y ago
  The issue isn't Nagle's algorithm but the delayed ACK & Nagle's algorithm combination. https://news.ycombinator.com/item?id=24786196 https://news.ycombinator.com/item?id=24786196
andmarios 6y ago
Try to build the Linux kernel on your own (it's easy) and you'll find hundreds of different algorithms for all things networking and TCP. There isn't a global best algorithm or setting. Depending on whether you are a server or a client, whether you are on a connection that offers high banwidth, low latency, both or none, depending on the application (streaming, torrenting, VoIP), some configuration will be better than another. If I recall correctly, delayed acks and tcp delay are there to provide higher throughput and prevent network congestion —think downloading from an ftp server as fast as possible, without causing trouble to the rest of the channel's tenants. The solution provided works very well for a specific problem, HAProxy talking to a backend application, but would probably make things worse if you were to use it for your laptop from your shared home connection.

65 points by ashitlerferad 6y ago

fosk 6y ago

Worth nothing that TCP_NODELAY is on by default on NGINX[1]. [1] http://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nodelay http://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_...

heyoni 6y ago
Should you keep it on then if you’re using SSL?
im3w1l 6y ago
Yeah this seems like some low-level detail that should really be outsourced to whatever library / framework / server you are using.

29athrowaway 6y ago

Transfer Control Protocol (TCP) provides solutions for many problems affecting data transmission. The most important aspect, reliability, is addressed via acknowledgments and retransmissions, and shapes the "TCP state machine" (insert diagram here). Congestion control is another one. One mechanism of congestion control in TCP is the Nagle's algorithm. https://en.wikipedia.org/wiki/Nagle%27s_algorithm https://en.wikipedia.org/wiki/Nagle%27s_algorithm Disabling Nagle's algorithm is usually done via the TCP_NODELAY flag. If you want to learn more about networking I recommend: - http://williamstallings.com/DataComm/ http://williamstallings.com/DataComm/ - https://beej.us/guide/bgnet/ https://beej.us/guide/bgnet/

SomaticPirate 6y ago

I’m always curious how someone gains this lower level knowledge. It seems there isn’t much substitute to actually implementing a reference yourself or keeping an ear to ground for blogs like this. Any other good resources to learn low-level (Linux) networking? I would love to know what conntrack actually does rather than just raising the limit in my applications when I see a special log line.

jcrawfordor 6y ago
This free textbook is a pretty good one on general computer networking: http://intronetworks.cs.luc.edu/current2/html/ http://intronetworks.cs.luc.edu/current2/html/ I find it hard to get a good single source on Linux networking as the situation tends to change fast enough to make big efforts into documenting it out of date just as they're finished. You can learn a lot by messing around at home though if you have a couple spare machines. These days it might be a better choice to focus on software-defined networking, but so much of SDN is proprietary that it's hard to feel great about learning it.
harry8 6y ago
The late Richard W. Stevens wrote the classic texts. TCP/IP illustrated and Unix Network Programming. I have old editions and haven't seen recent ones. There's murmourings that the updated editions are not as good, the writers emplyed to updated them don't have Steven's talents, can't confirm. Advanced Programming in the Unix Environment by the same author is also worthwhile IMHO. And absolutely, use the books to write your own webserver (or irc client or ...) and your own TCP/IP stack if you can find the time and motiviation you won't regret it. It's liberating to do it half-assed, doesn't have to be good at all. Improve it as much as you care to. edit: Julia Evans is inspiring. See how she goes about it. We should something like what she does when things crop up we don't fully understand. Don't have to write them up like she does, but why not? Feynman technique, if you can explain it simply to the person you were before you learned it you understand it.
- mr_toad 6y ago
  > Unix Network Programming. If you want to learn to send working TCP/IP packets across the internet with just plain hand crafted C code and no networking frameworks or libraries this is a must read.
toast0 6y ago
Read the books and the blogs (as time permits), look at tcpdump and wireshark when clients do weird stuff; then try to figure out why, and how you can fix it, when you can only control one side of the connection. Try a bunch of stuff. Conntrack is pretty simple though; if you've configured your firewall rules for low performance, Xtables keeps track of each of the active connections, and there's a limit. If you hit the limit, it won't track anymore connections and you're stuck. Try to configure stateless firewall rules instead :)

dosshell 6y ago

What does the author mean with "TCP packages"? How is it different from the IP packet?

kabes 6y ago
Tcp is a transport layer protocol on top of ip. So the tcp packet with stuff like sequence and port numbers in its header is encapsulated as the data part of an ip packet, which in turn is send via lower level packets like ethernet frames.
danwills 6y ago
I think they are different layers with TCP/UDP sitting on top of the IP layer: https://en.wikipedia.org/wiki/File%3AUDP_encapsulation.svg https://en.wikipedia.org/wiki/File%3AUDP_encapsulation.svg From: https://en.wikipedia.org/wiki/Internet_protocol_suite?wprov=sfla1 https://en.wikipedia.org/wiki/Internet_protocol_suite?wprov=...

helsinkiandrew 6y ago

Nagle’s Syndrome. This came up in a HN post a few days ago and seemingly every week. There should be posters made and widely distributed saying if you experience Unexpected 40ms delays in your application read up on the Nagle algorithm and TCP_NODELAY

tleb_ 6y ago
The issue isn't Nagle's algorithm but the delayed ACK & Nagle's algorithm combination. https://news.ycombinator.com/item?id=24786196 https://news.ycombinator.com/item?id=24786196

andmarios 6y ago

Try to build the Linux kernel on your own (it's easy) and you'll find hundreds of different algorithms for all things networking and TCP. There isn't a global best algorithm or setting. Depending on whether you are a server or a client, whether you are on a connection that offers high banwidth, low latency, both or none, depending on the application (streaming, torrenting, VoIP), some configuration will be better than another. If I recall correctly, delayed acks and tcp delay are there to provide higher throughput and prevent network congestion —think downloading from an ftp server as fast as possible, without causing trouble to the rest of the channel's tenants. The solution provided works very well for a specific problem, HAProxy talking to a backend application, but would probably make things worse if you were to use it for your laptop from your shared home connection.