3 ms·

SYN packet handling in the wild

51 points by majke 9y ago

majke 9y ago
What I didn't mention in the article is a sea of corner cases around the SYN handling code. For example: did you know, that when SYN Cookies are disabled (sysctl set to 0), a quarter of slots in SYN Queue is reserved for "IPs that we have in route cache". http://elixir.free-electrons.com/linux/v4.14.8/source/net/ipv4/tcp_input.c#L6333 http://elixir.free-electrons.com/linux/v4.14.8/source/net/ip... /* Without syncookies last quarter of * backlog is filled with destinations, * proven to be alive. * It means that we continue to communicate * to destinations, already remembered * to the moment of synflood. */ Anyway, setting SYN Cookie sysctl to zero is a terrible idea. There is more, the SYN Queue used to bleed-out SYN packets at a rate of 0.5HZ if full. A feature basically nobody understood - removed in 4.10.
- usrme 9y ago
  Am about to read the article, but knowing myself I'll forget doing the following if I don't do it now: I want to thank you for what will undoubtedly be another enlightening article! If you don't mind an unsolicited AMA then I'd love to know how you learn about the topics you write about, what kind of experience you have in your field, at what point do you allow yourself to authoritatively say "yes, what I'm talking about is more or less correct" and publish a post to the wider world (this is something I struggle with), and what are some of the resources you recommend keeping up with or being aware of if someone else wishes to gain more knowledge in topics similar to those you post.
  - majke 9y ago
    In this case I stumbled upon a problem (trying to dig through the SYN handling code, specifically [1]) and sat down to understand it. When I finally "got" it I moved on and forgot everything. After couple of months a colleague asks me a question and I think - oh boy, if only I wrote about it when I still understood it! I also regularly ask smarter people when I'm lost. On correctness, let me cite my boss: > Never underestimate the value of shipping. Ships blogs, > open source, pull requests, tweets, products, etc. > > It's better to ship something "wrong" than ship nothing at > all. You'll get feedback that you'd never get if you just > talked about an idea. I don't think being 100% right all the time matters. On every mistake I make I learn and it's easy to fix a problem in a blog post! I generally think for non-trivial technical stuff, it's better to create some documentation when previously there was none, than to get it 100% correct. Another example: many "man" pages are technically correct but useless. My favorite example [2]. You would be blessed if you could take 10% of the most useful options and actually explain them to human beings. [1] https://elixir.free-electrons.com/linux/v4.14.13/source/net/ipv4/tcp_ipv4.c#L1324 https://elixir.free-electrons.com/linux/v4.14.13/source/net/... [2] https://linux.die.net/man/1/ps https://linux.die.net/man/1/ps
phw 9y ago
If the SYN queue is more than half full, the kernel (used to?) adapts the number of SYN/ACK retransmissions to mitigate the load. Interestingly, this opens a side channel that can be determined remotely, and used to measure packet filtering world-wide. See Section 2.1 in: https://censorbib.nymity.ch/pdf/Ensafi2015a.pdf https://censorbib.nymity.ch/pdf/Ensafi2015a.pdf
- majke 9y ago
  I'm quite sure this was removed in recent kernels, the "young" logic is now greatly simplified. Anyway - great paper. Thanks for digging it up.

51 points by majke 9y ago

What I didn't mention in the article is a sea of corner cases around the SYN handling code. For example: did you know, that when SYN Cookies are disabled (sysctl set to 0), a quarter of slots in SYN Queue is reserved for "IPs that we have in route cache". http://elixir.free-electrons.com/linux/v4.14.8/source/net/ipv4/tcp_input.c#L6333 http://elixir.free-electrons.com/linux/v4.14.8/source/net/ip... /* Without syncookies last quarter of * backlog is filled with destinations, * proven to be alive. * It means that we continue to communicate * to destinations, already remembered * to the moment of synflood. */ Anyway, setting SYN Cookie sysctl to zero is a terrible idea. There is more, the SYN Queue used to bleed-out SYN packets at a rate of 0.5HZ if full. A feature basically nobody understood - removed in 4.10.

usrme 9y ago
Am about to read the article, but knowing myself I'll forget doing the following if I don't do it now: I want to thank you for what will undoubtedly be another enlightening article! If you don't mind an unsolicited AMA then I'd love to know how you learn about the topics you write about, what kind of experience you have in your field, at what point do you allow yourself to authoritatively say "yes, what I'm talking about is more or less correct" and publish a post to the wider world (this is something I struggle with), and what are some of the resources you recommend keeping up with or being aware of if someone else wishes to gain more knowledge in topics similar to those you post.
- majke 9y ago
  In this case I stumbled upon a problem (trying to dig through the SYN handling code, specifically [1]) and sat down to understand it. When I finally "got" it I moved on and forgot everything. After couple of months a colleague asks me a question and I think - oh boy, if only I wrote about it when I still understood it! I also regularly ask smarter people when I'm lost. On correctness, let me cite my boss: > Never underestimate the value of shipping. Ships blogs, > open source, pull requests, tweets, products, etc. > > It's better to ship something "wrong" than ship nothing at > all. You'll get feedback that you'd never get if you just > talked about an idea. I don't think being 100% right all the time matters. On every mistake I make I learn and it's easy to fix a problem in a blog post! I generally think for non-trivial technical stuff, it's better to create some documentation when previously there was none, than to get it 100% correct. Another example: many "man" pages are technically correct but useless. My favorite example [2]. You would be blessed if you could take 10% of the most useful options and actually explain them to human beings. [1] https://elixir.free-electrons.com/linux/v4.14.13/source/net/ipv4/tcp_ipv4.c#L1324 https://elixir.free-electrons.com/linux/v4.14.13/source/net/... [2] https://linux.die.net/man/1/ps https://linux.die.net/man/1/ps

phw 9y ago

If the SYN queue is more than half full, the kernel (used to?) adapts the number of SYN/ACK retransmissions to mitigate the load. Interestingly, this opens a side channel that can be determined remotely, and used to measure packet filtering world-wide. See Section 2.1 in: https://censorbib.nymity.ch/pdf/Ensafi2015a.pdf https://censorbib.nymity.ch/pdf/Ensafi2015a.pdf

majke 9y ago
I'm quite sure this was removed in recent kernels, the "young" logic is now greatly simplified. Anyway - great paper. Thanks for digging it up.