2 ms·

For people asking "do these small tricks/trivia really matter??" I have an example of when they do: - having strange networking issues on the blue side of a bl

For people asking "do these small tricks/trivia really matter??" I have an example of when they do: - having strange networking issues on the blue side of a blue/green deployment - networking engineers are involved - nobody seems to be able to figure out what's going on despite LOTS of tcpdump/wireshark etc - I suggest tcpflow[0] (which I used for protocol analysis of chat services etc) - (there is some skepticism as I was SRE and not networking) - Turns out that the TCP messages were getting truncated on the problem side - Networking guys realize the config issue and fix it I have other examples but this is why it's always good to learn new commands/tools etc. This, in turn, reminds me of a quote from an army jungle survival expert: "People ask me if it's a good idea to read survival books. I say: 100%. You would be surprised how many people survive an emergency situation because their brain pops out some critical piece of information from a book or article they read 10 years ago." 0 - https://linux.die.net/man/1/tcpflow https://linux.die.net/man/1/tcpflow

js2 14d ago
I debugged terrible git clone performance over the VPN at my last company. It turned out our firewall was stripping TCP window scaling by default, which included both our internal GitHub Enterprise instance hosted on AWS and github.com. I collected a bunch of packet traces (tcpdump was sufficient) and eventually got the networking folks to fix the firewall config. This was a company of a few thousand people with programmers working from home who all must've assumed it was fine to get no more than ~1 Mbps git clone performance. After a few months of putting up with it I finally got tired of the issue and spent 30 minutes tracking down the cause. It still took a week or more of back and forth between me, and the networking and security teams. And it became an ongoing issue as they were allow-listing IP addresses, not fixing the root cause. Why are Cisco firewalls stripping TCP window scaling by default in 2026? I have no idea! So, yeah...
xorcist 13d ago
No to belittle your find even the slightest, but it might be good to notice that tcpflow decodes tcp flows, which is exactly what Wireshark does, only the latter in a GUI and with a plethora of other protocol decoders. Just right click any tcp packet and select Follow. Both tools even use the same filtering language, coming from tcpdump itself. Wireshark's protocol decoder is more advanced than tcpflow. It does understand fragmented packets. So given a choice, Wireshark is often the preferred tool.

js2 14d ago

I debugged terrible git clone performance over the VPN at my last company. It turned out our firewall was stripping TCP window scaling by default, which included both our internal GitHub Enterprise instance hosted on AWS and github.com. I collected a bunch of packet traces (tcpdump was sufficient) and eventually got the networking folks to fix the firewall config. This was a company of a few thousand people with programmers working from home who all must've assumed it was fine to get no more than ~1 Mbps git clone performance. After a few months of putting up with it I finally got tired of the issue and spent 30 minutes tracking down the cause. It still took a week or more of back and forth between me, and the networking and security teams. And it became an ongoing issue as they were allow-listing IP addresses, not fixing the root cause. Why are Cisco firewalls stripping TCP window scaling by default in 2026? I have no idea! So, yeah...

xorcist 13d ago

No to belittle your find even the slightest, but it might be good to notice that tcpflow decodes tcp flows, which is exactly what Wireshark does, only the latter in a GUI and with a plethora of other protocol decoders. Just right click any tcp packet and select Follow. Both tools even use the same filtering language, coming from tcpdump itself. Wireshark's protocol decoder is more advanced than tcpflow. It does understand fragmented packets. So given a choice, Wireshark is often the preferred tool.