4 ms·

We had to debug an issue with ARP during a deploy of docker containers a year or two ago. A nearly complete understanding of the lower layers is not that much

We had to debug an issue with ARP during a deploy of docker containers a year or two ago. A nearly complete understanding of the lower layers is not that much knowledge and quite useful at times.

5e92cb50239222b 4y ago
"Nearly complete"? No matter how deep your knowledge is, you're only scratching the surface. To pick a small example, there's something like 20 TCP congestion avoidance algorithms alone (many are available in the mainline Linux kernel, and they can be picked depending on the task and network at hand), and I believe it took something like a decade of research, trial and error to solve the bufferbloat problem. https://en.wikipedia.org/wiki/TCP_congestion_control https://en.wikipedia.org/wiki/TCP_congestion_control https://lwn.net/Articles/616241/ https://lwn.net/Articles/616241/
- hn_go_brrrrr 4y ago
  Only CUBIC and BBR seem to have any meaningful usage, however: https://www.comp.nus.edu.sg/~ayush/images/sigmetrics2020-gordon.pdf https://www.comp.nus.edu.sg/~ayush/images/sigmetrics2020-gor...
  - icedchai 4y ago
    I tried out BBR on a system that had a ton of packet loss. It made a tremendous difference!
- n_kr 4y ago
  > No matter how deep your knowledge is, you're only scratching the surface. I understand this is just emphasis, but no, its not magic, its not innate ability, its just software man! If you have dug deep enough, and understood it, that's it. Key phrase is IMO 'understood', but that's universal.
  - aflag 4y ago
    I think the point is that it may be impossible for a single human to have "nearly complete understanding" of how the networking stack work. But maybe what was meant was nearly complete understanding of the fundamentals. That's certainly achievable. But networking in the kernel is a beast of a thing with specialists in small parts of it. But I don't think there's a single human that know nearly all that those specialists know combined.
  - deleted 4y ago
    [deleted]
sophacles 4y ago
I agree with your main point - a decent working knowledge down-stack really helps with network stuffs. "nearly complete" strikes me as a whole lot of hubris though - I've spent almost 20 years at a career that can be summed up as "weird stuff with packets on linux", and the only thing that I'm certain of is: every time I think I'm starting to get a handle on this networking thing, the universe shows me that my assessment of how much there is to know was off by at least an order of magnitude.
- macintux 4y ago
  I once interviewed someone with “Internet guru” on his résumé. I advised him there were only a handful of people I’d consider for that title.
- arpa 4y ago
  i have no idea how networks work, and i have been doing link splitting, bonding, packet scheduling and dabbling in bgp in 2004. The knowledge i've gained since helps immensely when debugging simple networks created by kube-proxy, but i am just petrified by amount of the iceberg that remains underneath. And to this day i'm slightly perplexed when udp socket operation in php throws `connection reset` exception.
- twic 4y ago
  Perhaps it can be "nearly complete" in a pareto sense. If you know the structure and semantics of IP, UDP, and TCP packets, including their most common few options, and DNS and HTTP, then you know <<10% of what there is to know about networks, but >90% of what you will see in a random pcap file, and >90% of what you know to solve a randomly selected networking problem.

We had to debug an issue with ARP during a deploy of docker containers a year or two ago. A nearly complete understanding of the lower layers is not that much knowledge and quite useful at times.

5e92cb50239222b 4y ago

"Nearly complete"? No matter how deep your knowledge is, you're only scratching the surface. To pick a small example, there's something like 20 TCP congestion avoidance algorithms alone (many are available in the mainline Linux kernel, and they can be picked depending on the task and network at hand), and I believe it took something like a decade of research, trial and error to solve the bufferbloat problem. https://en.wikipedia.org/wiki/TCP_congestion_control https://en.wikipedia.org/wiki/TCP_congestion_control https://lwn.net/Articles/616241/ https://lwn.net/Articles/616241/

hn_go_brrrrr 4y ago
Only CUBIC and BBR seem to have any meaningful usage, however: https://www.comp.nus.edu.sg/~ayush/images/sigmetrics2020-gordon.pdf https://www.comp.nus.edu.sg/~ayush/images/sigmetrics2020-gor...
- icedchai 4y ago
  I tried out BBR on a system that had a ton of packet loss. It made a tremendous difference!
n_kr 4y ago
> No matter how deep your knowledge is, you're only scratching the surface. I understand this is just emphasis, but no, its not magic, its not innate ability, its just software man! If you have dug deep enough, and understood it, that's it. Key phrase is IMO 'understood', but that's universal.
- aflag 4y ago
  I think the point is that it may be impossible for a single human to have "nearly complete understanding" of how the networking stack work. But maybe what was meant was nearly complete understanding of the fundamentals. That's certainly achievable. But networking in the kernel is a beast of a thing with specialists in small parts of it. But I don't think there's a single human that know nearly all that those specialists know combined.
- deleted 4y ago
  [deleted]

sophacles 4y ago

I agree with your main point - a decent working knowledge down-stack really helps with network stuffs. "nearly complete" strikes me as a whole lot of hubris though - I've spent almost 20 years at a career that can be summed up as "weird stuff with packets on linux", and the only thing that I'm certain of is: every time I think I'm starting to get a handle on this networking thing, the universe shows me that my assessment of how much there is to know was off by at least an order of magnitude.

macintux 4y ago
I once interviewed someone with “Internet guru” on his résumé. I advised him there were only a handful of people I’d consider for that title.
arpa 4y ago
i have no idea how networks work, and i have been doing link splitting, bonding, packet scheduling and dabbling in bgp in 2004. The knowledge i've gained since helps immensely when debugging simple networks created by kube-proxy, but i am just petrified by amount of the iceberg that remains underneath. And to this day i'm slightly perplexed when udp socket operation in php throws `connection reset` exception.
twic 4y ago
Perhaps it can be "nearly complete" in a pareto sense. If you know the structure and semantics of IP, UDP, and TCP packets, including their most common few options, and DNS and HTTP, then you know <<10% of what there is to know about networks, but >90% of what you will see in a random pcap file, and >90% of what you know to solve a randomly selected networking problem.