12 ms·
How Do Routers Work, Really?
- geerlingguy 6y agoI learned how routers really work from Ericsson's seminal video on the matter, The Good Warriors of the Net: https://www.youtube.com/watch?v=x9XWxD6cJuY https://www.youtube.com/watch?v=x9XWxD6cJuY Though I always thought the "router switch" was much more fun.
- dec0dedab0de 6y agoHaha I forgot about this video. It was required viewing at my first job.
- sgillen 6y agoHaha thanks for sharing. Interesting how much emphasis there is on "the ping of death" compared to literally any other exploit. Does anyone know if this was really such a big problem when this video came out?
- geerlingguy 6y agoI do remember hearing about it causing issues here and there in the 90s/early 00s, but rarely. Never hear about it anymore. But I do remember AppleTalk causing issues more frequently on a network I helped manage that had radio studios with two Macs per studio, but mostly Windows PCs through the rest of the building. That place also had a Macintosh 512K running its phone system until around 2010!
- schoen 6y agoWhat I remember is that the ping of death was extremely surprising in terms of the number of OSes affected, the ease of exploiting it, and the super-noticeable consequence of instantly crashing the target machine. And it came out at a time when there wasn't as much vulnerability research and very few extensively cross-platform vulnerabilities. Also, with the ping of death, the only way to use it was to very noticeably crash systems -- not to secretly build a botnet or something, as might have been done with RCE vulnerabilities.
- kitteh 6y agoIt was popular for booting people off IRC, but there were other exploits around the same era that did the same such as land and teardrop. It wasn't super notable. What was more horrific was the amount of windows machines that had tcp ports for various windows services open to the internet that led to not only crashing but remote compromise and rootkits/botnet stuff. That went on for years and only got mitigated by people deploying routers with fw/Nat functionality.
- jpxw 6y agoJust watched the whole video, amazing, nostalgic but also subtly wrong in a number of annoying ways!
- whoisburbansky 6y agoFor someone with only a passing understanding of router innards, what should I watch out for from this talk to avoid coming away with an incorrect understanding of how things work?
- fiatjaf 6y agoIn my experience programmers are very friendly and kind and are always eager to help everybody understand what they do understand about programming. In contrast with that, people from the "networking" world often look with despise to people who don't understand what they do and want to prevent them from learning, they love to just say what is wrong and never point to what would be right and why, they also will most of the times just keep saying they must hire someone to do the job instead of learning. That is my experience on Networking Stack Exchange, on ##networking channel on Freenode and also the impression I have from a friend that deals with networking, although I try to not talk about it with him for the reasons above.
- pfundstein 6y agoTo counter your anecdote with my own, I've been working for 10 years on a team which has seen Network Engineers, Systems Engineers and Software Engineers come and go, and I've seen three(!) very arrogant Software Engineers who as it turned out didn't know what they were doing. But the same goes for other disciplines, we've had a straight up antisocial Network Engineer who only worked from home and never answered his phone (he did excellent diagrams though). We've had an arrogant Systems Engineer that refused to document anything. These people were fired, but my point is that you should blame the person rather than the discipline.
- inopinatus 6y agoI've been working alongside network engineers for thirty years in a variety of ISP, IXP, RIR, corporate, carrier, DC and public cloud environments, and do not recognise the people you are describing. These colleagues run the usual gamut of human personalities, but invariably the most respected and senior contributors are those that enable others through sharing their knowledge and experience. They were never anything but helpful and patient even when I was just getting started and full of basic questions about BGP and mixing up my fibre modes. However, I have also contributed to Stack Overflow and managed IRC channels and servers. The negative traits you've described do correlate to the hostile attitudes endemic within many StackExchange and IRC communities. They are not correlated to my workplace experience of network engineers.
- Spare_account 6y agoI watched this decades ago and forgot just enough about it that I couldn't find it again recently when I tried. Thank you
- lelandbatey 6y agoSlightly higher quality version here: https://youtu.be/PBWhzz_Gn10 https://youtu.be/PBWhzz_Gn10
- hoten 6y ago"accidents happen [in LAN]", "at least the router is exact (for the most part)" What does this mean? Then towards the end... "the packet is recycled". What?
- geerlingguy 6y agoI don't know about packet recycling, but at least with the 'for the most part', packet collision and packet loss used to be a lot more common for some reason. Nowadays the only times I see them on local networks is when cables get badly kinked or terminations are poorly done.
- methou 6y agoSomeone gotta make this in Factorio
- pfarrell 6y agoI would suggest expanding your terminology section. I know almost nothing about routers and I'm lost in the first sentence of the High Level Overview section. "A switch (or an L2 switch :-) ) is an L2-only thing." I don't know what L2 means. I suspect a definition of the various levels would expand the audience for this post.
- AlphaSite 6y agoI think you need to know your audience and cater to them, trying to explain everything just ends in a book. L2 is especially googleable.
- pfarrell 6y agoThis is a good point. You have to have some assumptions of what your audience brings. I'm aware there are levels of information in an IP packet, but I don't know them offhand. If I have to google something on the first sentence in a high level overview, then I'm likely not going to read the piece and the author has lost me as a reader. Maybe I'm not the target audience, though I was interested. I'm providing that as feedback for the origial author since the piece mentions that's it's still a work in progress.
- deleted 6y ago[deleted]
- hinkley 6y agoTo be fair, L2 could be Layer 2 or Level 2 (cache) and it might be a crapshoot what you get. You might get confused trying to answer your own questions. Discoverability lives in the space between overexplaining and underexplaining.
- wruza 6y agoOne can just add switch, router, network, etc to the query until it works. Supposedly they'll all work. Weak google fu means no info today, and if OP and the author are not the same person, then the latter may not even have a clue that it was posted on hn, where such high standards apply. If someone brought an electronics forum wiki post, should one expect every TLA¹ to be explained there too? ¹ Three Letter Acronym/Abbreviation
- rabuse 6y agoI learned a lot about networking when setting up servers in racks. Had to deal with issues arising from terrible UI's on a lot of the routers out there, so I just kept digging deeper and deeper into how it all works. Also, if more are looking into how packets are actually routed, look into BGP, and how CDN's work. Great stuff.
- walshemj 6y agoI would start with how internal routing works before starting on WAN routing. Id look at the cisco press and CCNA training materials
- anotherkamila_ 6y agoHi, I'm the author. Uh hi w00t how why what's it doing here?! :D I promise to make it better and actually finish it now! Check back in a day or two I guess? Also I should post the code I promised. Hello from the ADHD squirrel!
- anotherkamila_ 6y agoAlso thanks a ton for your suggestions, I really appreciate them!
- coolgeek 6y agoLove the URL!
- xg15 6y ago> Note that the next hop’s IP address is in the router’s memory only: it does not appear in the packet at any time. This clears some points that always puzzled me: If the gateway is identified by an IP address, but the destination host is also an IP address, which address exactly is put into the packet? And how can a packet be routed if the gateway's IP is itself part of the subnet that's supposed to be routed to it. (E.g. 192.168.0.0/24 with default gateway 192.168.0.1) So the answer is, if I send the packet to host 1.1.1.1 but the routing table has 2.2.2.2 as the next hop, the packet will have 1.1.1.1 as the destination in the IP part but the MAC of 2.2.2.2 as destination of the Ethernet part (or equivalent). It doesn't matter which subnet the next hop's IP is in, as the routing table isn't consulted for it anyway - it's only used in ARP) This leaves the question, why the indirection and why the mucking around with ARP and IPs that are never used as the destination to anything? Couldn't you simply put the next hop's MAC address (instead of IP address) into the routing table and be able to route packets just as well, with a lot less complexity?
- wmf 6y agoHistorically, some links didn't have MAC addresses and different link types have different address types so it's easier for the routing protocols to work in terms of IP addresses.
- swinglock 6y ago> It doesn't matter which subnet the next hop's IP is in, as the routing table isn't consulted for it anyway - it's only used in ARP) You can only ARP for hosts on the same subnet as you, terrible hacks excluded. > This leaves the question, why the indirection and why the mucking around with ARP and IPs that are never used as the destination to anything? Because it was designed in layers so that different layers could be replaced. We didn't know we'd end up with mostly only IP and Ethernet in LANs back then. > Couldn't you simply put the next hop's MAC address (instead of IP address) into the routing table and be able to route packets just as well, with a lot less complexity? It could have been done in any number of ways. It's not that much complexity through and it would bake Ethernet MACs into everything IP, even in the cases where it's not needed.
- 6y ago
- Cyph0n 6y ago> If that is the case, my condolences. As a software engineer working on IOS-XR, that gave me a chuckle :p In the case of enterprise- and SP-grade routers, the data-plane - i.e., where the actual forwarding and lookups take place - runs entirely on a dedicated network processor (NP), mainly for performance reasons. Information on the NP is populated by the router's operating system in response to user configuration, network topology changes, or protocol state updates. On the other hand, the control plane runs mainly on the CPU(s). This is required so that the protocols running on the router OS (e.g., BGP) can receive and send out updates based on their state machines.
- peterwwillis 6y agoI think the simplest way for people familiar with PCs to visualize it are the FirePOWER devices. Network cards plugged into some slot have embedded chips which can be programmed to, say, filter specific kinds of traffic, or pass it onto the host CPU for more advanced logic. While the machine's central CPU runs a web interface, manages local databases, downloads updates, manages clusters, records metrics, etc. And either can even be hot-pluggable, interchangeable blades in a larger machine chassis. Protocol-wise, isn't it common now for the NP on higher end stuff to handle L4 and higher protocols? Or are those still largely managed by the CPU?
- Cyph0n 6y agoYeah, NPs can handle L4 protocols, but I believe it’s usually a hybrid approach where the logic is split between CPU and NP.
- mrkstu 6y agoNPs are generally ASICs so it depends on how flexible the code needs to be that is being executed. If it gets outside of the parameters of what the ASICs can handle it can severely limit performance. An interesting side effect is a lot of the time the tools running on the main CPU don't have visibility into what is happening on the ASICs as the code doesn't have hooks into the data path at all- it compiles the code and sends it down but it doesn't participate much after it starts executing.
- boryas 6y agoI believe this piece does a good job with forwarding, but would be improved by a discussion of termination. Routing is only triggered when the packet is L2 terminated: the destination MAC of the packet is one of the router's own MACs. If the packet's destination MAC does not belong to the router, it doesn't matter what is in its IP header, it will be switched in the LAN it came in on. This design also generalizes nicely to the case when the destination IP of a routed packet is one of the router's IPs.
- anotherkamila_ 6y agoGood point. Incorporating that would require more brain that I have right now (bad timezone :D), but you're right, I completely left that out. May I update the article with a link to this comment?
- boryas 6y agosure!
- bogomipz 6y ago>"It needs to be routed: the router, based on L3 information, decides where it needs to go ,in L3 speak – it will decide which host to send it to, but not how. This corresponds to the routing table (or FIB)." This is not correct. The FIB(forwarding information base) is concerned with layer 2. The RIB(routing information base) determines the next hop. The RIB is what is used to populate entries in the FIB with the correct outgoing interface. These two terms are basic router terms. It was kind of surprising to see this statement in a post titled "How Do Routers Work, Really?"
- anotherkamila_ 6y agoYou're right, I noticed it about an hour ago -- no idea what was going on in my head then :-/ Fixed already. Thank you!
- dnautics 6y agothis is great if for no other reason that in section 1 it explains the difference between a switch and a router (which took me a decade? to really understand). I really wish someone could have laid it out clearly for me.
- icedchai 6y agoMaybe a mention of other, non-ethernet, links. Serial PPP? Frame Relay? I realize these are mostly historical curiosities these days, but it might help to enforce the differences between L2 and L3. When I first started working with routers, over 25 years ago, it was all ethernet LAN to serial WAN, usually point-to-point T1 or frame relay. On site had a dual T1, load balanced on both ports of a Cisco 2501. Fun times.
- teleforce 6y agoI teach computer networking class with lab using Linux Switch Appliance (LISA) and Quagga router (based on Zebra) on embedded computer running x86 CPU with multi-port Ethernet. The embedded router need to be dual-boot for its specific function because LISA is based on custom Linux kernel but Quagga is just using normal/vanilla kernel. I am looking for a "layer 3 switch" than has switching and routing functionalities without rebooting. If anyone know any software based open source solution for this it will be very helpful. Preferably with Cisco IOS like user command interface but it is optional but not mandatory. Based on the article, it is explaining router internal based on P4. Perhaps I should try to use P4 for the above mentioned requirements?
- wmf 6y agoVyOS supports bridging and routing although the config is more like a Linux host and unlike a real Cisco/Arista switch.
- snuxoll 6y agoThe Vyatta/VyOS/EdgeOS CLI took heavy inspiration from Juniper’s JunOS, so saying the config is unlike a “real” switch is factually incorrect. It’s still a little odd, but as somebody quite comfortable with JunOS (I run Juniper switches in my homelab) it’s pretty easy to pick up any of the Vyatta forks and hit the ground running.
- zamadatix 6y agoGNS3 and run actual vendor virtual images if you want to have the actual vendor interface, it's made for this scenario.
- mitchs 6y agoFor labbing with quagga you can get pretty far with Linux containers to emulate multiple routers on a single host. (I've used both lxc and docker to manage containers.) You can create virtual ethernet device pairs (ip link add veth0 type veth peer name veth1) , and drop either end into running containers (ip link set veth0 netns <container process ID>.) Make sure to turn on the ip forwarding sysctls inside the containers and Linux will behave quite nicely as a virtual router. Also, consider consider upgrading to the more active fork called Free Range Routing.
- mrburton 6y agoI just have to say this "magnets how do they work"? ;) Anyone get the reference?
- wbsun 6y agoClick is a very good software router to read and learn: https://github.com/kohler/click https://github.com/kohler/click It can be more than a router though.