13 ms·

The journey of an internet packet: Exploring networks with traceroute

samstave 2y ago
Tracert is more powerful than you think: There are a lot of good talks on Tracert. This one is pretty good: https://www.youtube.com/watch?v=jGYAW5z6BJc https://www.youtube.com/watch?v=jGYAW5z6BJc The article OP links to is only talking from the perspective of an internal network. Tracrt in a 10. network is boring info. In the vid I posted, he gives you really good common advice about using tracert to show you actually the physical layout of the path: https://i.imgur.com/LxN9Mr4.png https://i.imgur.com/LxN9Mr4.png <-- Using the DNS name of the router is great, because us network nerds like to use naming conventions in a graph format: so you can tell that its edge router number N at location B in City X and using tracert - you get to see the national networks the packet hits. THen by seeing the carrier, you can also see where there is not just a change in carrier, but also that indicates that at that location is a datacenter.... You can go onto DatacenterMaps and find out who/what/where a DC is.... (There is a really exceptional tech talk on tracert thats quite long that goes into bitlevel detail of weaponized tracrt - but I cant find it).... --- WRT DataCenterMaps -- There was an HNer that posted about mapping nuclear facilities (active and decommissioned) - and by using his map, the UnderSeaCableMap and the DataCenterMap - then by looking at shipping supply-chains for components used_by/made_by/received_at companies that are either Data Centers, or NVIDIA - we could track where large scale AI componentry is being installed into what data centers, which are fed by which Nuclear Power Plants, who have to report on their Consumption Graph - and which Cable Infra is likely feeding each DC. We can see where AI traffic flows - and by using tracert at a deeper level - we can see exactly the AI's Physical NeuroNet' - and find a way to measure its power consumption and physical footprint. --- HNer @externedguy "..built interactive map of active & decommissioned nuclear stations/reactors" https://news.ycombinator.com/item?id=41189056 https://news.ycombinator.com/item?id=41189056 (I correlated the Nuclear reactor locations with DataCenters, undersea cable endpoints (which will be near both nukes and datacenters) As they could be layers - and we track shipments and we can see where AI consumes: --- ...if we add the layers of the SubmarinCableMap [0] DataCenterMap [1] - and we begin to track shipments And https://i.imgur.com/zO0yz6J.png https://i.imgur.com/zO0yz6J.png -- Left is nuke, top = cables, bottom = datacenters. I went to ImportYeti to look into the NVIDIA shipments: https://i.imgur.com/k9018EC.png https://i.imgur.com/k9018EC.png And you look at the suppliers that are coming from Taiwan, such as the water-coolers and power cables to sus out where they may be shipping to, https://i.imgur.com/B5iWFQ1.png https://i.imgur.com/B5iWFQ1.png -- but instead, it would be better to find shipping lables for datacenters that are receiving containers from Taiwan, and the same suppliers as NVIDIA for things such as power cables. While the free data is out of date on ImportYeti - it gives a good supply line idea for NVIDIA... with the goal to find out which datacenters that are getting such shipments, you can begin to measure the footprint of AI as it grows, and which nuke plants they are likely powered from. Then, looking into whatever reporting one may access for the consumption/util of the nuke's capacity in various regions, we can estimate the power footprint of growing Global Compute. DataCenterNews and all sorts of datasets are available - and now the ability to create this crawler/tracker is likely full implementable https://i.imgur.com/gsM75dz.png https://i.imgur.com/gsM75dz.png https://i.imgur.com/a7nGGKh.png https://i.imgur.com/a7nGGKh.png [0] https://www.submarinecablemap.com/ https://www.submarinecablemap.com/ [1] https://www.datacentermap.com/ https://www.datacentermap.com/ ---- And 8 months back I posted: In the increasingly interconnected global economy, the reliance on Cloud Services raises questions about the national security implications of data centers. As these critical economic infrastructure sites, often strategically located underground, underwater, or in remote-cold locales, play a pivotal role, considerations arise regarding the role of military forces in safeguarding their security. While physical security measures and location obscurity provide some protection, the integration of AI into various aspects of daily life and the pervasive influence of cloud-based technologies on devices, as evident in CES GPT-enabled products, further accentuates the importance of these infrastructure sites. Notably, instances such as the seizure of a college thesis mapping communication lines in the U.S. underscore the sensitivity of disclosing key communications infrastructure. Companies like AWS, running data centers for the Department of Defense (DoD) and Intelligence Community (IC), demonstrate close collaboration between private entities and defense agencies. The question remains: are major cloud service providers actively involved in a national security strategy to protect the private internet infrastructure that underpins the global economy, or does the responsibility solely rest with individual companies? (There was talk on this topic recently in the news) EDIT: If you like this sort of networking - then courses/material like this are great: https://www.youtube.com/watch?v=Ih3KgQnT6T0 https://www.youtube.com/watch?v=Ih3KgQnT6T0 <-- network recon, scanning, countermeasures - failryl vanilla, but concise. --- I still cant find the defcon-style talk that really dives into tracert sorcery....
sulandor 2y ago
seems like clickbait traceroute uses udp to a high-port for discovery, not icmp-echo.
- gatnoodle 2y ago
  traceroute supports multiple protocols, not just UDP and ICMP.
  - sulandor 2y ago
    true, the type of packet does not really matter. my apologies if this misconstrued my point about the lacking quality of the article.
- matja 2y ago
  Windows tracert uses ICMP Source: https://support.microsoft.com/en-gb/topic/how-to-use-tracert-to-troubleshoot-tcp-ip-problems-in-windows-e643d72b-2f4f-cdd6-09a0-fd2989c7ca8e https://support.microsoft.com/en-gb/topic/how-to-use-tracert... > The TRACERT diagnostic utility determines the route to a destination by sending Internet Control Message Protocol (ICMP) echo packets to the destination
  - sulandor 2y ago
    true, most computers don't run windows though. anyways, the type of packet does not really matter.
- bc569a80a344f9c 2y ago
  I agree that this doesn’t seem to have been written by someone that actually understands the topic. Even the network diagram at the beginning is not very good. Can you create network architectures where 10.0.0.1 and 10.0.0.2 are not layer 2 adjacent? Yes, but they’re fairly complex and would imply that a lot of other necessary information is missing from the diagram. And should you use such an architecture as an example to explain traceroute? Absolutely not. It’s hard to imagine someone with even a CCNA level understanding of networking coming up with this.
dopylitty 2y ago
On the other hand traceroute isn’t real[0] 0: https://gekk.info/articles/traceroute.htm https://gekk.info/articles/traceroute.htm
- chgs 2y ago
  > It is, generally speaking, not possible to call AT&T and say "Hey, when I try to ping one of your subscribers in California from a Level3 circuit in New York, I'm hitting a routing loop." I’ve reported similar things to my isp and they’ve changed their lock prefs to send the traffic via a different peer and bypass the problem.
  - d1sxeyes 2y ago
    There are more private subscribers that pretend they know what they’re talking about than actually do. Probably makes sense to ignore a good proportion of the nonsense people throw at ISP L1 teams.
- yusyusyus 2y ago
  this article is informative but misleading. i do the testing at a big ISP (your packets go over us for at least something). we test traceroute. we complain to vendors when traceroute doesn't work right. we do investigate weird traceroutes.
  - lostemptations5 2y ago
    > your packets go over us for at least something I'm in South East Asia -- are you sure?
    - AureliusMA 2y ago
      I think he meant “us” as ISPs in general.
      - lostemptations5 2y ago
globular-toast 2y ago
Once you know how routing is supposed to work a nice homelab challenge is to set up a redundant route between your PC and the internet, like going through your NAS or something, and watch it failover when you unplug your network cable.
chgs 2y ago
It always amazes me in-depth HN gets on pretty much any subject, yet the most basic cursory network page gets massive response Is there simply a total lack of understanding of how networks work in the tech community?
- mrbluecoat 2y ago
  I'm often tasked with explaining technical concepts to people with entry-level knowledge so simple, clear documents like this are very helpful resources.
  - chgs 2y ago
    Would a “a for loop in node js” or “serving an webpage in Python” get so much attention?
    - RALaBarge 2y ago
      Make one, and lets find out
- immibis 2y ago
  People who want to learn how ISP networks work can go and join DN42, a LARPing copy of the Internet.
- StrLght 2y ago
  Networking is a black box for many developers. Why would it be any different if you're developing a CRUD application with a bunch of SREs / datacenter engineers on the line? They are very good at abstracting networking away from software engineers.
- js2 2y ago
  Yes. I've worked across a range of companies for small startups to Fortune 500s for nearly 30 years, interviewing for positions from system administrators to dev ops to programmers, beginners to architects. There's just not a lot of generalists out there. It always surprises me how far folks can get in their careers with a very narrow knowledge base. A couple years ago I was bringing a Java programmer up to speed on some C code just to learn they had no idea what a call stack was or how it worked. They were familiar with the stack as a data structure, but had no idea how a CPU worked or that it has a stack pointer register. As long as the abstractions don't leak, I guess everything is fine. I mean, I can't really tell you at a physics level exactly how semiconductors work. At best I can hand wave an explanation. That said, the lack of even hand waving knowledge about how the internet works among professionals who use it every day continues to surprise me.
wil2095 2y ago
If the destination machine has disabled ping, what response is recieved?
- dec0dedab0de 2y ago
  nothing from that machine, but the way the ttls work, it doesn’t affect the responses from routers along the way. Same if the destination doesn’t exist at all.
  - mannyv 2y ago
    Totally untrue. Network admins will often disable traceroute responses because security. Edit: the less someone knows about your internal topology the better. Security through obscurity does work.
    - ru552 2y ago
      ~~ Security through obscurity does work. As a layer.
    - Hikikomori 2y ago
      Windows disables ping by default for what I'm guessing is security.
    - dec0dedab0de 2y ago
      I was answering a question about what happens if it is disabled on the destination machine. The destination machine has no say over any other device along the route.
- teddyh 2y ago
  “ping” is ICMP ECHO_REQUEST and ICMP ECHO_RESPONSE. Traceroute uses ICMP TIME_EXCEEDED. So blocking only “ping” will not affect traceroute. And if you block all ICMP, you break your own internet: <http://shouldiblockicmp.com/ http://shouldiblockicmp.com/>
  - 2y ago
myself248 2y ago
Traceroute doesn't see 90% of the machines your packet passes through. When your packet leaves a router at some-pop-some-port-wherever, that fiber isn't usually the same piece of glass that plugs into the next hop. There's a whole chain of amplifiers and possibly multiplexers that handle it between here and there. Some of those provide reliable transport service, giving you the illusion of a fiber that never breaks, despite backhoes doing what backhoes do. Some of those shift the wavelength of your signal, letting you use cheap optics without troubling in the nuances of DWDM that packs your signal alongside dozens of others onto the same long-haul fiber. Some of those just boost the signal, along with all those others on the same fiber. But what all those machines have in common, is that none of them speak IP. None of them touch the payload. None of them are capable of decrementing a hop count. They're "part of the wire" as far as the packet is concerned. In my experience, this leads to two types of network engineers, separated by their understanding of these underlying realities.
- zamfi 2y ago
  In traceroute’s defense, it is traceroute — for sure it doesn’t tell you anything about the devices that don’t operate at the IP level. Those devices either don’t affect the IP “route” abstraction (e.g., signal boosters) or do so in ways that end up plausibly visible in the next hop. There’s a reason the network layer abstraction is so strong, and an analogy to CPU ISAs here that have a similar strength. TCP, similarly, doesn’t tell you when packets are deduplicated/resent/reordered/etc. — that’s just not part of the presented abstraction. Want that? Use UDP.
  - PeterCorless 2y ago
    "It doesn't show me the local digital loop carrier!" "Is the digital loop carrier doing any IP-level routing?" "No, but..."
- latchkey 2y ago
  > In my experience, this leads to two types of network engineers, separated by their understanding of these underlying realities. What's wrong with that? Certainly, someone with a complete picture is "better", but it is effectively two different types of problems. Do they need to be combined?
nullindividual 2y ago
This is a significantly better technical presentation on how traceroute works[0]; for example, unlike the illustrations in the linked article, traceroute does not necessarily take a symmetrical return path; the return path is hidden from the client -- the client only sees the forward path. [0] https://archive.nanog.org/sites/default/files/traceroute-2014.pdf https://archive.nanog.org/sites/default/files/traceroute-201...
- wang_li 2y ago
  Traceroute doesn't even show you a path. It shows you a bunch of devices that happened to have a packet when its TTL expired. Every item listed in traceroute's output is a different packet and can take a different path towards the destination. On a different subject, why are people writing blogs about topics that are in the "literature" already?
  - deleted 2y ago
    [deleted]
  - FujiApple 2y ago
    It’s not guaranteed to be accurate, but tracing using the UDP/dublin strategy with a fixed dest port and varying src port per round can help to identify and visualize valid ECMP flows. I recently wrote some guidance [1] on using Trippy in this way. [1] https://github.com/fujiapple852/trippy?tab=readme-ov-file#udpdublin-with-fixed-target-port-and-variable-source-port https://github.com/fujiapple852/trippy?tab=readme-ov-file#ud...
    - Hikikomori 2y ago
      Once I used iperf3 with 100 different udp streams/srcports to troubleshoot an issue, a small % of connections had >90% packet loss and this caused the connection pool of this service to fill up until it had only failed connections waiting to time out or going extremely slowly. ISP told me it was a broken linecard in a router, so packets were being dropped/corrupted on the backplane between the linecards. Traceroute and mtr didn't use enough ports to show the issue clearly.
foobiekr 2y ago
Transport networks are often mpls, SR, whatever, and this approach reveals nothing inside the SP ASes for the most part.
a-dub 2y ago
building a visualization system for traceroute in the mid 90s was how i taught myself java. traceroute itself was wrapped by a perl rpc wrapper and then a perl www cgi script would take a list of hosts and then reach out to all of them to ask them to traceroute each other, then a java applet would render an interactive graph that you could rearrange as you saw fit. interesting learning: internet routing can be asymmetric!
jerf 2y ago
I miss traceroute. It seems like more and more, the linked page is hypothetical, and what I get in practice is a couple of hops, an arbitrary number of "* * *" lines, and maybe the last host or two. It's so nice when it works. $ traceroute youtube.com traceroute to youtube.com (142.250.114.91), 30 hops max, 60 byte packets 1 10.202.10.88 (10.202.10.88) 0.384 ms 0.356 ms 0.342 ms 2 10.202.35.103 (10.202.35.103) 36.386 ms 36.370 ms 36.325 ms 3 10.202.32.4 (10.202.32.4) 36.301 ms 10.202.32.3 (10.202.32.3) 36.301 ms 10.202.32.4 (10.202.32.4) 36.287 ms 4 10.202.32.2 (10.202.32.2) 0.764 ms 1.279 ms 1.250 ms 5 lo0-0.gw2.rin1.us.linode.com (45.79.12.102) 0.610 ms lo0-0.gw1.rin1.us.linode.com (45.79.12.101) 0.663 ms 0.650 ms 6 ae62.r22.dfw01.ien.netarch.akamai.com (23.203.147.40) 0.986 ms 0.881 ms 0.837 ms 7 72.14.204.254 (72.14.204.254) 1.164 ms 72.14.198.98 (72.14.198.98) 1.153 ms 142.250.47.248 (142.250.47.248) 2.926 ms 8 * * * 9 209.85.251.24 (209.85.251.24) 1.082 ms 142.251.71.114 (142.251.71.114) 1.302 ms 0.950 ms 10 142.251.234.214 (142.251.234.214) 1.267 ms 216.239.58.16 (216.239.58.16) 3.296 ms 142.251.66.192 (142.251.66.192) 20.440 ms 11 108.170.228.86 (108.170.228.86) 1.161 ms 108.170.228.82 (108.170.228.82) 4.548 ms 108.170.228.81 (108.170.228.81) 1.746 ms 12 108.170.231.7 (108.170.231.7) 3.484 ms 108.170.229.87 (108.170.229.87) 3.071 ms 142.251.70.211 (142.251.70.211) 2.938 ms 13 142.250.236.158 (142.250.236.158) 2.298 ms 216.239.51.220 (216.239.51.220) 27.388 ms 108.170.233.60 (108.170.233.60) 2.001 ms 14 142.250.224.27 (142.250.224.27) 2.085 ms 142.250.224.25 (142.250.224.25) 1.994 ms 142.250.224.23 (142.250.224.23) 2.558 ms 15 * * * 16 * * * 17 * * * 18 * * * 19 * * * 20 * * * 21 * * * 22 * * * 23 * * * 24 rr-in-f91.1e100.net (142.250.114.91) 1.997 ms 1.981 ms 1.967 ms Just an example. There's still a lot of stuff that works, but it just seems to me more and more often that when I have an actual problem I'm going to see a lot of * * *. So many things turning the requisite packets off. (To save people excited to see if I've leaked something important some time, this comes from my Linode-hosted website's box, not my home connection.)
- spelunker 2y ago
  traceroute noob - what do the * * * mean? A host is choosing not to participate and dropping data or something?
  - 2y ago
jkerherail 2y ago
[flagged]
123sereusername 2y ago
[dead]
justinsaccount 2y ago
> But then, the path is broken, and we can’t reach the destination computer. No. This is not what this output means. People should not read more into what traceroute says than what it actually does. * * * means that traceroute did not get an ICMP time exceeded for that probe. that's it. It says nothing about if you can actually reach the destination.
deleted 2y ago
[deleted]
runjake 2y ago
These days, us network engineers are more often using mtr to explore networks. https://www.cloudflare.com/learning/network-layer/what-is-mtr/ https://www.cloudflare.com/learning/network-layer/what-is-mt...
cwilby 2y ago
Seeing the title reminded me of "Warriors of the Net". We were genuinely told, as a class, to watch this video to learn how the internet worked. https://www.youtube.com/watch?v=RhvKm0RdUY0&themeRefresh=1 https://www.youtube.com/watch?v=RhvKm0RdUY0&themeRefresh=1 Good times!
lbeckman314 2y ago
Somewhat related is this cool project by hack club [0] (not affiliated just a big fan!): how-did-i-get-here: "A tool/website/article by @kognise about how routing on the Internet works." Site: https://how-did-i-get-here.net/ https://how-did-i-get-here.net/ Github: https://github.com/hackclub/how-did-i-get-here https://github.com/hackclub/how-did-i-get-here [0] https://github.com/hackclub https://github.com/hackclub
throwaway984393 2y ago
[dead]
dkga 2y ago
Back in the day, all I wanted is to be able to do like Natalya Simyonova in Goldeneye, in the train. I recall during the late 90s there was a (Windows?) programme I knew that did this, based on traceroute/tracert I would suppose. Just a small memory to share here, maybe others had it too :)
leinelissen 2y ago
Trace route has such a nice promise of untangling the internet into its constituent parts. We used it last year in an installation where we physicalised the internet as marble run. You would create a packet for any website, and the visit all the hops one-by-one across their various locations[1]. [1]: https://youtu.be/9uIs0sh4iYU https://youtu.be/9uIs0sh4iYU
shmerl 2y ago
Better to use IPv6 in all examples. Time to get used to it instead of IPv4.
alam2000 2y ago
[dead]

Tracert is more powerful than you think: There are a lot of good talks on Tracert. This one is pretty good: https://www.youtube.com/watch?v=jGYAW5z6BJc https://www.youtube.com/watch?v=jGYAW5z6BJc The article OP links to is only talking from the perspective of an internal network. Tracrt in a 10. network is boring info. In the vid I posted, he gives you really good common advice about using tracert to show you actually the physical layout of the path: https://i.imgur.com/LxN9Mr4.png https://i.imgur.com/LxN9Mr4.png <-- Using the DNS name of the router is great, because us network nerds like to use naming conventions in a graph format: so you can tell that its edge router number N at location B in City X and using tracert - you get to see the national networks the packet hits. THen by seeing the carrier, you can also see where there is not just a change in carrier, but also that indicates that at that location is a datacenter.... You can go onto DatacenterMaps and find out who/what/where a DC is.... (There is a really exceptional tech talk on tracert thats quite long that goes into bitlevel detail of weaponized tracrt - but I cant find it).... --- WRT DataCenterMaps -- There was an HNer that posted about mapping nuclear facilities (active and decommissioned) - and by using his map, the UnderSeaCableMap and the DataCenterMap - then by looking at shipping supply-chains for components used_by/made_by/received_at companies that are either Data Centers, or NVIDIA - we could track where large scale AI componentry is being installed into what data centers, which are fed by which Nuclear Power Plants, who have to report on their Consumption Graph - and which Cable Infra is likely feeding each DC. We can see where AI traffic flows - and by using tracert at a deeper level - we can see exactly the AI's Physical NeuroNet' - and find a way to measure its power consumption and physical footprint. --- HNer @externedguy "..built interactive map of active & decommissioned nuclear stations/reactors" https://news.ycombinator.com/item?id=41189056 https://news.ycombinator.com/item?id=41189056 (I correlated the Nuclear reactor locations with DataCenters, undersea cable endpoints (which will be near both nukes and datacenters) As they could be layers - and we track shipments and we can see where AI consumes: --- ...if we add the layers of the SubmarinCableMap [0] DataCenterMap [1] - and we begin to track shipments And https://i.imgur.com/zO0yz6J.png https://i.imgur.com/zO0yz6J.png -- Left is nuke, top = cables, bottom = datacenters. I went to ImportYeti to look into the NVIDIA shipments: https://i.imgur.com/k9018EC.png https://i.imgur.com/k9018EC.png And you look at the suppliers that are coming from Taiwan, such as the water-coolers and power cables to sus out where they may be shipping to, https://i.imgur.com/B5iWFQ1.png https://i.imgur.com/B5iWFQ1.png -- but instead, it would be better to find shipping lables for datacenters that are receiving containers from Taiwan, and the same suppliers as NVIDIA for things such as power cables. While the free data is out of date on ImportYeti - it gives a good supply line idea for NVIDIA... with the goal to find out which datacenters that are getting such shipments, you can begin to measure the footprint of AI as it grows, and which nuke plants they are likely powered from. Then, looking into whatever reporting one may access for the consumption/util of the nuke's capacity in various regions, we can estimate the power footprint of growing Global Compute. DataCenterNews and all sorts of datasets are available - and now the ability to create this crawler/tracker is likely full implementable https://i.imgur.com/gsM75dz.png https://i.imgur.com/gsM75dz.png https://i.imgur.com/a7nGGKh.png https://i.imgur.com/a7nGGKh.png [0] https://www.submarinecablemap.com/ https://www.submarinecablemap.com/ [1] https://www.datacentermap.com/ https://www.datacentermap.com/ ---- And 8 months back I posted: In the increasingly interconnected global economy, the reliance on Cloud Services raises questions about the national security implications of data centers. As these critical economic infrastructure sites, often strategically located underground, underwater, or in remote-cold locales, play a pivotal role, considerations arise regarding the role of military forces in safeguarding their security. While physical security measures and location obscurity provide some protection, the integration of AI into various aspects of daily life and the pervasive influence of cloud-based technologies on devices, as evident in CES GPT-enabled products, further accentuates the importance of these infrastructure sites. Notably, instances such as the seizure of a college thesis mapping communication lines in the U.S. underscore the sensitivity of disclosing key communications infrastructure. Companies like AWS, running data centers for the Department of Defense (DoD) and Intelligence Community (IC), demonstrate close collaboration between private entities and defense agencies. The question remains: are major cloud service providers actively involved in a national security strategy to protect the private internet infrastructure that underpins the global economy, or does the responsibility solely rest with individual companies? (There was talk on this topic recently in the news) EDIT: If you like this sort of networking - then courses/material like this are great: https://www.youtube.com/watch?v=Ih3KgQnT6T0 https://www.youtube.com/watch?v=Ih3KgQnT6T0 <-- network recon, scanning, countermeasures - failryl vanilla, but concise. --- I still cant find the defcon-style talk that really dives into tracert sorcery....

sulandor 2y ago

seems like clickbait traceroute uses udp to a high-port for discovery, not icmp-echo.

gatnoodle 2y ago
traceroute supports multiple protocols, not just UDP and ICMP.
- sulandor 2y ago
  true, the type of packet does not really matter. my apologies if this misconstrued my point about the lacking quality of the article.
matja 2y ago
Windows tracert uses ICMP Source: https://support.microsoft.com/en-gb/topic/how-to-use-tracert-to-troubleshoot-tcp-ip-problems-in-windows-e643d72b-2f4f-cdd6-09a0-fd2989c7ca8e https://support.microsoft.com/en-gb/topic/how-to-use-tracert... > The TRACERT diagnostic utility determines the route to a destination by sending Internet Control Message Protocol (ICMP) echo packets to the destination
- sulandor 2y ago
  true, most computers don't run windows though. anyways, the type of packet does not really matter.
bc569a80a344f9c 2y ago
I agree that this doesn’t seem to have been written by someone that actually understands the topic. Even the network diagram at the beginning is not very good. Can you create network architectures where 10.0.0.1 and 10.0.0.2 are not layer 2 adjacent? Yes, but they’re fairly complex and would imply that a lot of other necessary information is missing from the diagram. And should you use such an architecture as an example to explain traceroute? Absolutely not. It’s hard to imagine someone with even a CCNA level understanding of networking coming up with this.

dopylitty 2y ago

On the other hand traceroute isn’t real[0] 0: https://gekk.info/articles/traceroute.htm https://gekk.info/articles/traceroute.htm

chgs 2y ago
> It is, generally speaking, not possible to call AT&T and say "Hey, when I try to ping one of your subscribers in California from a Level3 circuit in New York, I'm hitting a routing loop." I’ve reported similar things to my isp and they’ve changed their lock prefs to send the traffic via a different peer and bypass the problem.
- d1sxeyes 2y ago
  There are more private subscribers that pretend they know what they’re talking about than actually do. Probably makes sense to ignore a good proportion of the nonsense people throw at ISP L1 teams.
yusyusyus 2y ago
this article is informative but misleading. i do the testing at a big ISP (your packets go over us for at least something). we test traceroute. we complain to vendors when traceroute doesn't work right. we do investigate weird traceroutes.
- lostemptations5 2y ago
  > your packets go over us for at least something I'm in South East Asia -- are you sure?
  - AureliusMA 2y ago
    I think he meant “us” as ISPs in general.
    - lostemptations5 2y ago

globular-toast 2y ago

Once you know how routing is supposed to work a nice homelab challenge is to set up a redundant route between your PC and the internet, like going through your NAS or something, and watch it failover when you unplug your network cable.

chgs 2y ago

It always amazes me in-depth HN gets on pretty much any subject, yet the most basic cursory network page gets massive response Is there simply a total lack of understanding of how networks work in the tech community?

mrbluecoat 2y ago
I'm often tasked with explaining technical concepts to people with entry-level knowledge so simple, clear documents like this are very helpful resources.
- chgs 2y ago
  Would a “a for loop in node js” or “serving an webpage in Python” get so much attention?
  - RALaBarge 2y ago
    Make one, and lets find out
immibis 2y ago
People who want to learn how ISP networks work can go and join DN42, a LARPing copy of the Internet.
StrLght 2y ago
Networking is a black box for many developers. Why would it be any different if you're developing a CRUD application with a bunch of SREs / datacenter engineers on the line? They are very good at abstracting networking away from software engineers.
js2 2y ago
Yes. I've worked across a range of companies for small startups to Fortune 500s for nearly 30 years, interviewing for positions from system administrators to dev ops to programmers, beginners to architects. There's just not a lot of generalists out there. It always surprises me how far folks can get in their careers with a very narrow knowledge base. A couple years ago I was bringing a Java programmer up to speed on some C code just to learn they had no idea what a call stack was or how it worked. They were familiar with the stack as a data structure, but had no idea how a CPU worked or that it has a stack pointer register. As long as the abstractions don't leak, I guess everything is fine. I mean, I can't really tell you at a physics level exactly how semiconductors work. At best I can hand wave an explanation. That said, the lack of even hand waving knowledge about how the internet works among professionals who use it every day continues to surprise me.

wil2095 2y ago

If the destination machine has disabled ping, what response is recieved?

dec0dedab0de 2y ago
nothing from that machine, but the way the ttls work, it doesn’t affect the responses from routers along the way. Same if the destination doesn’t exist at all.
- mannyv 2y ago
  Totally untrue. Network admins will often disable traceroute responses because security. Edit: the less someone knows about your internal topology the better. Security through obscurity does work.
  - ru552 2y ago
    ~~ Security through obscurity does work. As a layer.
  - Hikikomori 2y ago
    Windows disables ping by default for what I'm guessing is security.
  - dec0dedab0de 2y ago
    I was answering a question about what happens if it is disabled on the destination machine. The destination machine has no say over any other device along the route.
teddyh 2y ago
“ping” is ICMP ECHO_REQUEST and ICMP ECHO_RESPONSE. Traceroute uses ICMP TIME_EXCEEDED. So blocking only “ping” will not affect traceroute. And if you block all ICMP, you break your own internet: <http://shouldiblockicmp.com/ http://shouldiblockicmp.com/>
- 2y ago

myself248 2y ago

Traceroute doesn't see 90% of the machines your packet passes through. When your packet leaves a router at some-pop-some-port-wherever, that fiber isn't usually the same piece of glass that plugs into the next hop. There's a whole chain of amplifiers and possibly multiplexers that handle it between here and there. Some of those provide reliable transport service, giving you the illusion of a fiber that never breaks, despite backhoes doing what backhoes do. Some of those shift the wavelength of your signal, letting you use cheap optics without troubling in the nuances of DWDM that packs your signal alongside dozens of others onto the same long-haul fiber. Some of those just boost the signal, along with all those others on the same fiber. But what all those machines have in common, is that none of them speak IP. None of them touch the payload. None of them are capable of decrementing a hop count. They're "part of the wire" as far as the packet is concerned. In my experience, this leads to two types of network engineers, separated by their understanding of these underlying realities.

zamfi 2y ago
In traceroute’s defense, it is traceroute — for sure it doesn’t tell you anything about the devices that don’t operate at the IP level. Those devices either don’t affect the IP “route” abstraction (e.g., signal boosters) or do so in ways that end up plausibly visible in the next hop. There’s a reason the network layer abstraction is so strong, and an analogy to CPU ISAs here that have a similar strength. TCP, similarly, doesn’t tell you when packets are deduplicated/resent/reordered/etc. — that’s just not part of the presented abstraction. Want that? Use UDP.
- PeterCorless 2y ago
  "It doesn't show me the local digital loop carrier!" "Is the digital loop carrier doing any IP-level routing?" "No, but..."
latchkey 2y ago
> In my experience, this leads to two types of network engineers, separated by their understanding of these underlying realities. What's wrong with that? Certainly, someone with a complete picture is "better", but it is effectively two different types of problems. Do they need to be combined?

nullindividual 2y ago

This is a significantly better technical presentation on how traceroute works[0]; for example, unlike the illustrations in the linked article, traceroute does not necessarily take a symmetrical return path; the return path is hidden from the client -- the client only sees the forward path. [0] https://archive.nanog.org/sites/default/files/traceroute-2014.pdf https://archive.nanog.org/sites/default/files/traceroute-201...

wang_li 2y ago
Traceroute doesn't even show you a path. It shows you a bunch of devices that happened to have a packet when its TTL expired. Every item listed in traceroute's output is a different packet and can take a different path towards the destination. On a different subject, why are people writing blogs about topics that are in the "literature" already?
- deleted 2y ago
  [deleted]
- FujiApple 2y ago
  It’s not guaranteed to be accurate, but tracing using the UDP/dublin strategy with a fixed dest port and varying src port per round can help to identify and visualize valid ECMP flows. I recently wrote some guidance [1] on using Trippy in this way. [1] https://github.com/fujiapple852/trippy?tab=readme-ov-file#udpdublin-with-fixed-target-port-and-variable-source-port https://github.com/fujiapple852/trippy?tab=readme-ov-file#ud...
  - Hikikomori 2y ago
    Once I used iperf3 with 100 different udp streams/srcports to troubleshoot an issue, a small % of connections had >90% packet loss and this caused the connection pool of this service to fill up until it had only failed connections waiting to time out or going extremely slowly. ISP told me it was a broken linecard in a router, so packets were being dropped/corrupted on the backplane between the linecards. Traceroute and mtr didn't use enough ports to show the issue clearly.

foobiekr 2y ago

Transport networks are often mpls, SR, whatever, and this approach reveals nothing inside the SP ASes for the most part.

a-dub 2y ago

building a visualization system for traceroute in the mid 90s was how i taught myself java. traceroute itself was wrapped by a perl rpc wrapper and then a perl www cgi script would take a list of hosts and then reach out to all of them to ask them to traceroute each other, then a java applet would render an interactive graph that you could rearrange as you saw fit. interesting learning: internet routing can be asymmetric!

jerf 2y ago

I miss traceroute. It seems like more and more, the linked page is hypothetical, and what I get in practice is a couple of hops, an arbitrary number of "* * *" lines, and maybe the last host or two. It's so nice when it works. $ traceroute youtube.com traceroute to youtube.com (142.250.114.91), 30 hops max, 60 byte packets 1 10.202.10.88 (10.202.10.88) 0.384 ms 0.356 ms 0.342 ms 2 10.202.35.103 (10.202.35.103) 36.386 ms 36.370 ms 36.325 ms 3 10.202.32.4 (10.202.32.4) 36.301 ms 10.202.32.3 (10.202.32.3) 36.301 ms 10.202.32.4 (10.202.32.4) 36.287 ms 4 10.202.32.2 (10.202.32.2) 0.764 ms 1.279 ms 1.250 ms 5 lo0-0.gw2.rin1.us.linode.com (45.79.12.102) 0.610 ms lo0-0.gw1.rin1.us.linode.com (45.79.12.101) 0.663 ms 0.650 ms 6 ae62.r22.dfw01.ien.netarch.akamai.com (23.203.147.40) 0.986 ms 0.881 ms 0.837 ms 7 72.14.204.254 (72.14.204.254) 1.164 ms 72.14.198.98 (72.14.198.98) 1.153 ms 142.250.47.248 (142.250.47.248) 2.926 ms 8 * * * 9 209.85.251.24 (209.85.251.24) 1.082 ms 142.251.71.114 (142.251.71.114) 1.302 ms 0.950 ms 10 142.251.234.214 (142.251.234.214) 1.267 ms 216.239.58.16 (216.239.58.16) 3.296 ms 142.251.66.192 (142.251.66.192) 20.440 ms 11 108.170.228.86 (108.170.228.86) 1.161 ms 108.170.228.82 (108.170.228.82) 4.548 ms 108.170.228.81 (108.170.228.81) 1.746 ms 12 108.170.231.7 (108.170.231.7) 3.484 ms 108.170.229.87 (108.170.229.87) 3.071 ms 142.251.70.211 (142.251.70.211) 2.938 ms 13 142.250.236.158 (142.250.236.158) 2.298 ms 216.239.51.220 (216.239.51.220) 27.388 ms 108.170.233.60 (108.170.233.60) 2.001 ms 14 142.250.224.27 (142.250.224.27) 2.085 ms 142.250.224.25 (142.250.224.25) 1.994 ms 142.250.224.23 (142.250.224.23) 2.558 ms 15 * * * 16 * * * 17 * * * 18 * * * 19 * * * 20 * * * 21 * * * 22 * * * 23 * * * 24 rr-in-f91.1e100.net (142.250.114.91) 1.997 ms 1.981 ms 1.967 ms Just an example. There's still a lot of stuff that works, but it just seems to me more and more often that when I have an actual problem I'm going to see a lot of * * *. So many things turning the requisite packets off. (To save people excited to see if I've leaked something important some time, this comes from my Linode-hosted website's box, not my home connection.)

spelunker 2y ago
traceroute noob - what do the * * * mean? A host is choosing not to participate and dropping data or something?
- 2y ago

jkerherail 2y ago

[flagged]

123sereusername 2y ago

[dead]

justinsaccount 2y ago

> But then, the path is broken, and we can’t reach the destination computer. No. This is not what this output means. People should not read more into what traceroute says than what it actually does. * * * means that traceroute did not get an ICMP time exceeded for that probe. that's it. It says nothing about if you can actually reach the destination.

deleted 2y ago

[deleted]

runjake 2y ago

These days, us network engineers are more often using mtr to explore networks. https://www.cloudflare.com/learning/network-layer/what-is-mtr/ https://www.cloudflare.com/learning/network-layer/what-is-mt...

cwilby 2y ago

Seeing the title reminded me of "Warriors of the Net". We were genuinely told, as a class, to watch this video to learn how the internet worked. https://www.youtube.com/watch?v=RhvKm0RdUY0&themeRefresh=1 https://www.youtube.com/watch?v=RhvKm0RdUY0&themeRefresh=1 Good times!

lbeckman314 2y ago

Somewhat related is this cool project by hack club [0] (not affiliated just a big fan!): how-did-i-get-here: "A tool/website/article by @kognise about how routing on the Internet works." Site: https://how-did-i-get-here.net/ https://how-did-i-get-here.net/ Github: https://github.com/hackclub/how-did-i-get-here https://github.com/hackclub/how-did-i-get-here [0] https://github.com/hackclub https://github.com/hackclub

throwaway984393 2y ago

[dead]

dkga 2y ago

Back in the day, all I wanted is to be able to do like Natalya Simyonova in Goldeneye, in the train. I recall during the late 90s there was a (Windows?) programme I knew that did this, based on traceroute/tracert I would suppose. Just a small memory to share here, maybe others had it too :)

leinelissen 2y ago

Trace route has such a nice promise of untangling the internet into its constituent parts. We used it last year in an installation where we physicalised the internet as marble run. You would create a packet for any website, and the visit all the hops one-by-one across their various locations[1]. [1]: https://youtu.be/9uIs0sh4iYU https://youtu.be/9uIs0sh4iYU

shmerl 2y ago

Better to use IPv6 in all examples. Time to get used to it instead of IPv4.

alam2000 2y ago

[dead]