6 ms·
One of my favorite revelations about the network tracing tools (things like `traceroute` and `dig +trace`) that might not be obvious for people like me who work
by eat_veggies 6y ago
One of my favorite revelations about the network tracing tools (things like `traceroute` and `dig +trace`) that might not be obvious for people like me who work higher up in the stack, is that the data they provide isn't usually made available during "normal" usage. Packets don't just phone home and tell you where they've been. Something else is going on.
When you send a DNS query to a recursive server like your ISP's or something like 1.1.1.1, you make a single DNS query and get back a single response, because the recursive DNS server handles all the different queries that Julia outlines in the post. As the client, we have no idea what steps just happened in the background.
But when you run `dig +trace`, dig is actually pretending to be a recursive name server, and making all those queries itself instead of letting the real recursive name servers do their work. It's a fun hack but that means it's not always 100% accurate to what's going on in the real world [0]
[0] https://serverfault.com/questions/482913/is-dig-trace-always-accurate https://serverfault.com/questions/482913/is-dig-trace-always...
- nijave 6y agoYup, and to complicate matters more those resolvers you're talking to may be talking to more caching resolvers. For a given application server it might be: - check local dns caching resolver - check local network caching resolver - if internal domain, check local authoritative resolver - if public domain check isp resolver - recursively resolve from there
- fragmede 6y agoIn particular, one your ISPs/their ISP's DNS servers may be caching a record for longer than it's supposed to and will return incorrect and expired data. The other possibility is different IP's being returned by a DNS server based on where a query is coming from, eg a CDN. If you're in location A and your ISPs DNS server is in location B, the CDN's DNS server may return a different IP based on if the request is coming from A or B. ECS [0] is supposed to mitigate this, but may or may not be used. [0] https://en.wikipedia.org/wiki/EDNS_Client_Subnet https://en.wikipedia.org/wiki/EDNS_Client_Subnet
- qes 6y ago> In particular, one your ISPs/their ISP's DNS servers may be caching a record for longer than it's supposed to and will return incorrect and expired data. It's disturbing how many clients we'll see hitting an old IP address for 30 days after a change.
- dgl 6y agodig +trace takes one path, there’s also tools like dnstrace that attempt to show all the paths: https://github.com/rs/dnstrace https://github.com/rs/dnstrace Still there can be caches that don’t quite agree as the other comment mentions.
- LogicX 6y agoJust to add to the discussion -- 'whats happening in the background' -- more specifically is your operating system's stub resolver. So when you ask for www.amazon.com it ends up making multiple DNS lookups, as www.amazon.com is a CNAME record. Nothing about this CNAME lookup gets passed back up the stack to your application; you just get that end-result: the IP address. host www.amazon.com www.amazon.com is an alias for tp.47cf2c8c9-frontier.amazon.com. tp.47cf2c8c9-frontier.amazon.com is an alias for www.amazon.com.edgekey.net. www.amazon.com.edgekey.net is an alias for e15316.e22.akamaiedge.net. e15316.e22.akamaiedge.net has address 23.204.68.114
- chrisweekly 6y agoAm I the only one who finds it strange that Amazon uses Akamai instead of Cloudfront?
- _jal 6y agoWhy is that? (My first thought was, wow, surprised they outsource that.)
- blyry 6y agoPure speculation, but probably the same reason they ran Oracle for so long -- not many cdn providers when amazon.com launched. Afaik akamai still has one of the biggest edge networks of all the providers.
- Legogris 6y agoI'd wager at least part of it is the same reason many companies use CloudFront: Separate point of failure in case of downtime at the AWS side, some one else shouldering DDoS.
- MaxBarraclough 6y agoI asked the same question 7 days ago: https://news.ycombinator.com/item?id=23537135 https://news.ycombinator.com/item?id=23537135
- 6y ago
- closeparen 6y agoTraceroute’s trick is amusing. It abuses the TTL field, sending out packets with too-low TTLs and waiting to see who complains about them. When layers reveal themselves they are doing it voluntarily, and those wise to the game can choose not to participate, or to troll it. https://www.theregister.com/2013/02/15/star_wars_traceroute/ https://www.theregister.com/2013/02/15/star_wars_traceroute/
- m3047 6y agoThose are IP packet TTLs. Also useful in some cases for diagnosing mysterious RSTs and SYNs. Also useful in some cases to purposefully set low if there is route flapping.
- Tepix 6y agoIt's not abusing the TTL, it's working as designed
- closeparen 6y agoAFAIK the design purpose of TTL is to prevent an infinite loop in case the route contains a cycle.
- allarm 6y agoThe IP TTL field is unrelated to the topic - DNS and IP are sitting on the different OSI layers. Also, the traceroute command does not abuse this field, it’s just the way the IP protocol works. I.e. eBGP sets TTL to 1 by default, to prevent establishing relationships with the neighbors further that one hop away.
- m3047 6y agoMy favorite dig trick is asking caching resolvers with +norecurse to see what's in the cache. Of course some resolvers don't honor it, just like they don't support ANY.
- inamberclad 6y agoI thought DNS was a UDP protocol where you fire off a packet and use the first response that comes back. It's not always one to one.
- eat_veggies 6y agoUDP is stateless but DNS is definitely 1:1 in that there is a notion of a query and its reply. This relationship is made even clearer when you consider DoH.