6 ms·
Hi! I'm Lexi. I'm 17, and one of the things I'm interested in right now is gaining a deeper understanding of how computers work and showing that in new ways. A
by archmaster 3y ago
Hi! I'm Lexi. I'm 17, and one of the things I'm interested in right now is gaining a deeper understanding of how computers work and showing that in new ways.
A few months ago I published https://cpu.land https://cpu.land (discussion: https://news.ycombinator.com/item?id=37062422 https://news.ycombinator.com/item?id=37062422). After cpu.land, I felt a lot of pressure to make another Big Giant Thing but didn't really have anything compelling. So I just hacked away on personal projects and, through some coincidental learning on how the Internet works, ended up hacking together a traceroute program that could live stream to a website from scratch!
I realized I had never seen this sort of thing on the web before, and it was actually a kind of cool and novel way of visualizing the structure of the Internet, so I polished it up and built a pretty site around it. In the process, I learned some really interesting things about how BGP and the structure of The Internet, so I melted the traceroute tool with an article sharing that knowledge.
I'm still hacking on this and I'm sure my code will manage to break somehow, so please let me know if you have any suggestions! :)
(Side note: why Rust? I don’t think programming language choice matters that much, but I wanted to quickly write a very dependable low-level program, and I really like Rust’s error handling primitives.
Why do you care about this?)
- glandium 3y ago> I felt a lot of pressure to make another Big Giant Thing but didn't really have anything compelling. Easier said than done, but don't feel like you have to provide a constant flux of interesting things. That kind of pressure ends up being toxic pretty quickly. Do what you enjoy, if it hits an audience, great, but don't feel like you have to make it happen.
- r3trohack3r 3y agoThis is absolutely amazing <3 The narrative based traceroute in green is something I’ve never seen before. How many providers, like CDNs, did you take the time to map into a narrative? This feels targeted towards folks who kind of already understand computers. It be cool to repackage this in a way that can show non-technical people the stuff they take for granted. The mountains that move on a user’s behalf under every keystroke is humbling. One of my favorite books on this topic is Interconnections: Bridges, Routers, Switches, and Interconnections by Radia Perlman if you haven’t come across it.
- zX41ZdbW 3y agoI'm also interested in this topic (since I was 17 and didn't lose interest in twenty years). Recently, I've created a hobby project - the atlas of the Internet: https://reversedns.space/ https://reversedns.space/ I'm posting here because it might be interesting for you. How it was built: https://presentations.clickhouse.com/meetup85/app/index.html https://presentations.clickhouse.com/meetup85/app/index.html
- Multicomp 3y agoGood work, a nice site explaining the traceroute is very handy! I could see myself using this to explain what traceroute is for various technicians I'm training to maintain corporate networks. (Now if only I could figure out how to enable traceroute to work on each hop from a given workstation through corporate cisco access switch, core switch, BGP tunnel to aws transit gateways, and eventually land at the VPC route table on the EC2 instance, then i might actually be able to call myself a network guy)
- bongodongobob 3y agoYou can't use traceroute for switching as it operates on layer 3. Switches use MAC addresses, not IP. Switches are essentially transparent as they just forward traffic, they don't route.
- epistasis 3y agoThe switch and router terminology distinction has become fairly meaningless at enterprise level hardware as discussed in that comment. The same corporate access or top-of-rack "switch" is just as capable of layer 3 routing as it of layer 2 network handling. This happened because the desire to expand a layer 2 fabric across multiple physical switches basically reinvented a lot of layer 3 concepts on layer 2. The silicon is the same for both, just different algorithms.
- dfox 3y agoThere are multiple motivations and resulting technologies there: - “L3 Switches”, which in essence are routers that do not have discrete ports and instead have closely coupled switch. Alternatively the same thing can be viewed as switch with powerful enough control-plane CPU that it can do routing (pure L2 managed switches usually have ridiculously underpowered CPUs). This can be and often is combined with some amount of offloading the routing and even more complex upper-layer processing into hardware. - things like Shortest Path Bridging, which uses L3-style routing protocol (IIRC it is IS-IS) in order to build L2 FIB tables for L2 switches. The idea there is to not only have (R)STP-like redundant paths for ethernet, but to use full capacity of such paths when it is available. On the hardware level it is mostly about accelerating the fast-path, where the hardware FIB contains some cache of bit patterns seen in the received frame, where it should be forwarded and maybe how it should be rewritten. And packets that are not matched by HW FIB are passed to the CPU to be processed. In a “dumb” bridge (ie. L2 switch), such frames would be simply broadcasted to all ports. While the general idea is the same, there is material difference in the silicon complexity between matching destination MAC of ethernet frame and matching arbitrary bit patterns somewhere in TCP header of IPv6 packet.
- ta988 3y agoI found about "Lying for Money" on your website a few months ago when you posted your CPU thing. Thanks that was totally worth a read. I recommend you "When Mckinsey comes to town" (the book) in a similar vein, that was entertaining in human mischief as well.
- ReactiveJelly 3y agoCool! I was nervous it was gonna show my Tor circuit o_o Unfortunately so many nodes ignore traceroute packets that it basically said my exit node connected to Linode and then Linode connected to your computer. I have the same experience with forward traceroutes, my router replies, my server replies, and if I'm lucky, one node in my ISP's network. The rest is locked up tight.
- archmaster 3y ago(Part of this might be that all this HN traffic is overwhelming my tiny server a bit, I'm working on getting it smoother live rn)
- mring33621 3y agoWould love an update for what you had to do to make it "smoother" under the HN load!
- dfox 3y agoTor works on the level of circuits and from the inside of the circuit has nothing to do with IP. The exit node is the first node where that is converted to an actual TCP connection. This among other things means that the only protocol on the public internet that works over Tor is TCP.
- solardev 3y agoHey Lexi, Total side note, your other post (about cpu.land) is at exactly 1337 upvotes now :D https://share.cleanshot.com/ktVWL2pr https://share.cleanshot.com/ktVWL2pr How fitting.
- archmaster 3y agoI KNOW, right?! It's perfect.
- karlzt 3y agoWhy link to that instead of: https://news.ycombinator.com/item?id=37062422 https://news.ycombinator.com/item?id=37062422 I'm 17 and wrote this guide on how CPUs run programs (3 months ago)
- solardev 3y agoThe OP already linked to that, and it looks like it has still an up vote button? Couldn't the count go up?
- RSMDZ 3y agoYou are working on a lot more cool things than I was when I was 17. Rock on! If you enjoyed "I, Robot" I recommend Ted Chiang's "Stories of Your Life and Others". The movie Arrival was based of one of the short stories in this collection.
- archmaster 3y agoOh, thank you, I'll buy a copy! I thought Arrival was a fun movie
- solardev 3y agoSeconding that... Ted Chiang is one of my favorite authors! His short stories are a really fun mix of technology/sociology/religion/history, and each one only takes like half an hour to read.
- networkchad 3y ago[dead]
- forgotpwd16 3y agoBoth submitted and previous are cool and very well made. Grants!
- animal_spirits 3y agoKeep up the awesome work, Lexi. You are very skilled and I can tell that you will rise very quickly as an engineer and a teacher
- sonicanatidae 3y agoMost of the really cool things on the interwebs started just like this. Kudos and carry on!
- farmeroy 3y agoYour projects are super inspiring!
- zerojames 3y agoThis is incredible, Lexi. I have had a lot of fun learning about the low levels of networking; there are so many interesting details. Documenting those — and making them so intuitive, like you do here — is amazing. Keep up the amazing work!
- DylanSp 3y agoCool project! Two suggestions, one serious, one frivolous: - I wonder if you could get more accurate results by using TCP or UDP instead of ICMP. I think traditional traceroute has an option to use UDP, mtr [1] can use TCP or UDP, and tcptraceroute [2] can use TCP. - This would be a perfect fit for some Talking Heads references. "And you may ask yourself, well, how did I get here?" [3] [1] https://github.com/traviscross/mtr https://github.com/traviscross/mtr [2] https://linux.die.net/man/1/tcptraceroute https://linux.die.net/man/1/tcptraceroute [3] https://en.wikipedia.org/wiki/Once_in_a_Lifetime_(Talking_Heads_song) https://en.wikipedia.org/wiki/Once_in_a_Lifetime_(Talking_He...
- archmaster 3y ago1. ICMP seemed a lot faster / more reliable, but you're right that UDP does sometimes bring out better results. Something I want to do is make it use a hybrid approach — sending both ICMP and UDP packets and using the best of both. 2. Wayy ahead of you, check for HTML comments :))
- nullindividual 3y agoICMP isn't as reliable for trace routes. ICMP typically flows through the management plane instead of the data plane. The management plane is often driven by low-power MIPS processors, which can appear as additional latency in the traceroute results. When a network device is busy, ICMP may be dropped entirely. ICMP is a great tool to establish baseline connectivity, assuming the device responds to it. TCP will provide more accurate results (or UDP given the device responds to UDP packets) if you know a specific port is open. traceroute uses UDP by default. tracert.exe only uses ICMP.
- bwann 3y agoTraffic to vs through a router; both UDP and ICMP probes to a given hop will both go to the control plane which can be rate limited or handled by a general purpose CPU. Probes to hops beyond will virtually always go through the fast data plane.
- 3y ago
- js2 3y ago> I realized I had never seen this sort of thing on the web before. Search for: "looking glass bgp" and you'll find some[1]. One of the first CGI programs I wrote nearly three decades ago (ugh...) was a Perl script that wrapped traceroute and streamed the results via server push[2]. Everything old is new again. :-) That said, your site has a very nice presentation. BTW, ipv4 TTL is dejure seconds even though it's defacto hop count since no router takes more than a second and the minimum decrement is 1 (except middleboxes which wish to remain hidden won't decrement at all). Also, Linux/Unix traceroute by default use UDP to a high numbered (and usually closed) port for probe packets instead since UDP historically is less likely to be dropped/filtered than ICMP. Aside: asking how traceroute works is one of my interview questions, most people don't know (if they do the question is no good) and many are unable to figure it out from first principles no matter how many questions I answer about TCP/IP. I still think being able to figure it out is a reasonable problem solving question. 1. e.g. https://www.bgplookingglass.com/ https://www.bgplookingglass.com/ 2. https://www.oreilly.com/openbook/cgi/ch06_06.html https://www.oreilly.com/openbook/cgi/ch06_06.html
- emmelaich 3y agoIs that still true? https://www.rfc-editor.org/rfc/rfc3443.txt https://www.rfc-editor.org/rfc/rfc3443.txt has no mention of seconds. It just says that outputTTL is (inputTTL - 1). With some exceptions. [edit: I missed that that RFC is for MPLS but would be interested in your comment anyway; the definitive version seems to be https://datatracker.ietf.org/doc/html/rfc1122 https://datatracker.ietf.org/doc/html/rfc1122]
- js2 3y agoRFC 791: > The time is measured in units of seconds, but since every module that processes a datagram must decrease the TTL by at least one even if it process the datagram in less than a second, the TTL must be thought of only as an upper bound on the time a datagram may exist. https://www.rfc-editor.org/rfc/rfc791.html https://www.rfc-editor.org/rfc/rfc791.html The equivalent field in IPv6 is named hop limit in recognition of how the TTL field is used in practice with IPv4: https://www.rfc-editor.org/rfc/rfc2460.html https://www.rfc-editor.org/rfc/rfc2460.html
- foobarian 3y agoSince you're in the weeds with traceroute you might appreciate this one (if you haven't seen it already): traceroute bad.horse
- CSMastermind 3y agoYou used to be able to get the Star Wars via traceroute or telnet: https://itsfoss.com/star-wars-linux/ https://itsfoss.com/star-wars-linux/ But when I tested just now it didn't work for me so your milage may vary.
- archmaster 3y ago+1, love it
- herodotus 3y agoIs there music that goes along with these lyrics?
- archmaster 3y agohttps://www.youtube.com/watch?v=F7GDaLijr1w https://www.youtube.com/watch?v=F7GDaLijr1w
- dylan604 3y agookay, i had never seen this command used, but i got extremely lucky by it running pretty much in sync with this video. If I had tried to get it in sync, I couldn't have gotten any closer. sometimes, the internet smiles upon you and just gives you a moment. thanks for the smile to all in this thread from here up!
- aidenn0 3y agoif you don't have traceroute: tracepath -m128 bad.horse works just as well.
- mgkimsal 3y agoGreat job! I was reminded of working at a company in 1996... we had windows 95(!) with Trumpet WinSock and a dial-up modem (24k, IIRC). I was just learning how all this stuff worked and fit together. I stumbled on a traceroute screen that would slowly drip out each hop and... it was magical to me. Suddenly realizing the idea of 'a big global network' I'd read about was actually... right at my fingertips, and I could see which computers my traffic was being routed through... that kept me up at night for a while. Not sure I'll say it was life-changing, but it sort of felt like it for a bit at that time :)
- globular-toast 3y agoNice. I'm wearing black jeans and I'm always surprised by how little developers know about basic networking. I was always interested in it but also learnt about it at uni. I don't remember all the details, of course, but just generally knowing what routing is, what IP is etc. pays dividends when building anything that runs on networks/internet.
- lanewinfield 3y agoNice work, Lexi. Let me add that your design skills are pretty excellent as well.
- readyplayeremma 3y agoReally awesome project! A technical nitpick though: Routes can be asymmetric—going across one path in one direction and another for the opposite. This means that your tool potentially doesn't show the route packets from the user took to reach your server, but rather the route packets took from your server to reach the user. I believe that querying with BGP looking glass tools would allow you to construct the route in either direction, but it is maybe a bit less cool looking than the real-time traceroute that is a result of actual traffic.