7 ms·

What I learned from making a DNS client in Rust

145 points by adamch 5y ago

tedunangst 5y ago
All the links (beej, const generics) seem to be relative to the post and don't resolve.
- adamch 5y ago
  Oops, thanks so much! I forgot to check 'em. Fixed now.
eminence32 5y ago
> The other reason I wanted to make a DNS client is that I knew I could simplify every step using some great Rust crates This is one of the reasons I rather like rust and it's ecosystem: you can delegate "boring" functionality (aka the stuff you need, but aren't interested in writing) to the ecosystem and just focus on the stuff you want to. Other languages also have a similarly rich ecosystem (python in particular). But compared to python, I find that rust is slightly better; 2 reasons come to mind: * Docs in rust seem to be slightly better. I think this is partly due to the generally high quality of docs.rs, so crate authors seem to have settled on the idea that some minimal baseline docs are expected. * Error handling in rust feels better. In the python world, I would see None returned some library functions that didn't really document this possibility, yielding an uncomfortable number of runtime TypeErrors or AttributeErrors due to an unexpected NoneType
- vlovich123 5y ago
  Package management is also drastically simplified because cargo is standard vs you have to basically do everything from scratch in Python land
  - eminence32 5y ago
    As a consumer, pip is pretty good though
    - vlovich123 5y ago
      Pip isn’t suitable for development. Having a global shared package installation doesn’t work if working on separate codebases. It also doesn’t work because you get implicit dev dependencies and then how do you share those when you want to collaborate? Requirements.txt is a laugh for several reasons. Pip-compile and then installing that into a virtualenv is a neat solution that I found. Not sure if there’s anything meaningfully better yet.
  - SnowHill9902 5y ago
    Is cargo more vulnerable to supply chain attacks than downloading a C++ lib and compiling?
    - hamandcheese 5y ago
      I’m not a big Rust user, but my understanding is that Cargo.lock contains checksums. So the answer to your question would be no. Downloading a C++ lib and using Cargo are both effectively trust on first use.
    - eminence32 5y ago
      rust might be more vulnerable in practice because it's so easy to add 3rd-party code (and update 3rd-party code).
      - carlmr 5y ago
        Exactly. In C++ everybody just writes their own libraries which enable other kinds of bugs and vulnerabilities.
- deepsun 5y ago
  Java ecosystem is a quintessence of delegating boring stuff. But without wild westness of NPM, libraries (aka crates) feel more "serious".
  - jeroenhd 5y ago
    In my experience the Java world tries to look more serious with all dependencies having fancy corporate names, but the end result is exactly the same. Log4j proved to the world that Java has as much of a dependency tree hell problem as any other programming language. What I find annoying about Java is that very major dependencies are either slow-moving, making upgrading to a new Java version quite a challenge, or require constant rewrites with new releases. I've come to expect that upgrading dependencies usually means "the code works except for these methods being a bit different" but in Java libraries the entire API can change on a whim. The only thing I find "serious" about it is that everyone seems to run ancient versions of Java because their dependencies are impossible to upgrade. It's no wonder that Java 8 seems to be as popular as ever despite the two LTS releases since.
danamit 5y ago
I made one in JS years ago, what I learned is that DNS is simpler than I thought.
db48x 5y ago
Nice write–up. I happened to write a little dns client last month too, although I avoided all the low–level protocol stuff and concentrated on the high–level business logic: pull many rows from a database, then create and execute as many queries in parallel as possible. It’s a good demonstration of why I like Rust so much. All the pieces were already there, and Rust’s type system ensures that I cannot have any unexpected sources of errors. My error–handling code is honestly pretty crude, but the compiler ensures that I have actually handled them all, and that there can be no surprises. No innocent–looking library functions that – surprise! – can actually throw exceptions.
mike_d 5y ago
I encourage everyone to write a DNS implementation of some sorts. You quickly learn two things: 1) DNS is pretty simple. 2) handling all the edge cases and bugs in the hundreds of other implementations is not.
- deleted 5y ago
  [deleted]
ccouzens 5y ago
I made one too https://github.com/ccouzens/dns-packet https://github.com/ccouzens/dns-packet The differences: I followed this guide rather than the RFCs https://github.com/EmilHernvall/dnsguide/blob/master/chapter1.md https://github.com/EmilHernvall/dnsguide/blob/master/chapter... Mine isn't as polished. The command line parsing and output is more thrown together. I hardcoded the packet identifier (it's not production code, and I'm only looking up one at a time). I didn't use any bit manipulation libraries. I can see they would help because DNS packets don't line up their information with the byte boundaries.
iforgotpassword 5y ago
I once added ipv6 capabilities to a very old windows software. It turned out that before Windows 8, there was no way to asynchronously resolve an AAAA record in a single threaded app. For the good old A records, you could ask Windows to resolve a hostname and it would send you a window message when it was done. So after fighting with multithreading in this archane app - I forgot what exactly the problems were - I wrote a very simple resolver that would only be able to handle AAAA lookups. It really is surprisingly simple. I think the more involved part was googling how to retrieve the system's configured DNS servers. ;)
outofmyshed 5y ago
nslookup and host both exist. It’s interesting that the author went straight into a programming challenge rather than looking for another program that may already exist on their system and would solve their dig-hate problem.
- adamch 5y ago
  I was aware that other DNS clients existed, but just using one of them would have taught me as much!
fanf2 5y ago
The first thing I check for in new DNS code is how the name decompression works, because it is a surprisingly subtle part of the DNS: you have to have a way to stop chasing compression pointers forever, either by limiting the number of pointers you will follow, or by requiring them to point earlier in the message. It is very common to find DNS code that has a denial of service vulnerability in its name decompression; it is one of the best examples of a packet parsing security vulnerability that is not about buffer overflows.
- spc476 5y ago
  When I wrote my DNS parser [1], I set a follow limit. I'm not sure how viable it would be to limit the pointers to earlier in the packet would be. [1] https://github.com/spc476/SPCDNS https://github.com/spc476/SPCDNS
  - ate53 5y ago
    It's a viable strategy. No legitimate implementation produces compression pointers that go anywhere other than backwards. I track the start of a label sequence and whether I've followed a pointer. If I've just followed a pointer, I must be at a point earlier than the last start, or I consider it a malicious name. I also disallow pointers to pointers but I'm less confident that there's not something out there that does that.
    - spc476 5y ago
      So, using () as a marker for the length of a label, and (^name) for a label pointer, how do you handle ()a()b(^b) The pointer goes backwards, but it doesn't go back to the start of a label sequence. So do you track the start of each label? Or just the start of each domain name? Edit: nevermind, I see what you are doing.
  - acuozzo 5y ago
    One thing I noticed while using your code as a reference for a project I worked on several weeks ago is that it doesn't record the state before following the pointer, so unless the pointer you follow eventually leads to another pointer that brings it back to where you were, you can't simulate recursion. I'm referring to lines 1286-1288 in codec.c, FWIW. Obviously, you only need to support jumping back per RFC1035 and your code supports both jumping back AND forward, so I'm just telling you this for idle chitchat and so that you know that your code was helpful and I appreciate you putting it out there. Thanks! Edit: Also, before I forget, this implementation uses recursion even though it's not necessary. https://github.com/google/gopacket/blob/3eaba08943250fd212520e5cff00ed808b8fc60a/layers/dns.go#L538 https://github.com/google/gopacket/blob/3eaba08943250fd21252... Between your code, the one linked to above, and djbdns I was totally confused about what the "right" way is. I eventually read RFC1035 and saw that all of your approaches are correct. Edit 2: Also, DJB has some choice words about the compression scheme. https://github.com/abh/djbdns/blob/0715ca5de14ad5eae2c779c536beee2ef56d07a2/dns_packet.c#L2 https://github.com/abh/djbdns/blob/0715ca5de14ad5eae2c779c53...

145 points by adamch 5y ago

tedunangst 5y ago

All the links (beej, const generics) seem to be relative to the post and don't resolve.

adamch 5y ago
Oops, thanks so much! I forgot to check 'em. Fixed now.

eminence32 5y ago

> The other reason I wanted to make a DNS client is that I knew I could simplify every step using some great Rust crates This is one of the reasons I rather like rust and it's ecosystem: you can delegate "boring" functionality (aka the stuff you need, but aren't interested in writing) to the ecosystem and just focus on the stuff you want to. Other languages also have a similarly rich ecosystem (python in particular). But compared to python, I find that rust is slightly better; 2 reasons come to mind: * Docs in rust seem to be slightly better. I think this is partly due to the generally high quality of docs.rs, so crate authors seem to have settled on the idea that some minimal baseline docs are expected. * Error handling in rust feels better. In the python world, I would see None returned some library functions that didn't really document this possibility, yielding an uncomfortable number of runtime TypeErrors or AttributeErrors due to an unexpected NoneType

vlovich123 5y ago
Package management is also drastically simplified because cargo is standard vs you have to basically do everything from scratch in Python land
- eminence32 5y ago
  As a consumer, pip is pretty good though
  - vlovich123 5y ago
    Pip isn’t suitable for development. Having a global shared package installation doesn’t work if working on separate codebases. It also doesn’t work because you get implicit dev dependencies and then how do you share those when you want to collaborate? Requirements.txt is a laugh for several reasons. Pip-compile and then installing that into a virtualenv is a neat solution that I found. Not sure if there’s anything meaningfully better yet.
- SnowHill9902 5y ago
  Is cargo more vulnerable to supply chain attacks than downloading a C++ lib and compiling?
  - hamandcheese 5y ago
    I’m not a big Rust user, but my understanding is that Cargo.lock contains checksums. So the answer to your question would be no. Downloading a C++ lib and using Cargo are both effectively trust on first use.
  - eminence32 5y ago
    rust might be more vulnerable in practice because it's so easy to add 3rd-party code (and update 3rd-party code).
    - carlmr 5y ago
      Exactly. In C++ everybody just writes their own libraries which enable other kinds of bugs and vulnerabilities.
deepsun 5y ago
Java ecosystem is a quintessence of delegating boring stuff. But without wild westness of NPM, libraries (aka crates) feel more "serious".
- jeroenhd 5y ago
  In my experience the Java world tries to look more serious with all dependencies having fancy corporate names, but the end result is exactly the same. Log4j proved to the world that Java has as much of a dependency tree hell problem as any other programming language. What I find annoying about Java is that very major dependencies are either slow-moving, making upgrading to a new Java version quite a challenge, or require constant rewrites with new releases. I've come to expect that upgrading dependencies usually means "the code works except for these methods being a bit different" but in Java libraries the entire API can change on a whim. The only thing I find "serious" about it is that everyone seems to run ancient versions of Java because their dependencies are impossible to upgrade. It's no wonder that Java 8 seems to be as popular as ever despite the two LTS releases since.

danamit 5y ago

I made one in JS years ago, what I learned is that DNS is simpler than I thought.

db48x 5y ago

Nice write–up. I happened to write a little dns client last month too, although I avoided all the low–level protocol stuff and concentrated on the high–level business logic: pull many rows from a database, then create and execute as many queries in parallel as possible. It’s a good demonstration of why I like Rust so much. All the pieces were already there, and Rust’s type system ensures that I cannot have any unexpected sources of errors. My error–handling code is honestly pretty crude, but the compiler ensures that I have actually handled them all, and that there can be no surprises. No innocent–looking library functions that – surprise! – can actually throw exceptions.

mike_d 5y ago

I encourage everyone to write a DNS implementation of some sorts. You quickly learn two things: 1) DNS is pretty simple. 2) handling all the edge cases and bugs in the hundreds of other implementations is not.

deleted 5y ago
[deleted]

ccouzens 5y ago

I made one too https://github.com/ccouzens/dns-packet https://github.com/ccouzens/dns-packet The differences: I followed this guide rather than the RFCs https://github.com/EmilHernvall/dnsguide/blob/master/chapter1.md https://github.com/EmilHernvall/dnsguide/blob/master/chapter... Mine isn't as polished. The command line parsing and output is more thrown together. I hardcoded the packet identifier (it's not production code, and I'm only looking up one at a time). I didn't use any bit manipulation libraries. I can see they would help because DNS packets don't line up their information with the byte boundaries.

iforgotpassword 5y ago

I once added ipv6 capabilities to a very old windows software. It turned out that before Windows 8, there was no way to asynchronously resolve an AAAA record in a single threaded app. For the good old A records, you could ask Windows to resolve a hostname and it would send you a window message when it was done. So after fighting with multithreading in this archane app - I forgot what exactly the problems were - I wrote a very simple resolver that would only be able to handle AAAA lookups. It really is surprisingly simple. I think the more involved part was googling how to retrieve the system's configured DNS servers. ;)

outofmyshed 5y ago

nslookup and host both exist. It’s interesting that the author went straight into a programming challenge rather than looking for another program that may already exist on their system and would solve their dig-hate problem.

adamch 5y ago
I was aware that other DNS clients existed, but just using one of them would have taught me as much!

fanf2 5y ago

The first thing I check for in new DNS code is how the name decompression works, because it is a surprisingly subtle part of the DNS: you have to have a way to stop chasing compression pointers forever, either by limiting the number of pointers you will follow, or by requiring them to point earlier in the message. It is very common to find DNS code that has a denial of service vulnerability in its name decompression; it is one of the best examples of a packet parsing security vulnerability that is not about buffer overflows.

spc476 5y ago
When I wrote my DNS parser [1], I set a follow limit. I'm not sure how viable it would be to limit the pointers to earlier in the packet would be. [1] https://github.com/spc476/SPCDNS https://github.com/spc476/SPCDNS
- ate53 5y ago
  It's a viable strategy. No legitimate implementation produces compression pointers that go anywhere other than backwards. I track the start of a label sequence and whether I've followed a pointer. If I've just followed a pointer, I must be at a point earlier than the last start, or I consider it a malicious name. I also disallow pointers to pointers but I'm less confident that there's not something out there that does that.
  - spc476 5y ago
    So, using () as a marker for the length of a label, and (^name) for a label pointer, how do you handle ()a()b(^b) The pointer goes backwards, but it doesn't go back to the start of a label sequence. So do you track the start of each label? Or just the start of each domain name? Edit: nevermind, I see what you are doing.
- acuozzo 5y ago
  One thing I noticed while using your code as a reference for a project I worked on several weeks ago is that it doesn't record the state before following the pointer, so unless the pointer you follow eventually leads to another pointer that brings it back to where you were, you can't simulate recursion. I'm referring to lines 1286-1288 in codec.c, FWIW. Obviously, you only need to support jumping back per RFC1035 and your code supports both jumping back AND forward, so I'm just telling you this for idle chitchat and so that you know that your code was helpful and I appreciate you putting it out there. Thanks! Edit: Also, before I forget, this implementation uses recursion even though it's not necessary. https://github.com/google/gopacket/blob/3eaba08943250fd212520e5cff00ed808b8fc60a/layers/dns.go#L538 https://github.com/google/gopacket/blob/3eaba08943250fd21252... Between your code, the one linked to above, and djbdns I was totally confused about what the "right" way is. I eventually read RFC1035 and saw that all of your approaches are correct. Edit 2: Also, DJB has some choice words about the compression scheme. https://github.com/abh/djbdns/blob/0715ca5de14ad5eae2c779c536beee2ef56d07a2/dns_packet.c#L2 https://github.com/abh/djbdns/blob/0715ca5de14ad5eae2c779c53...