4 ms·

Resolving DNS Asynchronously - from 200 to 20000 resolutions per minute.

28 points by pkrumins 17y ago

deleted 17y ago
[deleted]
spicyj 17y ago
Isn't 340 requests per second a good way to get yourself blocked from the DNS server?
- tptacek 17y ago
  [8:26pm:~/dnscache] DIVISION:tqbf [0:9]% nsping -t 0.005 192.168.1.254 NSPING 192.168.1.254 (192.168.1.254): Domain = "local", Type = "IN A" - [ 2 ] 109 bytes from 192.168.1.254: 135.227 ms [ 135.227 san-avg ] - [ 3 ] 109 bytes from 192.168.1.254: 81.386 ms [ 108.306 san-avg ] ... Not at AT&T apparently.
tptacek 17y ago
Pretty much every synchronous protocol on the Internet benefits immensely from parallelizing. Compare mtrr to traceroute. Bulk DNS lookup is almost entirely I/O bound, and so async sockets are pretty optimal.
- petercooper 17y ago
  You get a +1 for mentioning mtr (I Googled "mtrr" but assume you must mean mtr). Never heard of it before - downloaded via MacPorts and.. great stuff - I recommend checking it out to people who use traceroute and/or ping frequently. http://en.wikipedia.org/wiki/MTR_%28Software%29 http://en.wikipedia.org/wiki/MTR_%28Software%29
  - tptacek 17y ago
    Sorry, I'm also coding and terms are bleeding about in my brain. In the mid-'90s, some friends and I had a small contest on who could write the fastest traceroute. Sending the probes asyncronously gets you most of the way there, but there's a trick that would get you the first N hops for free, too.
    - petercooper 17y ago
      but there's a trick that would get you the first N hops for free, too. I smell a blog post/new HN post..! Well, I'm intrigued anyway.
      - tptacek 17y ago
        I'll never tell. ;) It doesn't work that well anymore anyhow.
        petercooper 17y ago
        I set my favorite networking guru on to this and he suspects you're talking about RFC 1393 - http://tools.ietf.org/html/rfc1393 http://tools.ietf.org/html/rfc1393 Of course, you'll never tell, but just posting it here in case it's interesting to anyone else!
        tptacek 17y ago
        Was that ever implemented? Here's a fun response to that document, though: http://cseweb.ucsd.edu/classes/wi01/cse222/papers/savage-traceback-sigcomm00.pdf http://cseweb.ucsd.edu/classes/wi01/cse222/papers/savage-tra...
ars 17y ago
At that rate it would take you about half a year to resolve every IP4 on the net. According to http://www.zooknic.com/Domains/counts.html http://www.zooknic.com/Domains/counts.html there are 107,235,401 domains, so you could resolve them all in under 4 days. Allow some extra time for the country TLDs, and perhaps subdomains.
justin_vanw 17y ago
Yes, when a task consists of mostly waiting it makes sense to do more than one of that task at the same time! I'm glad someone cared enough to give back to the community and post this wonderful result.

28 points by pkrumins 17y ago

deleted 17y ago

[deleted]

spicyj 17y ago

Isn't 340 requests per second a good way to get yourself blocked from the DNS server?

tptacek 17y ago
[8:26pm:~/dnscache] DIVISION:tqbf [0:9]% nsping -t 0.005 192.168.1.254 NSPING 192.168.1.254 (192.168.1.254): Domain = "local", Type = "IN A" - [ 2 ] 109 bytes from 192.168.1.254: 135.227 ms [ 135.227 san-avg ] - [ 3 ] 109 bytes from 192.168.1.254: 81.386 ms [ 108.306 san-avg ] ... Not at AT&T apparently.

tptacek 17y ago

Pretty much every synchronous protocol on the Internet benefits immensely from parallelizing. Compare mtrr to traceroute. Bulk DNS lookup is almost entirely I/O bound, and so async sockets are pretty optimal.

petercooper 17y ago
You get a +1 for mentioning mtr (I Googled "mtrr" but assume you must mean mtr). Never heard of it before - downloaded via MacPorts and.. great stuff - I recommend checking it out to people who use traceroute and/or ping frequently. http://en.wikipedia.org/wiki/MTR_%28Software%29 http://en.wikipedia.org/wiki/MTR_%28Software%29
- tptacek 17y ago
  Sorry, I'm also coding and terms are bleeding about in my brain. In the mid-'90s, some friends and I had a small contest on who could write the fastest traceroute. Sending the probes asyncronously gets you most of the way there, but there's a trick that would get you the first N hops for free, too.
  - petercooper 17y ago
    but there's a trick that would get you the first N hops for free, too. I smell a blog post/new HN post..! Well, I'm intrigued anyway.
    - tptacek 17y ago
      I'll never tell. ;) It doesn't work that well anymore anyhow.
      - petercooper 17y ago
        I set my favorite networking guru on to this and he suspects you're talking about RFC 1393 - http://tools.ietf.org/html/rfc1393 http://tools.ietf.org/html/rfc1393 Of course, you'll never tell, but just posting it here in case it's interesting to anyone else!
        tptacek 17y ago
        Was that ever implemented? Here's a fun response to that document, though: http://cseweb.ucsd.edu/classes/wi01/cse222/papers/savage-traceback-sigcomm00.pdf http://cseweb.ucsd.edu/classes/wi01/cse222/papers/savage-tra...

ars 17y ago

At that rate it would take you about half a year to resolve every IP4 on the net. According to http://www.zooknic.com/Domains/counts.html http://www.zooknic.com/Domains/counts.html there are 107,235,401 domains, so you could resolve them all in under 4 days. Allow some extra time for the country TLDs, and perhaps subdomains.

justin_vanw 17y ago

Yes, when a task consists of mostly waiting it makes sense to do more than one of that task at the same time! I'm glad someone cared enough to give back to the community and post this wonderful result.