6 ms·

Nice writeup. Especially interesting to see that CloudFlare is using Go for its DNS service. I run a DNS hosting service (https://www.SlickDNS.com https://www.S

by jbarham 13y ago

Nice writeup. Especially interesting to see that CloudFlare is using Go for its DNS service. I run a DNS hosting service (https://www.SlickDNS.com https://www.SlickDNS.com) and currently use tinydns from the djbdns suite in my name servers, but I'm writing a drop-in replacement in Go. The goal is to make it much higher performance by using multiple goroutines (tinydns is a single process).

tptacek 13y ago
Have you done any analysis to see if tinydns was actually a bottleneck? The work of an authoritative DNS server is extraordinarily simple, and tinydns capitalizes on that.
- jbarham 13y ago
  Under normal load tinydns' performance is more than adequate, but under DDoS attack the fact that it's a single process is a definite limitation when it's running on a multi-core machine. By default tinydns opens its database file for every request which can lead to I/O errors under very heavy load. I've patched it (http://tinydns.org/one-second.patch http://tinydns.org/one-second.patch) to only open the database every second, but with my Go rewrite I can easily keep the database in memory and reload it when the disk file changes.
  - seunosewa 13y ago
    Could you run one instance of TinyDNS on each core?
halayli 13y ago
You can also write it in C and use lthread. It will be faster than what Go has to offer. https://github.com/halayli/lthread https://github.com/halayli/lthread
- tptacek 13y ago
  It will probably not be significantly faster than Golang for this workload.
  - halayli 13y ago
    It can be. You have total control of memory layout. You can gain a lot of performance by considering locality of reference, slab allocators, and parsing DNS packets in general will most probably be much faster.
    - tptacek 13y ago
      Golang also allows control over memory layout.
      - halayli 13y ago
        of course it does, but it does it in a way that aims to provide an overall good performance for all apps. But it doesn't mean that's the best performance you can get.
        tptacek 13y ago
        What "way" would that be? Golang's approach to data layout is very similar to C's.
        halayli 13y ago
        C doesn't control memory. brk() does. There are various malloc libraries out there to improve the way malloc manages memory. But ideally you don't want to allocate memory at all. You want to have memory preallocated and preferably accessing a page that's still in cache. You can control memory pages in C because it doesn't manage that for you.
        tonyplee 13y ago
        +1

by jbarham 13y ago

tptacek 13y ago

Have you done any analysis to see if tinydns was actually a bottleneck? The work of an authoritative DNS server is extraordinarily simple, and tinydns capitalizes on that.

jbarham 13y ago
Under normal load tinydns' performance is more than adequate, but under DDoS attack the fact that it's a single process is a definite limitation when it's running on a multi-core machine. By default tinydns opens its database file for every request which can lead to I/O errors under very heavy load. I've patched it (http://tinydns.org/one-second.patch http://tinydns.org/one-second.patch) to only open the database every second, but with my Go rewrite I can easily keep the database in memory and reload it when the disk file changes.
- seunosewa 13y ago
  Could you run one instance of TinyDNS on each core?

halayli 13y ago

You can also write it in C and use lthread. It will be faster than what Go has to offer. https://github.com/halayli/lthread https://github.com/halayli/lthread

tptacek 13y ago
It will probably not be significantly faster than Golang for this workload.
- halayli 13y ago
  It can be. You have total control of memory layout. You can gain a lot of performance by considering locality of reference, slab allocators, and parsing DNS packets in general will most probably be much faster.
  - tptacek 13y ago
    Golang also allows control over memory layout.
    - halayli 13y ago
      of course it does, but it does it in a way that aims to provide an overall good performance for all apps. But it doesn't mean that's the best performance you can get.
      - tptacek 13y ago
        What "way" would that be? Golang's approach to data layout is very similar to C's.
      - halayli 13y ago
        C doesn't control memory. brk() does. There are various malloc libraries out there to improve the way malloc manages memory. But ideally you don't want to allocate memory at all. You want to have memory preallocated and preferably accessing a page that's still in cache. You can control memory pages in C because it doesn't manage that for you.
        tonyplee 13y ago
        +1