4 ms·

RethinkDNS Resolver That Deploys to CF Workers, Deno Deploy, Fastly, Fly.io

106 points by indigodaddy 2y ago

chrisweekly 2y ago
Cool! "Pi-hole -esque" is a nice descriptor. Tangent: Bunny.net is my new favorite CDN / cloud service provider. They have scriptable DNS too.
- Imustaskforhelp 2y ago
  Bunny requires a credit card for its free trial and it seems that it would only give 7 days free trial which is really weird. So I am off to cloudflare workers free tier.
  - chrisweekly 1y ago
    FWIW Bunny added free credits (totaling $50) so my free trial is effectively MUCH longer than 7d. They're the fastest CDN, have awesome / intuitive UX, and very robust featureset.
- sieabahlpark 2y ago
  [dead]
Onavo 2y ago
The Rethink DNS mobile app hasn't seen a release in years. Is the project still alive? https://github.com/celzero/rethink-app https://github.com/celzero/rethink-app
- ementally 2y ago
  >in years. not even past 1 year. See https://github.com/celzero/rethink-app/discussions/1884 https://github.com/celzero/rethink-app/discussions/1884
  - withinboredom 2y ago
    > the development happens on the lead developer's feature branch Oof. This makes me cringe so hard. I once took over a project (but the developer didn't know they were getting fired) and the guy was doing everything on his laptop, from his laptop. Deployments and builds were from his laptop. Even dependencies weren't checked into the code (using global installs of them on unknown versions). The owner had me come in because after talking to several people realized he was in a bad place. It took me ~2 months to learn everything and document all the things. Then the owner fired him. That guy kept development back for so long by simply not documenting/sharing code and configuration. Now there's an entire team with a healthy development flow. But wow, I had some flashbacks reading that...
maxloh 2y ago
I doubt their choice of implementing something so performance-critical as a DNS server in JavaScript. It seems like a slow language for that purpose. When I was looking for ad-block solutions on Android, Rethink DNS was actually on the top of my list. However, when I found out that their server was written in JavaScript, I did some benchmarking. Rethink's server processed DNS requests in 400-500ms, which could potentially make a new webpage render up to half a second slower the first time: ~ > curl -w "DNS Time: %{time_namelookup}s\nConnect: %{time_connect}s\nTotal: %{time_total}s\n" -H "accept: application/dns-message" -H "content-type: application/dns-message" --data "<binary-data>" -o nul -s https://sky.rethinkdns.com/1:6AcDACIBLIDAAFQwIAAACA== DNS Time: 0.004995s Connect: 0.142332s Total: 0.462496s While the Cloudflare's server took just 5-10ms, as seen below: ~ > curl -w "DNS Time: %{time_namelookup}s\nConnect: %{time_connect}s\nTotal: %{time_total}s\n" -H "accept: application/dns-message" -H "content-type: application/dns-message" --data "<binary-data>" -o nul -s 1.1.1.1 DNS Time: 0.000026s Connect: 0.006273s Total: 0.008822s In the end, I chose AdAway and have stuck to that choice.
- ignoramous 2y ago
  > Rethink's server processed DNS requests in 40-60ms, which could potentially make a new webpage render up to 50ms slower the first time, while the system default took just 5-10ms You can choose "system default" in the Rethink Android app from Configure -> DNS -> System DNS. You don't have to use Rethink's DoH (DNS over HTTPS) / DoT (DNS over TLS) servers. You can setup any DNSCrypt, Oblivious DoH, or plain old DNS endpoint with Rethink to forward DNS requests to. > I chose AdAway and have stuck to that choice for now A decent choice. A word of caution when using it in non-root mode, though: AdAway borrows code from another (now discontinued but pretty solid) project viz. DNS66. From when I looked at the code, DNS66 never handled DNS over TCP. You can test this with Termux: dig +tcp example.com A. > doubt their choice of implementing something so performance-critical as a DNS server in JavaScript For a remotely-hosted DNS resolver, network latency & concurrency is likely to dominate than performance of the code itself. You're right that JS is slow (as compared to Rust, say), but looking at metrics from serving 300m to 3bn queries/day on Cloudflare, our median (CPU time) spent processing a DNS request was consistently less than 2ms, and IO wait (there are no "disks" so, this is mostly hitting the caches or upstream resolvers) was estimated to be less than 7ms. Also, we employ a bunch of optimizations on client & the server, as appropriate (if we're missing any we should implement, let us know). - Coalesce multiple requests into one upstream request: For example, if there's 40 clients all wanting to resolve the same domain (say, ipv4only.arpa) within milliseconds of each other, only one request is upstreamed. - Utilize in-process LFU cache and Cloudflare's Cache API to avoid upstreaming where possible. - Connection pool egress to avoid reconnect overheads. - Race response for a query from multiple upstreams in parallel. - Support TLS session resumption. - Prefer the lighter TLS_AES_128_GCM_SHA256 cipher suite. - Dynamically adjust the TLS frame size. DNS queries aren't that big. - Disable Nagling on TCP. - Rehydrate responses for top domains stored in the in-process cache in the background. - Admission Control (load shedding) & queuing disciplines (CoDel etc) --- > curl -w "DNS Time: %{time_namelookup}s\nConnect: %{time_connect}s\nTotal: %{time_total}s\n" -H "accept: application/dns-message" -H "content-type: application/dns-message" --data "<binary-data>" -o nul -s 1.1.1.1 Shouldn't the switch "-s" be "https://one.one.one.one/dns-query https://one.one.one.one/dns-query"? For DoH perf specifically, I use https://github.com/ameshkov/godnsbench https://github.com/ameshkov/godnsbench as it supports parallel and cache-busting (random) queries. Ex: Rethink on Workers: ./godnsbench -a https://sky.rethinkdns.com/dns-query -p 10 -c 300 -t 1 -q {random}.dnsleaktest.com godnsbench with the following configuration: { "Address": "https://sky.rethinkdns.com/dns-query", "Connections": 10, "Timeout": 1, "QueriesCount": 300, } The test results are: Elapsed: 6.120078941s Average QPS: 49.018394 Processed queries: 300 Average per query: 20.400765ms Errors count: 0 Cloudflare 1.1.1.1: ./godnsbench -a https://cloudflare-dns.com/dns-query -p 10 -c 300 -t 1 -q {random}.dnsleaktest.com godnsbench with the following configuration: { "Address": "https://cloudflare-dns.com/dns-query", "Connections": 10, "Timeout": 1, "QueriesCount": 300, } The test results are: Elapsed: 6.780474724s Average QPS: 44.244383 Processed queries: 299 Average per query: 22.677588ms Errors count: 1
karel-3d 2y ago
Just fyi (not talking down this project, more providing alternatives) - dnsdist (part of OpenDNS "suite") which is mostly "sold" as a DNS load balancer - is also a perfect stub resolver, that can do DoH/DoT on both ends, that can block certain websites, and is very easy on resources. Not that many people talk about its usage as a stub resolver, and I don't know why!

106 points by indigodaddy 2y ago

Cool! "Pi-hole -esque" is a nice descriptor. Tangent: Bunny.net is my new favorite CDN / cloud service provider. They have scriptable DNS too.

Imustaskforhelp 2y ago
Bunny requires a credit card for its free trial and it seems that it would only give 7 days free trial which is really weird. So I am off to cloudflare workers free tier.
- chrisweekly 1y ago
  FWIW Bunny added free credits (totaling $50) so my free trial is effectively MUCH longer than 7d. They're the fastest CDN, have awesome / intuitive UX, and very robust featureset.
sieabahlpark 2y ago
[dead]

Onavo 2y ago

The Rethink DNS mobile app hasn't seen a release in years. Is the project still alive? https://github.com/celzero/rethink-app https://github.com/celzero/rethink-app

ementally 2y ago
>in years. not even past 1 year. See https://github.com/celzero/rethink-app/discussions/1884 https://github.com/celzero/rethink-app/discussions/1884
- withinboredom 2y ago
  > the development happens on the lead developer's feature branch Oof. This makes me cringe so hard. I once took over a project (but the developer didn't know they were getting fired) and the guy was doing everything on his laptop, from his laptop. Deployments and builds were from his laptop. Even dependencies weren't checked into the code (using global installs of them on unknown versions). The owner had me come in because after talking to several people realized he was in a bad place. It took me ~2 months to learn everything and document all the things. Then the owner fired him. That guy kept development back for so long by simply not documenting/sharing code and configuration. Now there's an entire team with a healthy development flow. But wow, I had some flashbacks reading that...

maxloh 2y ago

I doubt their choice of implementing something so performance-critical as a DNS server in JavaScript. It seems like a slow language for that purpose. When I was looking for ad-block solutions on Android, Rethink DNS was actually on the top of my list. However, when I found out that their server was written in JavaScript, I did some benchmarking. Rethink's server processed DNS requests in 400-500ms, which could potentially make a new webpage render up to half a second slower the first time: ~ > curl -w "DNS Time: %{time_namelookup}s\nConnect: %{time_connect}s\nTotal: %{time_total}s\n" -H "accept: application/dns-message" -H "content-type: application/dns-message" --data "<binary-data>" -o nul -s https://sky.rethinkdns.com/1:6AcDACIBLIDAAFQwIAAACA== DNS Time: 0.004995s Connect: 0.142332s Total: 0.462496s While the Cloudflare's server took just 5-10ms, as seen below: ~ > curl -w "DNS Time: %{time_namelookup}s\nConnect: %{time_connect}s\nTotal: %{time_total}s\n" -H "accept: application/dns-message" -H "content-type: application/dns-message" --data "<binary-data>" -o nul -s 1.1.1.1 DNS Time: 0.000026s Connect: 0.006273s Total: 0.008822s In the end, I chose AdAway and have stuck to that choice.

ignoramous 2y ago
> Rethink's server processed DNS requests in 40-60ms, which could potentially make a new webpage render up to 50ms slower the first time, while the system default took just 5-10ms You can choose "system default" in the Rethink Android app from Configure -> DNS -> System DNS. You don't have to use Rethink's DoH (DNS over HTTPS) / DoT (DNS over TLS) servers. You can setup any DNSCrypt, Oblivious DoH, or plain old DNS endpoint with Rethink to forward DNS requests to. > I chose AdAway and have stuck to that choice for now A decent choice. A word of caution when using it in non-root mode, though: AdAway borrows code from another (now discontinued but pretty solid) project viz. DNS66. From when I looked at the code, DNS66 never handled DNS over TCP. You can test this with Termux: dig +tcp example.com A. > doubt their choice of implementing something so performance-critical as a DNS server in JavaScript For a remotely-hosted DNS resolver, network latency & concurrency is likely to dominate than performance of the code itself. You're right that JS is slow (as compared to Rust, say), but looking at metrics from serving 300m to 3bn queries/day on Cloudflare, our median (CPU time) spent processing a DNS request was consistently less than 2ms, and IO wait (there are no "disks" so, this is mostly hitting the caches or upstream resolvers) was estimated to be less than 7ms. Also, we employ a bunch of optimizations on client & the server, as appropriate (if we're missing any we should implement, let us know). - Coalesce multiple requests into one upstream request: For example, if there's 40 clients all wanting to resolve the same domain (say, ipv4only.arpa) within milliseconds of each other, only one request is upstreamed. - Utilize in-process LFU cache and Cloudflare's Cache API to avoid upstreaming where possible. - Connection pool egress to avoid reconnect overheads. - Race response for a query from multiple upstreams in parallel. - Support TLS session resumption. - Prefer the lighter TLS_AES_128_GCM_SHA256 cipher suite. - Dynamically adjust the TLS frame size. DNS queries aren't that big. - Disable Nagling on TCP. - Rehydrate responses for top domains stored in the in-process cache in the background. - Admission Control (load shedding) & queuing disciplines (CoDel etc) --- > curl -w "DNS Time: %{time_namelookup}s\nConnect: %{time_connect}s\nTotal: %{time_total}s\n" -H "accept: application/dns-message" -H "content-type: application/dns-message" --data "<binary-data>" -o nul -s 1.1.1.1 Shouldn't the switch "-s" be "https://one.one.one.one/dns-query https://one.one.one.one/dns-query"? For DoH perf specifically, I use https://github.com/ameshkov/godnsbench https://github.com/ameshkov/godnsbench as it supports parallel and cache-busting (random) queries. Ex: Rethink on Workers: ./godnsbench -a https://sky.rethinkdns.com/dns-query -p 10 -c 300 -t 1 -q {random}.dnsleaktest.com godnsbench with the following configuration: { "Address": "https://sky.rethinkdns.com/dns-query", "Connections": 10, "Timeout": 1, "QueriesCount": 300, } The test results are: Elapsed: 6.120078941s Average QPS: 49.018394 Processed queries: 300 Average per query: 20.400765ms Errors count: 0 Cloudflare 1.1.1.1: ./godnsbench -a https://cloudflare-dns.com/dns-query -p 10 -c 300 -t 1 -q {random}.dnsleaktest.com godnsbench with the following configuration: { "Address": "https://cloudflare-dns.com/dns-query", "Connections": 10, "Timeout": 1, "QueriesCount": 300, } The test results are: Elapsed: 6.780474724s Average QPS: 44.244383 Processed queries: 299 Average per query: 22.677588ms Errors count: 1

karel-3d 2y ago

Just fyi (not talking down this project, more providing alternatives) - dnsdist (part of OpenDNS "suite") which is mostly "sold" as a DNS load balancer - is also a perfect stub resolver, that can do DoH/DoT on both ends, that can block certain websites, and is very easy on resources. Not that many people talk about its usage as a stub resolver, and I don't know why!