3 ms·

> Why is that a problem for your use case? Because if the Google DNS servers were configured with a low rate limit, the DNS requests of the other clients of th

> Why is that a problem for your use case? Because if the Google DNS servers were configured with a low rate limit, the DNS requests of the other clients of the hotel would cause my DNS requests to fail, as they would all appear to be coming from the same IP address from Google's perspective. Fortunately, Google has a high DNS rate limit so this is not usually a problem. But the whole point of this thread is that Cloudflare DNS limits requests to only 10 queries per second, which is way too low for this scenario. > I expect you to run a dns cache/resolver at that scale and not freeload. You have plenty of customers/employees and must be making enough money. Yeah, me too. But as a customer, I also expect DNS servers not to censor results, not to hijack results, to properly resolve DNSSEC queries, to be available over DNS-over-TLS, to resolve queries reliably and to not be down frequently. Unfortunately these expectations are almost always broken and as a user, there's nothing I can do to change that, except to complain to the provider (which almost always does nothing useful, especially when I'm traveling) or to just use Cloudflare DNS or Google DNS myself.

willcipriano 3y ago
I feel like I'm missing something. Aren't these requests for your servers? Why are they getting censored and hijacked? Why not use HSTS, or if impossible sign the response with public-private key pair/encrpyt requests with same on device, if you are in such hostile territory?
- someplaceguy 3y ago
  > I feel like I'm missing something. Aren't these requests for your servers? Why are they getting censored and hijacked? I didn't mention I was running any servers in this conversation. Perhaps you are confusing me with the great-great...-grandparent? The scenarios I mentioned are all from the perspective of a user. They are getting censored because of laws in the countries I frequently visit and they are hijacked for multiple reasons (Wi-Fi portals and NXDOMAIN hijacking, mostly). Apart from that, I also do have servers (in different countries), but that's besides the point. But note that even these individual personal servers (a couple of which are forced to use a public DNS service due to network issues) hit the Cloudflare DNS rate limit during normal operation. As I mentioned in another thread, I don't do any benchmarking / load testing, web scraping nor anything of the sort, this is just normal operation for servers that are idle most of the time. It's especially noticeable for reverse-IP queries, for some reason (perhaps because these requests are bursty, therefore not cached, and cause several other queries to be performed? I'm not sure). As I mentioned before, even though I use caching resolvers, I have all of my machines configured to do DNSSEC verification, which contributes to the problem.
  - willcipriano 3y ago
    My mistake if I mistook you. > hit the Cloudflare DNS rate limit during normal operation. What I find implausible is that happening to a residential user, or even a apartment building full of them, frequently enough to matter unless some bit of software somewhere is doing something profoundly silly. I believe essentially all mainstream DNS lookup functions retry multiple times with exponential backoff, so its not just ten in a second, its ten in a second over a series of sliding windows that evaporate when they are satisfied for a time. That's a lot of requests.
    - someplaceguy 3y ago
      > I believe essentially all mainstream DNS lookup functions retry multiple times with exponential backoff, so its not just ten in a second, its ten in a second over a series of sliding windows that evaporate when they are satisfied for a time. That's a lot of requests. I only hit this issue when the machine has been idle for quite some time (hours, perhaps). Which indicates that part of the problem is that previous cached answers have expired. When I hit this issue, I am pretty sure that I am doing on the order of ~10 top-level DNS queries, for about ~4 different domains, a few of them being reverse-IP queries. It's possible that these requests are being amplified due to DNSSEC, so that might be part of the reason. It's also possible that my caching resolver, when answering a query for e.g. WWW.EXAMPLE.COM, is also doing queries for EXAMPLE.COM and .COM (and their DNSSEC keys), possibly even others, I don't know. I'm not exactly a DNS expert, unfortunately... All I know is that a dnsviz.net visualization seems to indicate that many queries are usually necessary to properly authenticate a domain that uses DNSSEC. Perhaps another issue is that, since the queries are simultaneous, the retries for these queries are getting synchronized and therefore also happen at the same time? I can tell you that often some of the DNS requests timeout after exactly 5 seconds, even though I have `options timeout:10 edns0` in my `/etc/resolv.conf` (which is being ignored due to a bug in bind's `host` command). Although I'm pretty sure the real problem is that I was getting a SERVFAIL response from Cloudflare DNS. If I used Google DNS instead of Cloudflare DNS the problem didn't happen. I don't know, perhaps you are right and some software is doing something silly.
      - 3y ago

willcipriano 3y ago

I feel like I'm missing something. Aren't these requests for your servers? Why are they getting censored and hijacked? Why not use HSTS, or if impossible sign the response with public-private key pair/encrpyt requests with same on device, if you are in such hostile territory?

someplaceguy 3y ago
> I feel like I'm missing something. Aren't these requests for your servers? Why are they getting censored and hijacked? I didn't mention I was running any servers in this conversation. Perhaps you are confusing me with the great-great...-grandparent? The scenarios I mentioned are all from the perspective of a user. They are getting censored because of laws in the countries I frequently visit and they are hijacked for multiple reasons (Wi-Fi portals and NXDOMAIN hijacking, mostly). Apart from that, I also do have servers (in different countries), but that's besides the point. But note that even these individual personal servers (a couple of which are forced to use a public DNS service due to network issues) hit the Cloudflare DNS rate limit during normal operation. As I mentioned in another thread, I don't do any benchmarking / load testing, web scraping nor anything of the sort, this is just normal operation for servers that are idle most of the time. It's especially noticeable for reverse-IP queries, for some reason (perhaps because these requests are bursty, therefore not cached, and cause several other queries to be performed? I'm not sure). As I mentioned before, even though I use caching resolvers, I have all of my machines configured to do DNSSEC verification, which contributes to the problem.
- willcipriano 3y ago
  My mistake if I mistook you. > hit the Cloudflare DNS rate limit during normal operation. What I find implausible is that happening to a residential user, or even a apartment building full of them, frequently enough to matter unless some bit of software somewhere is doing something profoundly silly. I believe essentially all mainstream DNS lookup functions retry multiple times with exponential backoff, so its not just ten in a second, its ten in a second over a series of sliding windows that evaporate when they are satisfied for a time. That's a lot of requests.
  - someplaceguy 3y ago
    > I believe essentially all mainstream DNS lookup functions retry multiple times with exponential backoff, so its not just ten in a second, its ten in a second over a series of sliding windows that evaporate when they are satisfied for a time. That's a lot of requests. I only hit this issue when the machine has been idle for quite some time (hours, perhaps). Which indicates that part of the problem is that previous cached answers have expired. When I hit this issue, I am pretty sure that I am doing on the order of ~10 top-level DNS queries, for about ~4 different domains, a few of them being reverse-IP queries. It's possible that these requests are being amplified due to DNSSEC, so that might be part of the reason. It's also possible that my caching resolver, when answering a query for e.g. WWW.EXAMPLE.COM, is also doing queries for EXAMPLE.COM and .COM (and their DNSSEC keys), possibly even others, I don't know. I'm not exactly a DNS expert, unfortunately... All I know is that a dnsviz.net visualization seems to indicate that many queries are usually necessary to properly authenticate a domain that uses DNSSEC. Perhaps another issue is that, since the queries are simultaneous, the retries for these queries are getting synchronized and therefore also happen at the same time? I can tell you that often some of the DNS requests timeout after exactly 5 seconds, even though I have `options timeout:10 edns0` in my `/etc/resolv.conf` (which is being ignored due to a bug in bind's `host` command). Although I'm pretty sure the real problem is that I was getting a SERVFAIL response from Cloudflare DNS. If I used Google DNS instead of Cloudflare DNS the problem didn't happen. I don't know, perhaps you are right and some software is doing something silly.
    - 3y ago