3 ms·

What do you mean by "fix the problem on their end"? As I explained, I am not able to fix the problem on my end, if that's what you mean. In fact, this is the ex

What do you mean by "fix the problem on their end"? As I explained, I am not able to fix the problem on my end, if that's what you mean. In fact, this is the exact reason why public DNS servers are so useful: they are much more reliable than the available alternatives, in many cases. Sure, if everything was perfect and all ISP/hosting provider DNS servers were modern and well-configured and had great availability and didn't censor or hijack results, then they wouldn't be necessary. But alas, unfortunately we have to live in the real world. I am not against rate limits, because as you said, sending a billion requests per second (due to misconfiguration, or a bug, or for malicious reasons) is not reasonable. But 10 requests/second is way, way too strict, especially if you use DNSSEC and/or have multiple client machines sharing the same IP address without a common DNS caching resolver in-between (which may not be possible to have for several reasons).

willcipriano 3y ago
> which may not be possible to have for several reasons Try me. What's the reason? Money can't be one of them if you expect someone else to scale up and handle your load for free as that would be very rude of you.
- someplaceguy 3y ago
  Sure, here are three off the top of my head that have affected me personally (not to mention the million other possible different scenarios that might exist): 1. In one case, I have multiple machines on the same network but simply speaking, none of them are turned on 24/7 (except for the router), so none of them can be configured to be a common caching resolver. The router is a proprietary Unifi gateway device, for which it is not possible to configure it to use DNS-over-TLS, neither on the local side nor when forwarding to the public DNS server. 2. The other common case for me are my mobile devices (laptops and smartphones). I simply cannot configure them to use a common local caching DNS server because there isn't one, as these devices frequently connect over multiple networks (such as 5G networks, hotel Wi-Fi, etc) and you cannot just use the network-provided DNS server without being vulnerable to man-in-the-middle attacks, censored and hijacked results, etc. 3. Another issue is public Wi-Fi networks. In large hotels, stadiums, etc, there might be hundreds or even thousands of devices behind the public same IP address(es), and as a user yourself, you have no control over them or the network. And obviously, you cannot just use their provided DNS server on these Wi-Fi networks, for multiple reasons: 1) why would you trust the DNS server of a random Wi-Fi network in the first place?, 2) even if you trusted the provider, it is not possible to authenticate these networks, so you could be talking to a man-in-the-middle attacker without realizing, and 3) even if there isn't a man-in-the-middle attacker and the provider is trustworthy, often these DNS servers are extremely poor, as they often censor and hijack results, pollute the DNS cache, don't support DNSSEC queries, etc. And I'm not even mentioning the privacy issues. To be clear, I always use local caching resolvers. But on these systems, I am forced to configure them to forward queries directly to public DNS servers, which means that they don't have a common caching resolver except for the public DNS server itself, hence the rate limit issue when they are all behind the same IP address.
  - jeroenhd 3y ago
    I don't know, the easiest solution would be "run your own DNS server that you point the weird servers at". A handful of euros per month for a VPS will get you more DNS queries than you'll ever need. No need to even hit Cloudflare when you can turn on the recursive setting on your own server; this protects you against the viable but unlikely scenario that Cloudflare messes with your DNS/gets its DNS cache poisoned/goes down. PowerDNS + DoH + DoT can be configured in an afternoon. Another if you need to read some dnsdist documentation and add a LUA script to prevent DDoS in case you're not able to authenticate clients (https://dnsdist.org/rules-actions.html https://dnsdist.org/rules-actions.html and https://dnsdist.org/advanced/qpslimits.html https://dnsdist.org/advanced/qpslimits.html are a nice start). Nothing wrong with using a public DNS resolver for your own devices, but I think using public services like these for benchmarking/load testing is abusing the generosity of the people running these servers.
    - someplaceguy 3y ago
      It's not a money issue, as I have multiple servers already which could easily provide this service, and it's even less of a configuration issue, as I am perfectly capable of configure this if I wanted. But these servers are in different countries, so they would be much slower than Cloudflare DNS due to network latency. And even if I had a server in the same country for my client devices, I frequently travel to other countries anyway. I don't think I am abusing these servers, as I only use them normally as a user, it's not like I am scraping the Internet or anything like that. Unlike the OP, I am not doing any benchmarking or load testing or anything similar.
  - willcipriano 3y ago
    I can't imagine what use case requires dozens of devices, hooked up to potentially untrusted random public wifi networks, making many DNS requests all at the same instant. For your very odd use case I'd suggest hosting your own recursive resolver someplace like AWS and pointing to that, but I suspect if you checked your priors you probably can change something about the first 3 conditions.
    - 3y ago
sumtechguy 3y ago
For a single home 10/sec is probably fine. But remember you are using other peoples infrastructure. Be nice. You probably want a local resolver anyway in a case where 10/s is an issue. Even for home use a local caching resolver is not a bad idea. It usually makes just day to day web surfing feel much snappier. Most home routers take care of it but are rather limited. I usually set up a local one then have my local DHCP just hand out that address so all the clients get the right thing instead of going to some external source. That way I can have a list of resolvers and one IP to hand out to my client computers. The default of many home routers is the router for the DNS field in DHCP.
- someplaceguy 3y ago
  > For a single home 10/sec is probably fine. But remember you are using other peoples infrastructure. Be nice. You probably want a local resolver anyway in a case where 10/s is an issue. In one case, I am already using a local caching resolver, which is configured to forward to Cloudflare DNS (as I have no better alternative, except Google DNS perhaps). It's a single machine which has a public IPv4 address, not used by any other machine. If the machine has been idle for some hours, it hits the Cloudflare DNS rate limit just by doing 5 or 6 normal queries, plus about 3 reverse IP address queries at the same time (I'd have to check for the exact numbers). It is configured to do DNSSEC verification, which I think contributes to the issue. Perhaps the fact that these requests are bursty, therefore probably not cached, is also part of the issue. > Even for home use a local caching resolver is not a bad idea. All of my machines have local caching resolvers, but this does not solve the issue for many different scenarios, especially those where there are multiple machines behind the same public IP address. > I usually set up a local one then have my local DHCP just hand out that address so all the clients get the right thing instead of going to some external source. Sure, but as I mentioned in another thread, it's not always possible to set up a local resolver on a network, especially when you are just a client of a network that is not yours (e.g. public/hotel Wi-Fi) and whose DNS server is unreliable or untrustworthy for many different reasons.

willcipriano 3y ago

> which may not be possible to have for several reasons Try me. What's the reason? Money can't be one of them if you expect someone else to scale up and handle your load for free as that would be very rude of you.

someplaceguy 3y ago
Sure, here are three off the top of my head that have affected me personally (not to mention the million other possible different scenarios that might exist): 1. In one case, I have multiple machines on the same network but simply speaking, none of them are turned on 24/7 (except for the router), so none of them can be configured to be a common caching resolver. The router is a proprietary Unifi gateway device, for which it is not possible to configure it to use DNS-over-TLS, neither on the local side nor when forwarding to the public DNS server. 2. The other common case for me are my mobile devices (laptops and smartphones). I simply cannot configure them to use a common local caching DNS server because there isn't one, as these devices frequently connect over multiple networks (such as 5G networks, hotel Wi-Fi, etc) and you cannot just use the network-provided DNS server without being vulnerable to man-in-the-middle attacks, censored and hijacked results, etc. 3. Another issue is public Wi-Fi networks. In large hotels, stadiums, etc, there might be hundreds or even thousands of devices behind the public same IP address(es), and as a user yourself, you have no control over them or the network. And obviously, you cannot just use their provided DNS server on these Wi-Fi networks, for multiple reasons: 1) why would you trust the DNS server of a random Wi-Fi network in the first place?, 2) even if you trusted the provider, it is not possible to authenticate these networks, so you could be talking to a man-in-the-middle attacker without realizing, and 3) even if there isn't a man-in-the-middle attacker and the provider is trustworthy, often these DNS servers are extremely poor, as they often censor and hijack results, pollute the DNS cache, don't support DNSSEC queries, etc. And I'm not even mentioning the privacy issues. To be clear, I always use local caching resolvers. But on these systems, I am forced to configure them to forward queries directly to public DNS servers, which means that they don't have a common caching resolver except for the public DNS server itself, hence the rate limit issue when they are all behind the same IP address.
- jeroenhd 3y ago
  I don't know, the easiest solution would be "run your own DNS server that you point the weird servers at". A handful of euros per month for a VPS will get you more DNS queries than you'll ever need. No need to even hit Cloudflare when you can turn on the recursive setting on your own server; this protects you against the viable but unlikely scenario that Cloudflare messes with your DNS/gets its DNS cache poisoned/goes down. PowerDNS + DoH + DoT can be configured in an afternoon. Another if you need to read some dnsdist documentation and add a LUA script to prevent DDoS in case you're not able to authenticate clients (https://dnsdist.org/rules-actions.html https://dnsdist.org/rules-actions.html and https://dnsdist.org/advanced/qpslimits.html https://dnsdist.org/advanced/qpslimits.html are a nice start). Nothing wrong with using a public DNS resolver for your own devices, but I think using public services like these for benchmarking/load testing is abusing the generosity of the people running these servers.
  - someplaceguy 3y ago
    It's not a money issue, as I have multiple servers already which could easily provide this service, and it's even less of a configuration issue, as I am perfectly capable of configure this if I wanted. But these servers are in different countries, so they would be much slower than Cloudflare DNS due to network latency. And even if I had a server in the same country for my client devices, I frequently travel to other countries anyway. I don't think I am abusing these servers, as I only use them normally as a user, it's not like I am scraping the Internet or anything like that. Unlike the OP, I am not doing any benchmarking or load testing or anything similar.
- willcipriano 3y ago
  I can't imagine what use case requires dozens of devices, hooked up to potentially untrusted random public wifi networks, making many DNS requests all at the same instant. For your very odd use case I'd suggest hosting your own recursive resolver someplace like AWS and pointing to that, but I suspect if you checked your priors you probably can change something about the first 3 conditions.
  - 3y ago

sumtechguy 3y ago

For a single home 10/sec is probably fine. But remember you are using other peoples infrastructure. Be nice. You probably want a local resolver anyway in a case where 10/s is an issue. Even for home use a local caching resolver is not a bad idea. It usually makes just day to day web surfing feel much snappier. Most home routers take care of it but are rather limited. I usually set up a local one then have my local DHCP just hand out that address so all the clients get the right thing instead of going to some external source. That way I can have a list of resolvers and one IP to hand out to my client computers. The default of many home routers is the router for the DNS field in DHCP.

someplaceguy 3y ago
> For a single home 10/sec is probably fine. But remember you are using other peoples infrastructure. Be nice. You probably want a local resolver anyway in a case where 10/s is an issue. In one case, I am already using a local caching resolver, which is configured to forward to Cloudflare DNS (as I have no better alternative, except Google DNS perhaps). It's a single machine which has a public IPv4 address, not used by any other machine. If the machine has been idle for some hours, it hits the Cloudflare DNS rate limit just by doing 5 or 6 normal queries, plus about 3 reverse IP address queries at the same time (I'd have to check for the exact numbers). It is configured to do DNSSEC verification, which I think contributes to the issue. Perhaps the fact that these requests are bursty, therefore probably not cached, is also part of the issue. > Even for home use a local caching resolver is not a bad idea. All of my machines have local caching resolvers, but this does not solve the issue for many different scenarios, especially those where there are multiple machines behind the same public IP address. > I usually set up a local one then have my local DHCP just hand out that address so all the clients get the right thing instead of going to some external source. Sure, but as I mentioned in another thread, it's not always possible to set up a local resolver on a network, especially when you are just a client of a network that is not yours (e.g. public/hotel Wi-Fi) and whose DNS server is unreliable or untrustworthy for many different reasons.