3 ms·

Anyone have an estimate for how many servers that takes? That's atleast 15M req/sec. PowerDNS Recursor docs[1] mention thousands of requests per second, so may

Anyone have an estimate for how many servers that takes? That's atleast 15M req/sec. PowerDNS Recursor docs[1] mention thousands of requests per second, so maybe they could do 15M = 15K × 1K servers? Between possible implementation improvements, clustering overhead, regional endpoints, and overprovisioning for peak load it gets fuzzy quick. [1] https://doc.powerdns.com/recursor/performance.html https://doc.powerdns.com/recursor/performance.html

dmw_ng 3y ago
Assuming 256 bytes per query (very high estimate) that's about 3851 MB/s or about 3.5 10 Gbit ports Packet count obviously matters, but a KV cache lookup followed by a short reply in the usual case is one of the easiest loads to engineer for
- hotpotamus 3y ago
  DNS also seems like one of the easiest cases for anycast and Cloudflare also seems like one of the best organizations to figure out that sort of internet hackery.
- zgluck 3y ago
  So assuming an efficient implementation and including high availability requirements: probably in the order of tens of machines, maybe hundreds but not thousands. Edit: eastdakota's tweet says 13% of the requests are https- or tls-encrypted. That throws a spanner into these calculations.
  - dmw_ng 3y ago
    I'd imagine somewhere well under 50, probably with plenty of CPUs
tveita 3y ago
Clicking around from your link you get to https://doc.powerdns.com/recursor/index.html https://doc.powerdns.com/recursor/index.html > A quad Xeon 3GHz has been measured functioning very well at 400000 real life replayed packets per second. So that's more like 40 servers. Realistically the complexity of serving DNS is probably somewhere between Fortunes and Json Serialization in the Techempower benchmarks: https://www.techempower.com/benchmarks/#section=data-r21&test=fortune https://www.techempower.com/benchmarks/#section=data-r21&tes... So it's possible you could do it with 10, far less than you'd want for proper redundancy.
ignoramous 3y ago
Jio, India's largest ISP, serves about the same level of traffic as 1.1.1.1. They use "23 DNS resolver farms" and "256 DNS resolver engines" to do so: https://twitter.com/pgl/status/1599933560806608899 https://twitter.com/pgl/status/1599933560806608899 / https://archive.is/IF0ky https://archive.is/IF0ky Cloudflare probably runs 1.1.1.1 on all its servers in 300+ locations worldwide.
mike_d 3y ago
OpenDNS when I left was running I think 12 locations with ~4 machines each for approximately a million queries a second. But we did a lot more work than Cloudflare (loading each users preferences into memory, making a decision of how to answer based on them, logging stats for the user, etc) and had plenty of overhead. DNS isn't hard to serve at that scale compute or bandwidth wise, your problems are more administrative and maintaining enough overhead for misbehaving clients to accidently DoS you.

dmw_ng 3y ago

Assuming 256 bytes per query (very high estimate) that's about 3851 MB/s or about 3.5 10 Gbit ports Packet count obviously matters, but a KV cache lookup followed by a short reply in the usual case is one of the easiest loads to engineer for

hotpotamus 3y ago
DNS also seems like one of the easiest cases for anycast and Cloudflare also seems like one of the best organizations to figure out that sort of internet hackery.
zgluck 3y ago
So assuming an efficient implementation and including high availability requirements: probably in the order of tens of machines, maybe hundreds but not thousands. Edit: eastdakota's tweet says 13% of the requests are https- or tls-encrypted. That throws a spanner into these calculations.
- dmw_ng 3y ago
  I'd imagine somewhere well under 50, probably with plenty of CPUs

tveita 3y ago

Clicking around from your link you get to https://doc.powerdns.com/recursor/index.html https://doc.powerdns.com/recursor/index.html > A quad Xeon 3GHz has been measured functioning very well at 400000 real life replayed packets per second. So that's more like 40 servers. Realistically the complexity of serving DNS is probably somewhere between Fortunes and Json Serialization in the Techempower benchmarks: https://www.techempower.com/benchmarks/#section=data-r21&test=fortune https://www.techempower.com/benchmarks/#section=data-r21&tes... So it's possible you could do it with 10, far less than you'd want for proper redundancy.

ignoramous 3y ago

Jio, India's largest ISP, serves about the same level of traffic as 1.1.1.1. They use "23 DNS resolver farms" and "256 DNS resolver engines" to do so: https://twitter.com/pgl/status/1599933560806608899 https://twitter.com/pgl/status/1599933560806608899 / https://archive.is/IF0ky https://archive.is/IF0ky Cloudflare probably runs 1.1.1.1 on all its servers in 300+ locations worldwide.

mike_d 3y ago

OpenDNS when I left was running I think 12 locations with ~4 machines each for approximately a million queries a second. But we did a lot more work than Cloudflare (loading each users preferences into memory, making a decision of how to answer based on them, logging stats for the user, etc) and had plenty of overhead. DNS isn't hard to serve at that scale compute or bandwidth wise, your problems are more administrative and maintaining enough overhead for misbehaving clients to accidently DoS you.