11 ms·

“1.1.1.1 is now handling more than 1.3T requests per day”

209 points by frankjr 3y ago

m348e912 3y ago
What's the benefit to cloudflare for offering dns services for free other than data-mining? Facilitating 1.3T DNS requests per day can't be cheap.
- devwastaken 3y ago
  If we think more complexly we can find a much better answer. Cloudflares profitable services depend on DNS working properly for both the customer and the customers userbase. DNS servers outside of cloudflares control may not be correct, or may not route properly in various edge conditions. Therefore an easy way to solve this is to simply host their own DNS.
  - bombcar 3y ago
    This is probably it; the very first thing support can ask is "try it with 1.1.1.1 and see if it works" - if it does, it's a DNS issue. Also, by running their own DNS they can ensure that some percentage of the Internet does NOT cache results beyond what they want (to some extent) - much of what Cloudflare does is DNS trickery to route to the appropriate CDN.
- dec0dedab0de 3y ago
  They have an accurate internal number of what websites people are going to. They could use this to direct their sales people. They might also use it with their secret score on every user to decide if they need a captcha. They could also just have it in their back pocket as a potential extra revenue stream selling user data.
  - Zambyte 3y ago
    The "other than data-mining" part is still unanswered
    - wddkcs 3y ago
      Isn't this typical tech development? Gain monopoly through free offerings and then squeeze the segment for max profit
      - scarlson 3y ago
        Do no evil always comes to a hard stop when investors realize it also means do no capitalism.
        TylerE 3y ago
        Almost like the modern corporate structure is what's really evil. Sixty or Seventy years ago, in the golden age of US manufacturing, companies were mostly privately owned. No shareholders to please. That let them make long-range strategic plays, instead of making short term anti-plays just to make the next quarter number go up.
      - benced 3y ago
        Google has been offering 8.8.8.8 forever and never charged.
        ceejayoz 3y ago
        I would presume they're getting plenty of value out of that data.
        dylan604 3y ago
        Tap tap tap, is this thing on? To quote: The "other than data-mining" part is still unanswered
      - aednichols 3y ago
        Commoditize your complement. Users need working DNS to use Cloudflare’s paid services. That used to be a business in and of itself, where OpenDNS made money showing people ads. But now the market price has been driven to zero.
    - 3y ago
- pornel 3y ago
  Latency. Cloudflare is selling speed. When you use their DNS, then lookup of their CDN domains is going to be faster, since you're querying the source directly rather than through intermediate DNS servers.
- kokojie 3y ago
  One of the more evil things they could do is show you ads, for example every 100 requests, you are shown a full page ad for a few seconds, then directs to where you were going. Pretty much like how youtube does it. I'm not saying they will do this, but they could if they wanted to. I've seen this from DNS providers in other countries.
  - bbkane 3y ago
    Would that work over HTTPS? My understanding is that, when the browser connects to the IP returned over DNS, it validates the TLS cert returned by the server at that IP contains the website name.
  - Ajedi32 3y ago
    Thankfully that's probably not feasible anymore given how prevalent HTTPS is these days.
  - lxgr 3y ago
    Given that almost every site now uses HTTPS, they could only do this for users navigating to non-existing domains (and furthermore have to register the domain names themselves, or they wouldn't be able to get any TLS certificate for them).
  - aednichols 3y ago
    OpenDNS used to hijack DNS to show ads, which is one of the reasons Google launched the non-hijacking 8.8.8.8 in 2009. OpenDNS quickly followed suit. No one would attempt it today, seeing as it would instantly make you the worst DNS resolver on the Internet and switching away is trivial.
- toast0 3y ago
  More control over ip selection for clients to better direct traffic for their customers is the obvious first order benefit. Passive competitor data collection is useful too. DNS traffic has good traffic ratios, at least compared to typical http traffic. That's got to help with peering negotiations. Although DNS traffic isn't that much, so it probably doesn't help much. Same for accepting and dropping inbound volumetric DDoS. Larger ISPs want to see roughly balanced inbound and outbound flows before they'll agree to settlement free peering, and options for attracting inbound flows from residential ISP customers are limited (run a backup service, run a 1:1 multimedia messaging service and/or video chat, DNS helps a bit)
- lxgr 3y ago
  It (coincidentially or not) limits the performance of competing regional CDNs to that of Cloudflare, since 1.1.1.1 truncates the EDNS client subnet parameter, which means that the only location/region hint a third-party CDN DNS has to go by is Cloudflare's location, not the user's. As an example, consider a user in Paris, and assume for the sake of this example that Cloudflare's closest POP is in Berlin. Assume that user visiting a site hosted by a French CDN, with POPs in both Paris and Berlin. If that user tries to resolve cdn.mysite.fr via 1.1.1.1, they'd get a Berlin-based location, since the French CDN has no way of knowing that the user is actually closer to them, and Paris would have been the better option.
  - johncolanduoni 3y ago
    Cloudflare doesn’t use GeoDNS for their CDN, it’s all anycast. They have no use for EDNS for their own traffic. Honestly I don’t take any CDN that doesn’t operate on anycast seriously these days, the issues with GeoDNS are well established at this point and if your network operations can’t handle the complexity of an anycast deployment you’re probably dropping the ball on a lot of other fronts.
    - mike_d 3y ago
      Have you ever built a CDN or anycast network? I have, many times. Anycast isn't magic, and it can't be used in all situations. For example Akamai has 16 times as many POPs as Cloudflare does, and many of them reside within smaller ISPs where you might not be able to establish BGP sessions. Long running downloads and websockets are better served over unicast addresses. Not to mention you don't have a lot of knobs to fix things when anycast does go wrong and you have a network in the US sending your traffic to an EU POP because they have cheaper transit in Europe. You might find this work from LinkedIn informative: https://engineering.linkedin.com/network-performance/tcp-over-ip-anycast-pipe-dream-or-reality https://engineering.linkedin.com/network-performance/tcp-ove...
    - lxgr 3y ago
      > Cloudflare doesn’t use GeoDNS for their CDN, it’s all anycast. They have no use for EDNS for their own traffic. Which makes it all the more dubious to block it…
- whoopdedo 3y ago
  Considering their core products, DNS might be an early-warning sign of DDoS. A sudden surge in requests for a domain will trip a circuit breaker before the HTTP sessions begin. There's probably a way to tell the difference between a bot attack and a hug-of-death by the pattern of DNS.
greenie_beans 3y ago
what percentage is that from ads?
- warent 3y ago
  According to my pihole, about 30% for my network
  - dylan604 3y ago
    I'm honestly surprised by how low that number is.
- SSLy 3y ago
  My family phones: 30% my own devices: 6.5%
8organicbits 3y ago
https://nitter.net/eastdakota/status/1686927537740611584 https://nitter.net/eastdakota/status/1686927537740611584
- Seb-C 3y ago
  You can also use this extension: https://chrome.google.com/webstore/detail/nitter-redirect/mohaicophfnifehkkkdbcejkflmgfkof https://chrome.google.com/webstore/detail/nitter-redirect/mo...
8organicbits 3y ago
Anyone have an estimate for how many servers that takes? That's atleast 15M req/sec. PowerDNS Recursor docs[1] mention thousands of requests per second, so maybe they could do 15M = 15K × 1K servers? Between possible implementation improvements, clustering overhead, regional endpoints, and overprovisioning for peak load it gets fuzzy quick. [1] https://doc.powerdns.com/recursor/performance.html https://doc.powerdns.com/recursor/performance.html
- dmw_ng 3y ago
  Assuming 256 bytes per query (very high estimate) that's about 3851 MB/s or about 3.5 10 Gbit ports Packet count obviously matters, but a KV cache lookup followed by a short reply in the usual case is one of the easiest loads to engineer for
  - hotpotamus 3y ago
    DNS also seems like one of the easiest cases for anycast and Cloudflare also seems like one of the best organizations to figure out that sort of internet hackery.
  - zgluck 3y ago
    So assuming an efficient implementation and including high availability requirements: probably in the order of tens of machines, maybe hundreds but not thousands. Edit: eastdakota's tweet says 13% of the requests are https- or tls-encrypted. That throws a spanner into these calculations.
    - dmw_ng 3y ago
      I'd imagine somewhere well under 50, probably with plenty of CPUs
- tveita 3y ago
  Clicking around from your link you get to https://doc.powerdns.com/recursor/index.html https://doc.powerdns.com/recursor/index.html > A quad Xeon 3GHz has been measured functioning very well at 400000 real life replayed packets per second. So that's more like 40 servers. Realistically the complexity of serving DNS is probably somewhere between Fortunes and Json Serialization in the Techempower benchmarks: https://www.techempower.com/benchmarks/#section=data-r21&test=fortune https://www.techempower.com/benchmarks/#section=data-r21&tes... So it's possible you could do it with 10, far less than you'd want for proper redundancy.
theideaofcoffee 3y ago
It's a nice accomplishment to build a user-facing feature to that scale, but I'm not all that impressed by the raw number. Let's break it down: 1.3T requests per day is roughly 15M req/sec, and assuming that each of the approximately 300 PoP locations [0] is serving this via anycast, that's 'only' 50k req/sec/PoP, easily doable by a couple, a handful at most, of hosts in each location. Small request and response sizes increase total request rate but keep down overall bit rate so one can handle a larger number of requests with a small number of hosts. Obviously these numbers will vary with location, higher and lower based on overall pop utilization usage, but rough enough to plan for. Even with encryption, these are well within the reach of single digit machine numbers in each location. I don't think the raw number is difficult, the difficult problem was building up the supporting infrastructure, both technical and product-wise. My source for the above? I've built at-scale, geo-distributed internet-facing services such as anycast DNS, video streaming delivery and the like. [0] https://www.cloudflare.com/network/ https://www.cloudflare.com/network/
- mathiasgredal 3y ago
  For reference opentrackr.org runs on a single machine[1] and handles ~200k connections pr. second[2] [1]: https://twitter.com/opentrackr/status/1383937485403693062 https://twitter.com/opentrackr/status/1383937485403693062 [2]: http://opentrackr.org/ http://opentrackr.org/
- MuffinFlavored 3y ago
  What is a number that would impress you?
  - Aachen 3y ago
    It's not the digits themselves but rather their unit and whose bank account it is we're talking about Any number of requests is going to be reasonable when you break it down, because if it were unreasonable, it wouldn't be run this way
- deleted 3y ago
  [deleted]
lxgr 3y ago
That's (mostly) great! I personally don't use it, but do appreciate competition in public DNS resolvers and having a high-quality fallback for networks that have a mediocre native resolver. Maybe this would be a good opportunity to stop truncating EDNS client subnets from requests [1]: It doesn't really help user privacy, but is quite anti-competitive since it can degrade latency needlessly for other (DNS-based) CDNs [2], especially in regions where Cloudflare does not have a POP closer to the user than the other CDN. [1] https://news.ycombinator.com/item?id=36970702 https://news.ycombinator.com/item?id=36970702 [2] https://news.ycombinator.com/item?id=36987315 https://news.ycombinator.com/item?id=36987315
- jadbox 3y ago
  What do you use for dns if I may ask?
  - lxgr 3y ago
    Generally just my ISPs resolver, since they can eventually see all of my traffic anyway [1] and I don't see the need to have either Google or Cloudflare added in between. For browsing though, I frequently use iCloud Private Relay, which uses oblivious DNS [2] for name lookups. Cloudflare powers that much of the time too, but at least Apple anonymizes my side of the connection. [1] At least until ESNI or one of its successors finally become widely supported – then I'll reconsider. [2] https://www.apple.com/icloud/docs/iCloud_Private_Relay_Overview_Dec2021.pdf https://www.apple.com/icloud/docs/iCloud_Private_Relay_Overv...
    - richardwhiuk 3y ago
      Even after ESNI, they can still see IPs you visit.
      - lxgr 3y ago
        Yes, but IPs are more ambiguous than hostnames in many cases.
  - 3y ago
tambourine_man 3y ago
It's impressive and good to have options for fast, free and frequently updated DNS servers, but… DNS is probably the most successful decentralized protocol we have, one that I use as a mental model when day dream about solving identity problems. I hope we don't end up with 2 or 3 DNS servers eventually.
- nvarsj 3y ago
  Not sure if you’re being sarcastic. In my experience DNS is probably in the top 2-3 causes of global outages at large internet tech companies. It is a giant SPOF in most services, often having many hidden dependencies. I can generally tell the maturity of a service based on whether it has stopped using DNS in favour of anycast or static IP lists.
  - pixl97 3y ago
    Eh, you can also tell the size and scaling of a service if it went to static list, and I'd say pretty damned small. Most problems with DNS are ones of dependency loops from cold start conditions. It's kinda difficult to blame DNS on bad design. Edit: Oh, anycast routing at IP level...Also, anycast is DNS, so I'm not sure how anycast saves you from DNS?
    - cuu508 3y ago
      > Also, anycast is DNS, so I'm not sure how anycast saves you from DNS? Are you sure about that? I thought anycast is a routing methodology where many servers share a single IP (example: 1.1.1.1).
  - tambourine_man 3y ago
    Not sarcastic at all. DNS solves a difficult problem (decentralized identity) in a deceptively simple way. We have the “it was DNS” joke because it’s a hard problem with lots of unforeseeable edge cases, not because DNS is poorly designed.
  - crote 3y ago
    How many of those are due to companies trying to do weird things with DNS, though? DNS itself is quite well-designed for its age. It is decentralized, so there is no single-point-of-failure. A request can fit in a single UDP package, so load balancing is trivial. A response can contain multiple records, making it possible to use it for load balancing or failover. The format is designed to be extensible, so it can be used for more than just the original few applications. It is definitely showing its age when it comes to privacy and security, though. Can a failure of an authoritative DNS server bring down a website? Absolutely! Set a short TTL and bring down the servers due to fatfingering a config, and your website will go offline. But you can hardly blame DNS itself for that.
tmikaeld 3y ago
That's great, but as a nasty surprise we found when using it on servers, is that it's rate-limited at 10req/s which is easy to hit even on a development server. Compared to googles 8.8.8.8 and 8.8.4.4 that has a very high 1500 QPS limit [aednichols][0]. I find it weird when they have so much linux tooling for encrypted DNS..
- simlevesque 3y ago
  > which is easy to hit even on a development server That's actually a good thing.
  - tmikaeld 3y ago
    Not a good thing when doing load-testing and DNS resolution rate-limiting silently fails.
    - growse 3y ago
      What are you load testing and hooking up to a public DNS resolver? Would a local caching resolver not be much more suitable?
      - someplaceguy 3y ago
        I'm not the OP, but I'm in the same boat. On some servers, I can use a local caching resolver. However, on others, I'm forced to configure the caching resolver to forward queries to an upstream public DNS server like 1.1.1.1 (via DNS-over-TLS) because of network issues. These can arise from DNS (or perhaps UDP?) rate limiting by the server hosting provider itself, at the network level. In another instance, I noticed that the hosting provider outright drops all fragmented IP packets. I'm concerned this could lead to DNS query failures, especially if the authoritative DNS server is not available over TCP. When contacted, they told me they do this for security reasons and would not be able to disable this filter for me. Like the OP, on these systems I often encounter the 1.1.1.1 rate limit. This is particularly the case since I have DNSSEC verification enabled, so I'm considering switching over to 8.8.8.8.
AtNightWeCode 3y ago
I mostly use 1.1.1.2 or 1.1.1.3 depending on the device. That is CF with different restrictions. On Windows the big problem with DNS is that it can be changed randomly. Among all the garbage on Windows this is peak garbage. Any con can change the DNS that I explicitly told it use. DNS by it self is probably top 10 sec risk on desktop envs and it so completely broken by design. I hack it more or less daily to "debug" our sites. Conservative people leave us with DNS, TLS/Public CAs and other sht and then they whine on HN day in and out why did a "root" cert of MS leak or whatever. Securing TLS or DNS with automatons at a large scale company is a complete nightmare.
mindslight 3y ago
When you think about it, this is pretty depressing statistic for anyone interested in decentralized/libre services. It's trivial to run recursive resolvers yourself, and this demonstrates just how much people can't be bothered. (and heading off the inevitable "but that increases the load on the authoritative servers" - this is only relevant to my comment if you're arguing that people are consciously choosing to outsource DNS because of this concern)
0cf8612b2e1e 3y ago
What is the easiest/trouble free way of hosting my own local DNS? It seems silly how frequently I leak web browsing habits to a third party.
- survirtual 3y ago
  Use pihole and a recursive dns (unbound). Here is a guide: https://docs.pi-hole.net/guides/dns/unbound/ https://docs.pi-hole.net/guides/dns/unbound/
- adrian_b 3y ago
  On any Linux or *BSD system it is very simple to run a DNS resolver on your own computer (e.g. unbound, https://nlnetlabs.nl/projects/unbound/about/ https://nlnetlabs.nl/projects/unbound/about/) that you configure to listen on localhost, so you can put 127.0.0.1 as the DNS server in your "/etc/resolv.conf". I have never used on any of my computers a DNS server controlled by other people.
  - 0cf8612b2e1e 3y ago
    Heh, appreciate the link, but definitely not the foolproof mechanism I was hoping to see. Leaves a bit much for the non-expert to infer + not an easy way to guarantee the system is always being utilized.
- deleted 3y ago
  [deleted]
rsingel 3y ago
This is great from a legal standpoint for net neutrality. ISPs (and their FCC champions) like to claim they are not telecom services that funnel data back and forth which makes them Title II common carriers, subject to strong FCC oversight. Instead they say they are information services who process and change data. Then they are Title I services that the FCC has little authority over. One way they do this is by saying they have DNS, that translates domains into IP addresses. Of course that's incidental but the more that people actually use alternative DNS, the clearer it is that DNS services aren't core to their offers, and the more evidence there is to show to a court that ISPs are Title II services. So if you want to support the upcoming return of federal net neutrality protections and FCC oversight of ISPs, one thing you can do is choose DNS NOT provided by your ISP.
- l33t233372 3y ago
  How many judges are able to make deep technical adjudications about which use of DNS is sufficient to be a title I or title II carrier?
  - pests 3y ago
    Probably a lot? Judges are perfectly capable of learning technical topics and are pretty smart in general. The judge who presided over Google vs Oracle taught himself to program. Some may be old but don't forget the people who invented the internet are 70s+ right now.
  - cvoss 3y ago
    Deep technical adjudication is exactly what judges in higher courts are equipped to do. No matter the domain, they are presented with tortuous technical language and jargon about a specialized topic (the statute), a collection of nuanced facts, and two or more carefully organized logical arguments, prepared by the parties' capable lawyers, about why those facts activate or don't activate this or that clause of the statute.
  - rsingel 3y ago
    That's a good question. Tech issues are tough, which is why you see lots of friend of the court briefs to explain it. For FCC matters around net neutrality, there's generally two courts: the DC Circuit Court of Appeals and then the Supreme Court. And, yes, they deal with it and have clerks to help. For instance, in the biggest decision about net neutrality, the BrandX case, where the FCC was allowed to determine that Cable companies were Title I information services, DNS is mentioned 13 times in the majority decision. https://supreme.justia.com/cases/federal/us/545/967/#tab-opinion-1961903 https://supreme.justia.com/cases/federal/us/545/967/#tab-opi... And if you find that interesting, I recommend reading the Scalia dissent, where he said that it's totally clear that broadband is a Title II service, and letting the FCC say otherwise was wickedly wrong.
jftuga 3y ago
Here is a ping short cut. It probably the fastest way to check for Internet access. This is also a CloudFlare DNS resolver. $ ping 1.1 Pinging 1.0.0.1 with 32 bytes of data: Reply from 1.0.0.1: bytes=32 time=17ms TTL=55 For IPv6, you can use: $ ping6 2600:: Interestingly enough, these are also pingable: $ ping 1234567890 Pinging 73.150.2.210 with 32 bytes of data: $ ping 987654321 Pinging 58.222.104.177 with 32 bytes of data:
- uoaei 3y ago
  Article from three months ago about these sorts of shorthand: https://news.ycombinator.com/item?id=35744130 https://news.ycombinator.com/item?id=35744130
hot_gril 3y ago
1.1.1.1 DNS is yet another example of why I don't view ipv6 as an "upgrade." Their v6 equivalent is 2606:4700:4700::1111.
- babypuncher 3y ago
  I know you're probably joking, but harder to remember numbers was always going to happen when you expand the address space from 4.3 billion combinations to 3.4 * 10^38 combinations.
  - hot_gril 3y ago
    It's not a joke. Just because you expanded the address space doesn't mean you need to redistribute every number across the space evenly. 1.1.1.1 should still be 1.1.1.1 in ipv6 land, encoded as 0x11110000..., and some random new device can have 2221:db8:1234::f350:2256:f3dd. They didn't need to change everyone's pre-existing addresses just to do a protocol change, same way adding unicode support to domain names wouldn't require google.com to become ∆Íßխեղճå∂ß.çøm. They also didn't need to change the default format to the ::::: stuff.
    - urinotherapist 3y ago
      It was at the start of the IPv6, but then disabled because companies were afraid of IPv6 compatibility with IPv4, which may be used to penetrate firewalls.
      - hot_gril 3y ago
        Huh, I had no idea, but I can see that happening. The bigger "nope" that comes to mind is wanting to replace NAT with every device having a public IPv6. Yes a firewall is theoretically superior to NAT, but it matters in practice how stupidly secure NAT is by default.
  - garbagecoder 3y ago
    Really? No shades of gray there for you? Why couldn't it be 1.1.1.1.1.1.1.1?
junon 3y ago
Cloudflare still refuses to resolve OpenNIC addresses though, and their forums parrot nonsense responses about why.
- riffic 3y ago
  OpenNIC is an alternative root, are you being daft or something? ICANN has committed to a unique authoritative root and if 1.1.1.1 were to resolve OpenNIC addresses it would violate ICP-3 and RFC 2826 https://www.icann.org/resources/pages/unique-authoritative-root-2012-02-25-en https://www.icann.org/resources/pages/unique-authoritative-r... https://www.rfc-editor.org/rfc/rfc2826.txt https://www.rfc-editor.org/rfc/rfc2826.txt
  - junon 3y ago
    That's a silly RFC in my opinion. The justifications are "we do it the right and blessed way so nobody else should be allowed". Also, of course ICANN wants to maintain the monopoly. They get insane salaries at the exec level, of which I'm sure trickles on over to cloudflare. Don't allow roots to be resolved if they service TLDs that conflict with other roots you already resolve, or they're off. Simple as that.
    - riffic 3y ago
      you've clearly answered my question in the affirmative.
NoZebra120vClip 3y ago
Once I was a member of a Linux user group, and we had regular meetings and also social gatherings, and during the latter, there were also mini-job-fairs; we could bring a résumé and chat with hiring managers. So I was seated next to a gentleman who worked for a giant mobile telco, and I asked him about what he did, and he gave me a little spiel and quoted a figure of something like 5 exabits per second of data being processed for security and fraud prevention. And I reacted with amazement, and I said something like "that sounds exciting!" And he says yeah, it's exciting, and then the conversation continued, and 15-20 minutes later, across the very large table, in another circle of conversation, somebody goes "you know, when I quote a large figure like 5 exabits per second, I'm always pleased to hear a reaction like 'that sounds exciting!' which is better than indifference or 'what's an exabit?'" and I chuckled at how the conversation had cross-pollinated, and my small comment was used as a good example. I should've asked what kind of hardware equipment processes 5 exabits/second.
idonegoofd 3y ago
Im more partial to quad9/9.9.9.9, but cool to see quad1 doing good. :)

209 points by frankjr 3y ago

What's the benefit to cloudflare for offering dns services for free other than data-mining? Facilitating 1.3T DNS requests per day can't be cheap.

devwastaken 3y ago
If we think more complexly we can find a much better answer. Cloudflares profitable services depend on DNS working properly for both the customer and the customers userbase. DNS servers outside of cloudflares control may not be correct, or may not route properly in various edge conditions. Therefore an easy way to solve this is to simply host their own DNS.
- bombcar 3y ago
  This is probably it; the very first thing support can ask is "try it with 1.1.1.1 and see if it works" - if it does, it's a DNS issue. Also, by running their own DNS they can ensure that some percentage of the Internet does NOT cache results beyond what they want (to some extent) - much of what Cloudflare does is DNS trickery to route to the appropriate CDN.
dec0dedab0de 3y ago
They have an accurate internal number of what websites people are going to. They could use this to direct their sales people. They might also use it with their secret score on every user to decide if they need a captcha. They could also just have it in their back pocket as a potential extra revenue stream selling user data.
- Zambyte 3y ago
  The "other than data-mining" part is still unanswered
  - wddkcs 3y ago
    Isn't this typical tech development? Gain monopoly through free offerings and then squeeze the segment for max profit
    - scarlson 3y ago
      Do no evil always comes to a hard stop when investors realize it also means do no capitalism.
      - TylerE 3y ago
        Almost like the modern corporate structure is what's really evil. Sixty or Seventy years ago, in the golden age of US manufacturing, companies were mostly privately owned. No shareholders to please. That let them make long-range strategic plays, instead of making short term anti-plays just to make the next quarter number go up.
    - benced 3y ago
      Google has been offering 8.8.8.8 forever and never charged.
      - ceejayoz 3y ago
        I would presume they're getting plenty of value out of that data.
      - dylan604 3y ago
        Tap tap tap, is this thing on? To quote: The "other than data-mining" part is still unanswered
    - aednichols 3y ago
      Commoditize your complement. Users need working DNS to use Cloudflare’s paid services. That used to be a business in and of itself, where OpenDNS made money showing people ads. But now the market price has been driven to zero.
  - 3y ago
pornel 3y ago
Latency. Cloudflare is selling speed. When you use their DNS, then lookup of their CDN domains is going to be faster, since you're querying the source directly rather than through intermediate DNS servers.
kokojie 3y ago
One of the more evil things they could do is show you ads, for example every 100 requests, you are shown a full page ad for a few seconds, then directs to where you were going. Pretty much like how youtube does it. I'm not saying they will do this, but they could if they wanted to. I've seen this from DNS providers in other countries.
- bbkane 3y ago
  Would that work over HTTPS? My understanding is that, when the browser connects to the IP returned over DNS, it validates the TLS cert returned by the server at that IP contains the website name.
- Ajedi32 3y ago
  Thankfully that's probably not feasible anymore given how prevalent HTTPS is these days.
- lxgr 3y ago
  Given that almost every site now uses HTTPS, they could only do this for users navigating to non-existing domains (and furthermore have to register the domain names themselves, or they wouldn't be able to get any TLS certificate for them).
- aednichols 3y ago
  OpenDNS used to hijack DNS to show ads, which is one of the reasons Google launched the non-hijacking 8.8.8.8 in 2009. OpenDNS quickly followed suit. No one would attempt it today, seeing as it would instantly make you the worst DNS resolver on the Internet and switching away is trivial.
toast0 3y ago
More control over ip selection for clients to better direct traffic for their customers is the obvious first order benefit. Passive competitor data collection is useful too. DNS traffic has good traffic ratios, at least compared to typical http traffic. That's got to help with peering negotiations. Although DNS traffic isn't that much, so it probably doesn't help much. Same for accepting and dropping inbound volumetric DDoS. Larger ISPs want to see roughly balanced inbound and outbound flows before they'll agree to settlement free peering, and options for attracting inbound flows from residential ISP customers are limited (run a backup service, run a 1:1 multimedia messaging service and/or video chat, DNS helps a bit)
lxgr 3y ago
It (coincidentially or not) limits the performance of competing regional CDNs to that of Cloudflare, since 1.1.1.1 truncates the EDNS client subnet parameter, which means that the only location/region hint a third-party CDN DNS has to go by is Cloudflare's location, not the user's. As an example, consider a user in Paris, and assume for the sake of this example that Cloudflare's closest POP is in Berlin. Assume that user visiting a site hosted by a French CDN, with POPs in both Paris and Berlin. If that user tries to resolve cdn.mysite.fr via 1.1.1.1, they'd get a Berlin-based location, since the French CDN has no way of knowing that the user is actually closer to them, and Paris would have been the better option.
- johncolanduoni 3y ago
  Cloudflare doesn’t use GeoDNS for their CDN, it’s all anycast. They have no use for EDNS for their own traffic. Honestly I don’t take any CDN that doesn’t operate on anycast seriously these days, the issues with GeoDNS are well established at this point and if your network operations can’t handle the complexity of an anycast deployment you’re probably dropping the ball on a lot of other fronts.
  - mike_d 3y ago
    Have you ever built a CDN or anycast network? I have, many times. Anycast isn't magic, and it can't be used in all situations. For example Akamai has 16 times as many POPs as Cloudflare does, and many of them reside within smaller ISPs where you might not be able to establish BGP sessions. Long running downloads and websockets are better served over unicast addresses. Not to mention you don't have a lot of knobs to fix things when anycast does go wrong and you have a network in the US sending your traffic to an EU POP because they have cheaper transit in Europe. You might find this work from LinkedIn informative: https://engineering.linkedin.com/network-performance/tcp-over-ip-anycast-pipe-dream-or-reality https://engineering.linkedin.com/network-performance/tcp-ove...
  - lxgr 3y ago
    > Cloudflare doesn’t use GeoDNS for their CDN, it’s all anycast. They have no use for EDNS for their own traffic. Which makes it all the more dubious to block it…
whoopdedo 3y ago
Considering their core products, DNS might be an early-warning sign of DDoS. A sudden surge in requests for a domain will trip a circuit breaker before the HTTP sessions begin. There's probably a way to tell the difference between a bot attack and a hug-of-death by the pattern of DNS.

greenie_beans 3y ago

what percentage is that from ads?

warent 3y ago
According to my pihole, about 30% for my network
- dylan604 3y ago
  I'm honestly surprised by how low that number is.
SSLy 3y ago
My family phones: 30% my own devices: 6.5%

8organicbits 3y ago

https://nitter.net/eastdakota/status/1686927537740611584 https://nitter.net/eastdakota/status/1686927537740611584

Seb-C 3y ago
You can also use this extension: https://chrome.google.com/webstore/detail/nitter-redirect/mohaicophfnifehkkkdbcejkflmgfkof https://chrome.google.com/webstore/detail/nitter-redirect/mo...

8organicbits 3y ago

Anyone have an estimate for how many servers that takes? That's atleast 15M req/sec. PowerDNS Recursor docs[1] mention thousands of requests per second, so maybe they could do 15M = 15K × 1K servers? Between possible implementation improvements, clustering overhead, regional endpoints, and overprovisioning for peak load it gets fuzzy quick. [1] https://doc.powerdns.com/recursor/performance.html https://doc.powerdns.com/recursor/performance.html

dmw_ng 3y ago
Assuming 256 bytes per query (very high estimate) that's about 3851 MB/s or about 3.5 10 Gbit ports Packet count obviously matters, but a KV cache lookup followed by a short reply in the usual case is one of the easiest loads to engineer for
- hotpotamus 3y ago
  DNS also seems like one of the easiest cases for anycast and Cloudflare also seems like one of the best organizations to figure out that sort of internet hackery.
- zgluck 3y ago
  So assuming an efficient implementation and including high availability requirements: probably in the order of tens of machines, maybe hundreds but not thousands. Edit: eastdakota's tweet says 13% of the requests are https- or tls-encrypted. That throws a spanner into these calculations.
  - dmw_ng 3y ago
    I'd imagine somewhere well under 50, probably with plenty of CPUs
tveita 3y ago
Clicking around from your link you get to https://doc.powerdns.com/recursor/index.html https://doc.powerdns.com/recursor/index.html > A quad Xeon 3GHz has been measured functioning very well at 400000 real life replayed packets per second. So that's more like 40 servers. Realistically the complexity of serving DNS is probably somewhere between Fortunes and Json Serialization in the Techempower benchmarks: https://www.techempower.com/benchmarks/#section=data-r21&test=fortune https://www.techempower.com/benchmarks/#section=data-r21&tes... So it's possible you could do it with 10, far less than you'd want for proper redundancy.

theideaofcoffee 3y ago

It's a nice accomplishment to build a user-facing feature to that scale, but I'm not all that impressed by the raw number. Let's break it down: 1.3T requests per day is roughly 15M req/sec, and assuming that each of the approximately 300 PoP locations [0] is serving this via anycast, that's 'only' 50k req/sec/PoP, easily doable by a couple, a handful at most, of hosts in each location. Small request and response sizes increase total request rate but keep down overall bit rate so one can handle a larger number of requests with a small number of hosts. Obviously these numbers will vary with location, higher and lower based on overall pop utilization usage, but rough enough to plan for. Even with encryption, these are well within the reach of single digit machine numbers in each location. I don't think the raw number is difficult, the difficult problem was building up the supporting infrastructure, both technical and product-wise. My source for the above? I've built at-scale, geo-distributed internet-facing services such as anycast DNS, video streaming delivery and the like. [0] https://www.cloudflare.com/network/ https://www.cloudflare.com/network/

mathiasgredal 3y ago
For reference opentrackr.org runs on a single machine[1] and handles ~200k connections pr. second[2] [1]: https://twitter.com/opentrackr/status/1383937485403693062 https://twitter.com/opentrackr/status/1383937485403693062 [2]: http://opentrackr.org/ http://opentrackr.org/
MuffinFlavored 3y ago
What is a number that would impress you?
- Aachen 3y ago
  It's not the digits themselves but rather their unit and whose bank account it is we're talking about Any number of requests is going to be reasonable when you break it down, because if it were unreasonable, it wouldn't be run this way
deleted 3y ago
[deleted]

lxgr 3y ago

That's (mostly) great! I personally don't use it, but do appreciate competition in public DNS resolvers and having a high-quality fallback for networks that have a mediocre native resolver. Maybe this would be a good opportunity to stop truncating EDNS client subnets from requests [1]: It doesn't really help user privacy, but is quite anti-competitive since it can degrade latency needlessly for other (DNS-based) CDNs [2], especially in regions where Cloudflare does not have a POP closer to the user than the other CDN. [1] https://news.ycombinator.com/item?id=36970702 https://news.ycombinator.com/item?id=36970702 [2] https://news.ycombinator.com/item?id=36987315 https://news.ycombinator.com/item?id=36987315

jadbox 3y ago
What do you use for dns if I may ask?
- lxgr 3y ago
  Generally just my ISPs resolver, since they can eventually see all of my traffic anyway [1] and I don't see the need to have either Google or Cloudflare added in between. For browsing though, I frequently use iCloud Private Relay, which uses oblivious DNS [2] for name lookups. Cloudflare powers that much of the time too, but at least Apple anonymizes my side of the connection. [1] At least until ESNI or one of its successors finally become widely supported – then I'll reconsider. [2] https://www.apple.com/icloud/docs/iCloud_Private_Relay_Overview_Dec2021.pdf https://www.apple.com/icloud/docs/iCloud_Private_Relay_Overv...
  - richardwhiuk 3y ago
    Even after ESNI, they can still see IPs you visit.
    - lxgr 3y ago
      Yes, but IPs are more ambiguous than hostnames in many cases.
- 3y ago

tambourine_man 3y ago

It's impressive and good to have options for fast, free and frequently updated DNS servers, but… DNS is probably the most successful decentralized protocol we have, one that I use as a mental model when day dream about solving identity problems. I hope we don't end up with 2 or 3 DNS servers eventually.

nvarsj 3y ago
Not sure if you’re being sarcastic. In my experience DNS is probably in the top 2-3 causes of global outages at large internet tech companies. It is a giant SPOF in most services, often having many hidden dependencies. I can generally tell the maturity of a service based on whether it has stopped using DNS in favour of anycast or static IP lists.
- pixl97 3y ago
  Eh, you can also tell the size and scaling of a service if it went to static list, and I'd say pretty damned small. Most problems with DNS are ones of dependency loops from cold start conditions. It's kinda difficult to blame DNS on bad design. Edit: Oh, anycast routing at IP level...Also, anycast is DNS, so I'm not sure how anycast saves you from DNS?
  - cuu508 3y ago
    > Also, anycast is DNS, so I'm not sure how anycast saves you from DNS? Are you sure about that? I thought anycast is a routing methodology where many servers share a single IP (example: 1.1.1.1).
- tambourine_man 3y ago
  Not sarcastic at all. DNS solves a difficult problem (decentralized identity) in a deceptively simple way. We have the “it was DNS” joke because it’s a hard problem with lots of unforeseeable edge cases, not because DNS is poorly designed.
- crote 3y ago
  How many of those are due to companies trying to do weird things with DNS, though? DNS itself is quite well-designed for its age. It is decentralized, so there is no single-point-of-failure. A request can fit in a single UDP package, so load balancing is trivial. A response can contain multiple records, making it possible to use it for load balancing or failover. The format is designed to be extensible, so it can be used for more than just the original few applications. It is definitely showing its age when it comes to privacy and security, though. Can a failure of an authoritative DNS server bring down a website? Absolutely! Set a short TTL and bring down the servers due to fatfingering a config, and your website will go offline. But you can hardly blame DNS itself for that.

tmikaeld 3y ago

That's great, but as a nasty surprise we found when using it on servers, is that it's rate-limited at 10req/s which is easy to hit even on a development server. Compared to googles 8.8.8.8 and 8.8.4.4 that has a very high 1500 QPS limit [aednichols][0]. I find it weird when they have so much linux tooling for encrypted DNS..

simlevesque 3y ago
> which is easy to hit even on a development server That's actually a good thing.
- tmikaeld 3y ago
  Not a good thing when doing load-testing and DNS resolution rate-limiting silently fails.
  - growse 3y ago
    What are you load testing and hooking up to a public DNS resolver? Would a local caching resolver not be much more suitable?
    - someplaceguy 3y ago
      I'm not the OP, but I'm in the same boat. On some servers, I can use a local caching resolver. However, on others, I'm forced to configure the caching resolver to forward queries to an upstream public DNS server like 1.1.1.1 (via DNS-over-TLS) because of network issues. These can arise from DNS (or perhaps UDP?) rate limiting by the server hosting provider itself, at the network level. In another instance, I noticed that the hosting provider outright drops all fragmented IP packets. I'm concerned this could lead to DNS query failures, especially if the authoritative DNS server is not available over TCP. When contacted, they told me they do this for security reasons and would not be able to disable this filter for me. Like the OP, on these systems I often encounter the 1.1.1.1 rate limit. This is particularly the case since I have DNSSEC verification enabled, so I'm considering switching over to 8.8.8.8.

AtNightWeCode 3y ago

I mostly use 1.1.1.2 or 1.1.1.3 depending on the device. That is CF with different restrictions. On Windows the big problem with DNS is that it can be changed randomly. Among all the garbage on Windows this is peak garbage. Any con can change the DNS that I explicitly told it use. DNS by it self is probably top 10 sec risk on desktop envs and it so completely broken by design. I hack it more or less daily to "debug" our sites. Conservative people leave us with DNS, TLS/Public CAs and other sht and then they whine on HN day in and out why did a "root" cert of MS leak or whatever. Securing TLS or DNS with automatons at a large scale company is a complete nightmare.

mindslight 3y ago

When you think about it, this is pretty depressing statistic for anyone interested in decentralized/libre services. It's trivial to run recursive resolvers yourself, and this demonstrates just how much people can't be bothered. (and heading off the inevitable "but that increases the load on the authoritative servers" - this is only relevant to my comment if you're arguing that people are consciously choosing to outsource DNS because of this concern)

0cf8612b2e1e 3y ago

What is the easiest/trouble free way of hosting my own local DNS? It seems silly how frequently I leak web browsing habits to a third party.

survirtual 3y ago
Use pihole and a recursive dns (unbound). Here is a guide: https://docs.pi-hole.net/guides/dns/unbound/ https://docs.pi-hole.net/guides/dns/unbound/
adrian_b 3y ago
On any Linux or *BSD system it is very simple to run a DNS resolver on your own computer (e.g. unbound, https://nlnetlabs.nl/projects/unbound/about/ https://nlnetlabs.nl/projects/unbound/about/) that you configure to listen on localhost, so you can put 127.0.0.1 as the DNS server in your "/etc/resolv.conf". I have never used on any of my computers a DNS server controlled by other people.
- 0cf8612b2e1e 3y ago
  Heh, appreciate the link, but definitely not the foolproof mechanism I was hoping to see. Leaves a bit much for the non-expert to infer + not an easy way to guarantee the system is always being utilized.
deleted 3y ago
[deleted]

rsingel 3y ago

This is great from a legal standpoint for net neutrality. ISPs (and their FCC champions) like to claim they are not telecom services that funnel data back and forth which makes them Title II common carriers, subject to strong FCC oversight. Instead they say they are information services who process and change data. Then they are Title I services that the FCC has little authority over. One way they do this is by saying they have DNS, that translates domains into IP addresses. Of course that's incidental but the more that people actually use alternative DNS, the clearer it is that DNS services aren't core to their offers, and the more evidence there is to show to a court that ISPs are Title II services. So if you want to support the upcoming return of federal net neutrality protections and FCC oversight of ISPs, one thing you can do is choose DNS NOT provided by your ISP.

l33t233372 3y ago
How many judges are able to make deep technical adjudications about which use of DNS is sufficient to be a title I or title II carrier?
- pests 3y ago
  Probably a lot? Judges are perfectly capable of learning technical topics and are pretty smart in general. The judge who presided over Google vs Oracle taught himself to program. Some may be old but don't forget the people who invented the internet are 70s+ right now.
- cvoss 3y ago
  Deep technical adjudication is exactly what judges in higher courts are equipped to do. No matter the domain, they are presented with tortuous technical language and jargon about a specialized topic (the statute), a collection of nuanced facts, and two or more carefully organized logical arguments, prepared by the parties' capable lawyers, about why those facts activate or don't activate this or that clause of the statute.
- rsingel 3y ago
  That's a good question. Tech issues are tough, which is why you see lots of friend of the court briefs to explain it. For FCC matters around net neutrality, there's generally two courts: the DC Circuit Court of Appeals and then the Supreme Court. And, yes, they deal with it and have clerks to help. For instance, in the biggest decision about net neutrality, the BrandX case, where the FCC was allowed to determine that Cable companies were Title I information services, DNS is mentioned 13 times in the majority decision. https://supreme.justia.com/cases/federal/us/545/967/#tab-opinion-1961903 https://supreme.justia.com/cases/federal/us/545/967/#tab-opi... And if you find that interesting, I recommend reading the Scalia dissent, where he said that it's totally clear that broadband is a Title II service, and letting the FCC say otherwise was wickedly wrong.

jftuga 3y ago

Here is a ping short cut. It probably the fastest way to check for Internet access. This is also a CloudFlare DNS resolver. $ ping 1.1 Pinging 1.0.0.1 with 32 bytes of data: Reply from 1.0.0.1: bytes=32 time=17ms TTL=55 For IPv6, you can use: $ ping6 2600:: Interestingly enough, these are also pingable: $ ping 1234567890 Pinging 73.150.2.210 with 32 bytes of data: $ ping 987654321 Pinging 58.222.104.177 with 32 bytes of data:

uoaei 3y ago
Article from three months ago about these sorts of shorthand: https://news.ycombinator.com/item?id=35744130 https://news.ycombinator.com/item?id=35744130

hot_gril 3y ago

1.1.1.1 DNS is yet another example of why I don't view ipv6 as an "upgrade." Their v6 equivalent is 2606:4700:4700::1111.

babypuncher 3y ago
I know you're probably joking, but harder to remember numbers was always going to happen when you expand the address space from 4.3 billion combinations to 3.4 * 10^38 combinations.
- hot_gril 3y ago
  It's not a joke. Just because you expanded the address space doesn't mean you need to redistribute every number across the space evenly. 1.1.1.1 should still be 1.1.1.1 in ipv6 land, encoded as 0x11110000..., and some random new device can have 2221:db8:1234::f350:2256:f3dd. They didn't need to change everyone's pre-existing addresses just to do a protocol change, same way adding unicode support to domain names wouldn't require google.com to become ∆Íßխեղճå∂ß.çøm. They also didn't need to change the default format to the ::::: stuff.
  - urinotherapist 3y ago
    It was at the start of the IPv6, but then disabled because companies were afraid of IPv6 compatibility with IPv4, which may be used to penetrate firewalls.
    - hot_gril 3y ago
      Huh, I had no idea, but I can see that happening. The bigger "nope" that comes to mind is wanting to replace NAT with every device having a public IPv6. Yes a firewall is theoretically superior to NAT, but it matters in practice how stupidly secure NAT is by default.
- garbagecoder 3y ago
  Really? No shades of gray there for you? Why couldn't it be 1.1.1.1.1.1.1.1?

junon 3y ago

Cloudflare still refuses to resolve OpenNIC addresses though, and their forums parrot nonsense responses about why.

riffic 3y ago
OpenNIC is an alternative root, are you being daft or something? ICANN has committed to a unique authoritative root and if 1.1.1.1 were to resolve OpenNIC addresses it would violate ICP-3 and RFC 2826 https://www.icann.org/resources/pages/unique-authoritative-root-2012-02-25-en https://www.icann.org/resources/pages/unique-authoritative-r... https://www.rfc-editor.org/rfc/rfc2826.txt https://www.rfc-editor.org/rfc/rfc2826.txt
- junon 3y ago
  That's a silly RFC in my opinion. The justifications are "we do it the right and blessed way so nobody else should be allowed". Also, of course ICANN wants to maintain the monopoly. They get insane salaries at the exec level, of which I'm sure trickles on over to cloudflare. Don't allow roots to be resolved if they service TLDs that conflict with other roots you already resolve, or they're off. Simple as that.
  - riffic 3y ago
    you've clearly answered my question in the affirmative.

NoZebra120vClip 3y ago

Once I was a member of a Linux user group, and we had regular meetings and also social gatherings, and during the latter, there were also mini-job-fairs; we could bring a résumé and chat with hiring managers. So I was seated next to a gentleman who worked for a giant mobile telco, and I asked him about what he did, and he gave me a little spiel and quoted a figure of something like 5 exabits per second of data being processed for security and fraud prevention. And I reacted with amazement, and I said something like "that sounds exciting!" And he says yeah, it's exciting, and then the conversation continued, and 15-20 minutes later, across the very large table, in another circle of conversation, somebody goes "you know, when I quote a large figure like 5 exabits per second, I'm always pleased to hear a reaction like 'that sounds exciting!' which is better than indifference or 'what's an exabit?'" and I chuckled at how the conversation had cross-pollinated, and my small comment was used as a good example. I should've asked what kind of hardware equipment processes 5 exabits/second.

idonegoofd 3y ago

Im more partial to quad9/9.9.9.9, but cool to see quad1 doing good. :)