8 ms·

More people should be running their own mail servers, their own web servers, their own IRC servers, etc. But I don't think we are ever going back to that direc

by rmdoss 8y ago

More people should be running their own mail servers, their own web servers, their own IRC servers, etc. But I don't think we are ever going back to that direction. The arguments and benefits for running one locally are not enough the trouble as well. Performance? Due to DNS caching at the resolver level, it is probably faster to use Google's 8.8.8.8 or CloudFlare's 1.1.1.1, than anything local (where all dns requests are a MISS). Privacy? With DNS over TLS/DNS over HTTPS, your ISPs can't see what you are doing. If you run DNS locally, they can. Yes, they will see all the requests your resolvers are doing to the auth DNS servers. Security? Some good resolvers, like Quad9 or CleanBrowsing will block malicious domains. CleanBrowsing will also help blocking adult content if you have kids. I don't think maintaining such control is practical for most people (pi-hole helps, but still hard to keep it updated and find good enough databases to use). I would love a de-centralized web, but it is pretty hard to go back.

fiddlerwoaroof 8y ago
My browsing habits are pretty regular, though: there’s half a dozen sites I visit regularly and the rest are random blogs/etc. I suspect about 20% of the domains I visit account for 80% of my browsing traffic and there would be enormous benefits from a dns speed perspective to a local caching resolver.
- js2 8y ago
  Maybe. Many sites use pretty short TTLs. So your sites may be dropping out of your cache more frequently than you think. Easy to test though. Run dnsmasq and enable query logging and see how often it’s having to forward requests. Then realize a recursing resolver is potentially having to go all the way to the TLDs for those requests.
  - zcid 8y ago
    unbound for instance provides 'cache-min-ttl' which allows you to prevent excessively small ttl values.
jasonhansel 8y ago
I'd really like it if there was an easy way for "ordinary" consumers to setup a web/email/etc. server on, say, an AWS or GCP instance. Wouldn't exactly be decentralized, but it'd give users more control.
- fonosip 8y ago
  For pi-hole dns vpn self-hosted https://ba.net/adblock/vpn/doc/howto.html https://ba.net/adblock/vpn/doc/howto.html
- johnny22 8y ago
  It must handle backups and restores too, otherwise you're just making it too easy for folks to lose all their data.
tambourine_man 8y ago
The only chance we have for people to run their own services is if we make it dead easy to do so and the advantages are clearly communicated. Unfortunately, that hasn’t been open source’s forte historically.
- wvh 8y ago
  Until one of the big players decides they don't like your domain or IP range. I've been running my own email/web server for years, but even if you'd fully automate installation, there's a can of worms full of fun things like spam, DoS and shenanigans by the big players that really are not worth your time for just one person's email setup. I suppose we could come up with some easy configurable templates that would automatically install servers for privacy conscious individuals, but if at some point something goes wrong, most people are going to be stuck without service and no easy fix.
- Reason077 8y ago
  > ”The only chance we have for people to run their own services is if we make it dead easy” In the case of DNS, it’s dead easy if the server is built in to home routers. Many of them do, in fact, ship with local DNS already - it’s just that many users override it with 8.8.8.8 or whatever.
  - codetrotter 8y ago
    The DNS servers shipped in home routers are usually set up as caching DNS proxies by default, and will forward queries for cache misses to the DNS resolvers or caching proxies that your ISP is running, rather than the DNS servers in home routers doing the full resolution themselves. So it’s not what I would consider “true” “local DNS”. However, even if we did switch everyone to running their own DNS resolvers, what would happen then? Without the massive shared caching we have today, the load would significantly increase on the authorative DNS servers for each domain. Even with client local caching. So the number of companies running their own authorative DNS servers would probably decrease — more of them would be using hosted DNS provided by a third party. A lot of companies host the authorative DNS of their domains with a third party big already. Including myself — I use Cloudflare for all my sites because of the HTTP caching and other things they offer on top of hosted DNS. Increased load on authorative servers will likely lead to further centralization of DNS hosting with a few big providers IMO. Because even a lot companies that specialize in hosting DNS might not be able to handle the load when everyone is running their own resolver. Only the big DNS hosting companies will be able to afford it. So we end up with everyone hosting DNS with a few DNS hosting providers — Cloudflare, Amazon Route 53, etc. So by decentralizing the DNS resolvers that clients use, you push companies to centralize the authorative DNS servers further. The net effect is that you will only have shifted where in the resolution the queries centralize. And let’s say that this happens and Google sees the amount of queries received by 8.8.8.8 drop to near zero over night. Odds are that if Google values the data they gain from clients using these resolvers, they will make a big push to ensure that they host the DNS for as many companies as possible, so that they still end up with their hands on the query data. (And Google does value this data — otherwise they wouldn’t still be offering public DNS query servers.) And also, what about the root servers? Will they be able to handle the massive increase in load? And won’t the root server traffic be subject to surveillance by state actors wanting to know what sites someone is browsing? DNS is kind of funny because in a way it is both centralized and decentralized at the same time. But if you want the web to be truly decentralized I believe for the reasons stated above that having people run their own DNS resolvers is not part of the solution. You are going to have to replace DNS altogether. Realistically I don’t think DNS is going away anytime soon. The web and the internet in general is too reliant on it. But I really wish we could.
timbit42 8y ago
Won't IPv6 make this easier? Isn't the primary reason people don't run their own servers now because of NAT?
- Vosporos 8y ago
  The other reason is that maintaining a server and its services is sometimes a full-time job :/
  - stoolpigeon 8y ago
    This is anecdotal but I decided to host my own sites. I bought the cheapest droplet from digital ocean that I could and I set it up running Fedora. Then I installed Apache, MySQL and PHP so I could run some wordpress sites. The server kept running out of memory and shutting down MySQL so my sites stopped working. I started to learn how to read logs and saw that there is a huge amount of malicious activity directed at my server all the time. Yesterday I installed Fail2ban which meant installing postfix. I got it working (took most of the day) but I can't send emails to my gmail account because I need to set up dkim and dmarc and other stuff. I have a list. But first I have to learn how to do all this stuff. I use Fedora every day at work but I'm obviously no sysadmin and you are so right. All this takes hours and hours to learn and then to stay on top of it. When I was on shared hosting I had a lot less control and options but it was a lot easier as well.
    - wolco 8y ago
      Move mysql to a separate droplet. vultr: $2.50 otherwise you have to change mysql defaults to get things to fit. Great tutorials are out there in general reduce your workers/processes. Why do you need dkim or dmarc to send to gmail? If you send a test php mail does gmail pick it up?
      - stoolpigeon 8y ago
        If I send a test email with postfix - gmail rejects it and gives me a link to their page explaining why I need to add a bunch of stuff so they know I'm not spamming or fishing. Specifically I get this: gmail-smtp-in.l.google.com[173.194.207.27] said: 550-5.7.1 This message does not have authentication information or fails to pass 550-5.7.1 authentication checks. To best protect our users from spam, the 550-5.7.1 message has been blocked. Please visit 550-5.7.1 https://support.google.com/mail/answer/81126#authentication https://support.google.com/mail/answer/81126#authentication for more 550 5.7.1 information. d203si1756652qkb.228 - gsmtp (in reply to end of DATA command) there are tons of guides on troubleshooting mysql resource issues. Are they great? I don't know. Have I tried what is mentioned in many of them? Yes. I still have issues. I don't think it's just mysql though. I think it is a lot of little things that I'm slowly eliminating one by one.
johnklos 8y ago
Performance for Google or Cloudflare isn't going to be better. Where do you get that idea? Do you think all DNS simply lives in their caches? "your ISPs can't see what you are doing". If they're analyzing traffic, they can, and if they're doing that, they can see to whom I'm connecting, anyway. But you say nothing about why we should trust Google or Cloudflare. I trust my ISP to be big and dumb. I trust Goole and Cloudflare to want to make money.
- icedchai 8y ago
  Yes. Performance of Google / Cloudflare DNS will be better simply because so many other people are using them: any common DNS query result will probably already be cached... FYI I run my own DNS server anyway.
  - zAy0LfpBZLC8mAC 8y ago
    But what is common for you will usually be cached already in your own server as well, so most requests will still be cache hits--and all those hits avoid the ~ 20 to 100 ms round trip to the internet. Take news.ycombinator.com, for example: The A record has a TTL of 300 seconds. So, after the first visit, which probably will take a bit longer than asking Google/CF, for every request in the next five minutes, you will have a reduced lookup latency. Then, after five minutes, the next lookup will go out to the authoritative server. But mind you that the NS records for ycombinator.com have a TTL of two days, so those are still cached, and the refresh is indeed a single request to the authoritative server--which more often than not takes about as long as a cache hit from the recursive Google/CF resolvers (it's also one round trip to the internet ...). And then, there is stuff like BIND's prefetch mechanism which will start the refresh of an expiring record when it sees a query for that record shortly before its expiry: That query is answered immediately from the cache, and a refresh is started in the background, so that the refreshed record should arrive in the cache before the old version expires ... thus completely eliminating the lookup latency for often-used records. Though you might need to tune it to trigger earlier for your personal use than in the default configuration ...
    - 8y ago
meruru 8y ago
>With DNS over TLS/DNS over HTTPS, your ISPs can't see what you are doing. Is this true? They still can see what IPs you're connecting to can't they?
- w8rbt 8y ago
  Yes, they can always see the metadata. To, from, ports, bytes sent, packets, time, duration of the flow, etc.
- gaadd33 8y ago
  Given so much is (sadly) behind cloudflare or on AWS, I'm not sure that helps them a huge amount without seeing actual packet contents
nullc 8y ago
> Performance? Due to DNS caching at the resolver level, it is probably faster to use Google's 8.8.8.8 or CloudFlare's 1.1.1.1, than anything local (where all dns requests are a MISS). It's perfectly possible to recursive resolve your misses to Google's DNS server if you want.
- spudlyo 8y ago
  > It's perfectly possible to recursive resolve your misses to Google's DNS server if you want. I don't think that's how recursive name servers work. It's been a while since I've had to reason about this but, for example if 'www.google.com' in not cache, it contacts a root domain server, then a 'com' domain server, then a 'google.com' domain server, which finally answers the query, which then gets cached by the recursive name server. Eventually the recursive name server on your family or organization's local LAN will have a decent cache hit ratio, and the round trip times to your local recursive server could be potentially an order of magnitude (2ms vs. 20ms) faster than talking to Google or CloudFlare. It's possible that your ISP can still know which DNS lookups you're doing by snooping the traffic between your recursive DNS server and other DNS servers on the net, but I'm guessing that they're not doing this because it's not as easy as just ingesting their own DNS logs.
  - hug 8y ago
    What you said is incorrect. Almost all DNS servers ever have an ability to set a DNS forwarder instead of using root hints.
rannug 8y ago
> I would love a de-centralized web, but it is pretty hard to go back. As the powers that be continue to centralize and exert control in a negative way, I have a feeling the pendulum will swing the other way once people get annoyed with it. Kids are already using VPNs to circumvent controls. Need to make a cyberpunk-esque decentralization kit for the next generation to adopt.
- deleted 8y ago
  [deleted]
- bo1024 8y ago
  That would be really cool! I feel like one of the surprisingly-big barriers is just the difficulty of geting a static IP address assigned to your home. If you could do that, then (I think) they could run everything from a Raspberry Pi: their own website, hosting their own email, etc etc. (It's actually not much harder technically to set this all up on a VPS, but then the kid has to put a monthly fee on a card -- probably a big barrier for parents.) Am I right, or is there an easy way to get around the static IP issue?
  - rannug 8y ago
    I don't have a static IP at home. It changes every few weeks. I have a hack where a cron job uploads my home IP to a cloud server so I can know what it is when the IP changes. Maybe a Distributed Hash Table DNS interface is in order? I think it could work if you cache your peers, and reach out to let them know your current state. Even during the Venezuela blackouts, not all IPs went dark.
    - bo1024 8y ago
      That would be really cool! The question I see is how to handle resolutions where two people both claim to be example.com. Maybe you could use some cryptography to make sure it's always whoever first claims a domain.
    - jlgaddis 8y ago
      > I have a hack where a cron job uploads my home IP to a cloud server so I can know what it is when the IP changes. I don't have a clue what a "Distributed Hash Table DNS interface" is, but I'll note that a solution [0] to this problem has been around for over two decades. [0]: https://tools.ietf.org/html/rfc2136 https://tools.ietf.org/html/rfc2136
rhizome 8y ago
I've been thinking there could be a kind of NUC with cable or DSL or fiber adapter cards, and a bunch of basic services, with a good firewall and whatnot. And an update service, ideally free, for as forever as possible.
uponcoffee 8y ago
Pihole's default lists and using opendns upstream is pretty solid. In my experience their databases are thorough.
mrighele 8y ago
> Performance? Due to DNS caching at the resolver level, it is probably faster to use Google's 8.8.8.8 or CloudFlare's 1.1.1.1, than anything local (where all dns requests are a MISS). Some resolvers (like unbound) can be configured to prefetch cached entries before they expire. And in any case if you use something locally you are probably going to configure it to fetch recursively from one of the "big guys"
hazeii 8y ago
Single data point, but I've been running a home DNS server (bind) for many years; it's set to be authoritative for the .local domain and caching for everything else (except for major tracking and advertising sites, which it blackholes). For hits that are in the cache (the usual case) it's obviously faster than going out to the 'net,. The black-holing combined with ad-blockers mean browsing is a lot faster and considerably more peaceful. In terms of maintenance, it's no real effort other than updating every few years (being behind NAT the security risks aren't huge) and it means the entire household sees the benefit (plus access to webservers/wiki's etc on the .local domain). Obviously the downside is it needs to be on something that runs 24x7 and a modicum of IT skills are required to set it up. One other catch is that your ISP might block you for not using their DNS; BT (UK ISP) did this, but it is possible to turn off this 'security feature' via a rather obfuscated web page (may have changed since I last did it).
- kosinus 8y ago
  The .local TLD is reserved for mDNS. You may run into devices that completely refuse to resolve hosts in it using regular DNS.
  - hazeii 8y ago
    Indeed, 'zero-configuration' only applies to users.
- zAy0LfpBZLC8mAC 8y ago
  > One other catch is that your ISP might block you for not using their DNS; BT (UK ISP) did this Erm ... what? How does that work? If they don't see DNS requests from you at their resolver for a week, they disable your connection?!
  - hazeii 8y ago
    What happened is going online resulted in everything being redirected to a BT page saying you're not using our DNS and to change settings so you do. Some googling revealed a few people who'd had the same issue and found the (obscure) page that allowed you to undo the block. I assume they detected it simply by seeing DNS queries going to non-BT servers. Note this was a few years ago when it was pretty common for PC malware to hijack DNS requests, so could be it's changed in the meantime. N.B. I also recall BT redirecting requests for non-existent domains to some partner of theirs, I assume experimentally as I haven't seen or heard of that for a while.
znpy 8y ago
Regarding mailservers, it’s feasible to run one at home. But many people rely on spamlists, ie lists of ips known to relay spam. The proble with this is twofold: 1. Some people just took the authority to decide who sends spam and who does not. If you get on one of those lists, usually you have to get in touch and pay to get out. 2. Such people usually include residential ip subnets by default, for no technical reason whatsoever. So in the end my mailserver at home has been in a spamlist for years, even though i never relayed spam and was very careful to configure outbound relay authentication/authorisation, spf, dkim etc.
- zAy0LfpBZLC8mAC 8y ago
  > 2. Such people usually include residential ip subnets by default, for no technical reason whatsoever. Are you talking about residential or about dynamic? Because there kinda is a reason for this for dynamic addresses (PC malware sending spam, and the impossibility to list the particular affected PC because it's constantly changing addreses, so you only can block all the addresses those PCs could be using). If you do have static addresses, whether residential or not, those should not be listed in dialup block lists. If your home internet connection has dynamic addresses, you still can run your mail server at home by renting some tiny VPS and tunneling its addresses to your home server ...
  - znpy 8y ago
    > If you do have static addresses, whether residential or not, those should not be listed in dialup block lists. Indeed my residential internet connection has a static address but it still gets flagged for spam because it’s inside a residential subnet.
    - zAy0LfpBZLC8mAC 8y ago
      > because it’s inside a residential subnet. As in? I mean, what makes it a "residential subnet"? Have you tried talking to your ISP about this?
      - 8y ago
vlkramer 8y ago
With 8.8.8.8 Google sees all requests, which is probably the reason for its existence. How can an ISP not see what you are doing anyway? traceroute $addr obviously always includes ISP servers. Unless you use a VPN, but that is a different story.
xorcist 8y ago
> Performance? Due to DNS caching at the resolver level, Not sure about the validity these arguments. Yes, Google is likely to have more cached data than you, but they can also be a half a country or two away. In my experience caches matter less than one might think, since more popular data is usually the one that's low latency anyway. > Privacy? With DNS over TLS/DNS over HTTPS, your ISPs can't see Let's agree on one thing: That surf data is a lot more valuable to Google than most other actors, including your ISP, because they're the ones in a position to monetize it. Your ISP has more data than they know what to do with anyway. Should they try to monetize it despite the murky legal waters (they really wouldn't want to knowingly help copyright infringement, for example), the realistic option would be for them to sell it to someone very much like Google. It should not come as a surprise that the latter is happy to shortcut the process.
- desdiv 8y ago
  >Yes, Google is likely to have more cached data than you, but they can also be a half a country or two away. Google has edge nodes in pretty much every ISP of every country except China. A query to Google's public DNS never leaves your ISP, much less your country. >When clients send queries to Google Public DNS, they are routed to the nearest location advertising the anycast address used (8.8.8.8, 8.8.4.4, or one of the IPv6 addresses in 2001:4860:4860::). The specific locations advertising these anycast addresses change due to network conditions and traffic load, and include nearly all of the Core data centers and Edge Points of Presence (PoPs) in the Google Edge Network. https://developers.google.com/speed/public-dns/faq https://developers.google.com/speed/public-dns/faq
vbezhenar 8y ago
Is there resolver with Tor proxy? I love being able to use onion websites from browser.
mobilemidget 8y ago
Google cache only exists at local nodes in the cluster behind 8.8.8.8; chances you hitting the same node behind 8.8.8.8 for non fb like domains are slim.
kingo55 8y ago
> Privacy? With DNS over TLS/DNS over HTTPS, your ISPs can't see what you are doing. If you run DNS locally, they can. Use DNS Crypt Proxy. It acts as a DNS forwarder, cache and ad blocker for your entire network whilst also encrypting your lookups to 8.8.8.8 and 1.1 (Fun fact: did you know 1.1 is short for CloudFlare 1.0.0.1 BTW?)
psyclobe 8y ago
I run my own dns server with a forward rule to a local cloudflared dns proxy. ISP's can't see my queries.

by rmdoss 8y ago

fiddlerwoaroof 8y ago

My browsing habits are pretty regular, though: there’s half a dozen sites I visit regularly and the rest are random blogs/etc. I suspect about 20% of the domains I visit account for 80% of my browsing traffic and there would be enormous benefits from a dns speed perspective to a local caching resolver.

js2 8y ago
Maybe. Many sites use pretty short TTLs. So your sites may be dropping out of your cache more frequently than you think. Easy to test though. Run dnsmasq and enable query logging and see how often it’s having to forward requests. Then realize a recursing resolver is potentially having to go all the way to the TLDs for those requests.
- zcid 8y ago
  unbound for instance provides 'cache-min-ttl' which allows you to prevent excessively small ttl values.

jasonhansel 8y ago

I'd really like it if there was an easy way for "ordinary" consumers to setup a web/email/etc. server on, say, an AWS or GCP instance. Wouldn't exactly be decentralized, but it'd give users more control.

fonosip 8y ago
For pi-hole dns vpn self-hosted https://ba.net/adblock/vpn/doc/howto.html https://ba.net/adblock/vpn/doc/howto.html
johnny22 8y ago
It must handle backups and restores too, otherwise you're just making it too easy for folks to lose all their data.

tambourine_man 8y ago

The only chance we have for people to run their own services is if we make it dead easy to do so and the advantages are clearly communicated. Unfortunately, that hasn’t been open source’s forte historically.

wvh 8y ago
Until one of the big players decides they don't like your domain or IP range. I've been running my own email/web server for years, but even if you'd fully automate installation, there's a can of worms full of fun things like spam, DoS and shenanigans by the big players that really are not worth your time for just one person's email setup. I suppose we could come up with some easy configurable templates that would automatically install servers for privacy conscious individuals, but if at some point something goes wrong, most people are going to be stuck without service and no easy fix.
Reason077 8y ago
> ”The only chance we have for people to run their own services is if we make it dead easy” In the case of DNS, it’s dead easy if the server is built in to home routers. Many of them do, in fact, ship with local DNS already - it’s just that many users override it with 8.8.8.8 or whatever.
- codetrotter 8y ago
  The DNS servers shipped in home routers are usually set up as caching DNS proxies by default, and will forward queries for cache misses to the DNS resolvers or caching proxies that your ISP is running, rather than the DNS servers in home routers doing the full resolution themselves. So it’s not what I would consider “true” “local DNS”. However, even if we did switch everyone to running their own DNS resolvers, what would happen then? Without the massive shared caching we have today, the load would significantly increase on the authorative DNS servers for each domain. Even with client local caching. So the number of companies running their own authorative DNS servers would probably decrease — more of them would be using hosted DNS provided by a third party. A lot of companies host the authorative DNS of their domains with a third party big already. Including myself — I use Cloudflare for all my sites because of the HTTP caching and other things they offer on top of hosted DNS. Increased load on authorative servers will likely lead to further centralization of DNS hosting with a few big providers IMO. Because even a lot companies that specialize in hosting DNS might not be able to handle the load when everyone is running their own resolver. Only the big DNS hosting companies will be able to afford it. So we end up with everyone hosting DNS with a few DNS hosting providers — Cloudflare, Amazon Route 53, etc. So by decentralizing the DNS resolvers that clients use, you push companies to centralize the authorative DNS servers further. The net effect is that you will only have shifted where in the resolution the queries centralize. And let’s say that this happens and Google sees the amount of queries received by 8.8.8.8 drop to near zero over night. Odds are that if Google values the data they gain from clients using these resolvers, they will make a big push to ensure that they host the DNS for as many companies as possible, so that they still end up with their hands on the query data. (And Google does value this data — otherwise they wouldn’t still be offering public DNS query servers.) And also, what about the root servers? Will they be able to handle the massive increase in load? And won’t the root server traffic be subject to surveillance by state actors wanting to know what sites someone is browsing? DNS is kind of funny because in a way it is both centralized and decentralized at the same time. But if you want the web to be truly decentralized I believe for the reasons stated above that having people run their own DNS resolvers is not part of the solution. You are going to have to replace DNS altogether. Realistically I don’t think DNS is going away anytime soon. The web and the internet in general is too reliant on it. But I really wish we could.

timbit42 8y ago

Won't IPv6 make this easier? Isn't the primary reason people don't run their own servers now because of NAT?

Vosporos 8y ago
The other reason is that maintaining a server and its services is sometimes a full-time job :/
- stoolpigeon 8y ago
  This is anecdotal but I decided to host my own sites. I bought the cheapest droplet from digital ocean that I could and I set it up running Fedora. Then I installed Apache, MySQL and PHP so I could run some wordpress sites. The server kept running out of memory and shutting down MySQL so my sites stopped working. I started to learn how to read logs and saw that there is a huge amount of malicious activity directed at my server all the time. Yesterday I installed Fail2ban which meant installing postfix. I got it working (took most of the day) but I can't send emails to my gmail account because I need to set up dkim and dmarc and other stuff. I have a list. But first I have to learn how to do all this stuff. I use Fedora every day at work but I'm obviously no sysadmin and you are so right. All this takes hours and hours to learn and then to stay on top of it. When I was on shared hosting I had a lot less control and options but it was a lot easier as well.
  - wolco 8y ago
    Move mysql to a separate droplet. vultr: $2.50 otherwise you have to change mysql defaults to get things to fit. Great tutorials are out there in general reduce your workers/processes. Why do you need dkim or dmarc to send to gmail? If you send a test php mail does gmail pick it up?
    - stoolpigeon 8y ago
      If I send a test email with postfix - gmail rejects it and gives me a link to their page explaining why I need to add a bunch of stuff so they know I'm not spamming or fishing. Specifically I get this: gmail-smtp-in.l.google.com[173.194.207.27] said: 550-5.7.1 This message does not have authentication information or fails to pass 550-5.7.1 authentication checks. To best protect our users from spam, the 550-5.7.1 message has been blocked. Please visit 550-5.7.1 https://support.google.com/mail/answer/81126#authentication https://support.google.com/mail/answer/81126#authentication for more 550 5.7.1 information. d203si1756652qkb.228 - gsmtp (in reply to end of DATA command) there are tons of guides on troubleshooting mysql resource issues. Are they great? I don't know. Have I tried what is mentioned in many of them? Yes. I still have issues. I don't think it's just mysql though. I think it is a lot of little things that I'm slowly eliminating one by one.

johnklos 8y ago

Performance for Google or Cloudflare isn't going to be better. Where do you get that idea? Do you think all DNS simply lives in their caches? "your ISPs can't see what you are doing". If they're analyzing traffic, they can, and if they're doing that, they can see to whom I'm connecting, anyway. But you say nothing about why we should trust Google or Cloudflare. I trust my ISP to be big and dumb. I trust Goole and Cloudflare to want to make money.

icedchai 8y ago
Yes. Performance of Google / Cloudflare DNS will be better simply because so many other people are using them: any common DNS query result will probably already be cached... FYI I run my own DNS server anyway.
- zAy0LfpBZLC8mAC 8y ago
  But what is common for you will usually be cached already in your own server as well, so most requests will still be cache hits--and all those hits avoid the ~ 20 to 100 ms round trip to the internet. Take news.ycombinator.com, for example: The A record has a TTL of 300 seconds. So, after the first visit, which probably will take a bit longer than asking Google/CF, for every request in the next five minutes, you will have a reduced lookup latency. Then, after five minutes, the next lookup will go out to the authoritative server. But mind you that the NS records for ycombinator.com have a TTL of two days, so those are still cached, and the refresh is indeed a single request to the authoritative server--which more often than not takes about as long as a cache hit from the recursive Google/CF resolvers (it's also one round trip to the internet ...). And then, there is stuff like BIND's prefetch mechanism which will start the refresh of an expiring record when it sees a query for that record shortly before its expiry: That query is answered immediately from the cache, and a refresh is started in the background, so that the refreshed record should arrive in the cache before the old version expires ... thus completely eliminating the lookup latency for often-used records. Though you might need to tune it to trigger earlier for your personal use than in the default configuration ...
  - 8y ago

meruru 8y ago

>With DNS over TLS/DNS over HTTPS, your ISPs can't see what you are doing. Is this true? They still can see what IPs you're connecting to can't they?

w8rbt 8y ago
Yes, they can always see the metadata. To, from, ports, bytes sent, packets, time, duration of the flow, etc.
gaadd33 8y ago
Given so much is (sadly) behind cloudflare or on AWS, I'm not sure that helps them a huge amount without seeing actual packet contents

nullc 8y ago

spudlyo 8y ago
> It's perfectly possible to recursive resolve your misses to Google's DNS server if you want. I don't think that's how recursive name servers work. It's been a while since I've had to reason about this but, for example if 'www.google.com' in not cache, it contacts a root domain server, then a 'com' domain server, then a 'google.com' domain server, which finally answers the query, which then gets cached by the recursive name server. Eventually the recursive name server on your family or organization's local LAN will have a decent cache hit ratio, and the round trip times to your local recursive server could be potentially an order of magnitude (2ms vs. 20ms) faster than talking to Google or CloudFlare. It's possible that your ISP can still know which DNS lookups you're doing by snooping the traffic between your recursive DNS server and other DNS servers on the net, but I'm guessing that they're not doing this because it's not as easy as just ingesting their own DNS logs.
- hug 8y ago
  What you said is incorrect. Almost all DNS servers ever have an ability to set a DNS forwarder instead of using root hints.

rannug 8y ago

> I would love a de-centralized web, but it is pretty hard to go back. As the powers that be continue to centralize and exert control in a negative way, I have a feeling the pendulum will swing the other way once people get annoyed with it. Kids are already using VPNs to circumvent controls. Need to make a cyberpunk-esque decentralization kit for the next generation to adopt.

deleted 8y ago
[deleted]
bo1024 8y ago
That would be really cool! I feel like one of the surprisingly-big barriers is just the difficulty of geting a static IP address assigned to your home. If you could do that, then (I think) they could run everything from a Raspberry Pi: their own website, hosting their own email, etc etc. (It's actually not much harder technically to set this all up on a VPS, but then the kid has to put a monthly fee on a card -- probably a big barrier for parents.) Am I right, or is there an easy way to get around the static IP issue?
- rannug 8y ago
  I don't have a static IP at home. It changes every few weeks. I have a hack where a cron job uploads my home IP to a cloud server so I can know what it is when the IP changes. Maybe a Distributed Hash Table DNS interface is in order? I think it could work if you cache your peers, and reach out to let them know your current state. Even during the Venezuela blackouts, not all IPs went dark.
  - bo1024 8y ago
    That would be really cool! The question I see is how to handle resolutions where two people both claim to be example.com. Maybe you could use some cryptography to make sure it's always whoever first claims a domain.
  - jlgaddis 8y ago
    > I have a hack where a cron job uploads my home IP to a cloud server so I can know what it is when the IP changes. I don't have a clue what a "Distributed Hash Table DNS interface" is, but I'll note that a solution [0] to this problem has been around for over two decades. [0]: https://tools.ietf.org/html/rfc2136 https://tools.ietf.org/html/rfc2136

rhizome 8y ago

I've been thinking there could be a kind of NUC with cable or DSL or fiber adapter cards, and a bunch of basic services, with a good firewall and whatnot. And an update service, ideally free, for as forever as possible.

uponcoffee 8y ago

Pihole's default lists and using opendns upstream is pretty solid. In my experience their databases are thorough.

mrighele 8y ago

> Performance? Due to DNS caching at the resolver level, it is probably faster to use Google's 8.8.8.8 or CloudFlare's 1.1.1.1, than anything local (where all dns requests are a MISS). Some resolvers (like unbound) can be configured to prefetch cached entries before they expire. And in any case if you use something locally you are probably going to configure it to fetch recursively from one of the "big guys"

hazeii 8y ago

Single data point, but I've been running a home DNS server (bind) for many years; it's set to be authoritative for the .local domain and caching for everything else (except for major tracking and advertising sites, which it blackholes). For hits that are in the cache (the usual case) it's obviously faster than going out to the 'net,. The black-holing combined with ad-blockers mean browsing is a lot faster and considerably more peaceful. In terms of maintenance, it's no real effort other than updating every few years (being behind NAT the security risks aren't huge) and it means the entire household sees the benefit (plus access to webservers/wiki's etc on the .local domain). Obviously the downside is it needs to be on something that runs 24x7 and a modicum of IT skills are required to set it up. One other catch is that your ISP might block you for not using their DNS; BT (UK ISP) did this, but it is possible to turn off this 'security feature' via a rather obfuscated web page (may have changed since I last did it).

kosinus 8y ago
The .local TLD is reserved for mDNS. You may run into devices that completely refuse to resolve hosts in it using regular DNS.
- hazeii 8y ago
  Indeed, 'zero-configuration' only applies to users.
zAy0LfpBZLC8mAC 8y ago
> One other catch is that your ISP might block you for not using their DNS; BT (UK ISP) did this Erm ... what? How does that work? If they don't see DNS requests from you at their resolver for a week, they disable your connection?!
- hazeii 8y ago
  What happened is going online resulted in everything being redirected to a BT page saying you're not using our DNS and to change settings so you do. Some googling revealed a few people who'd had the same issue and found the (obscure) page that allowed you to undo the block. I assume they detected it simply by seeing DNS queries going to non-BT servers. Note this was a few years ago when it was pretty common for PC malware to hijack DNS requests, so could be it's changed in the meantime. N.B. I also recall BT redirecting requests for non-existent domains to some partner of theirs, I assume experimentally as I haven't seen or heard of that for a while.

znpy 8y ago

Regarding mailservers, it’s feasible to run one at home. But many people rely on spamlists, ie lists of ips known to relay spam. The proble with this is twofold: 1. Some people just took the authority to decide who sends spam and who does not. If you get on one of those lists, usually you have to get in touch and pay to get out. 2. Such people usually include residential ip subnets by default, for no technical reason whatsoever. So in the end my mailserver at home has been in a spamlist for years, even though i never relayed spam and was very careful to configure outbound relay authentication/authorisation, spf, dkim etc.

zAy0LfpBZLC8mAC 8y ago
> 2. Such people usually include residential ip subnets by default, for no technical reason whatsoever. Are you talking about residential or about dynamic? Because there kinda is a reason for this for dynamic addresses (PC malware sending spam, and the impossibility to list the particular affected PC because it's constantly changing addreses, so you only can block all the addresses those PCs could be using). If you do have static addresses, whether residential or not, those should not be listed in dialup block lists. If your home internet connection has dynamic addresses, you still can run your mail server at home by renting some tiny VPS and tunneling its addresses to your home server ...
- znpy 8y ago
  > If you do have static addresses, whether residential or not, those should not be listed in dialup block lists. Indeed my residential internet connection has a static address but it still gets flagged for spam because it’s inside a residential subnet.
  - zAy0LfpBZLC8mAC 8y ago
    > because it’s inside a residential subnet. As in? I mean, what makes it a "residential subnet"? Have you tried talking to your ISP about this?
    - 8y ago

vlkramer 8y ago

With 8.8.8.8 Google sees all requests, which is probably the reason for its existence. How can an ISP not see what you are doing anyway? traceroute $addr obviously always includes ISP servers. Unless you use a VPN, but that is a different story.

xorcist 8y ago

> Performance? Due to DNS caching at the resolver level, Not sure about the validity these arguments. Yes, Google is likely to have more cached data than you, but they can also be a half a country or two away. In my experience caches matter less than one might think, since more popular data is usually the one that's low latency anyway. > Privacy? With DNS over TLS/DNS over HTTPS, your ISPs can't see Let's agree on one thing: That surf data is a lot more valuable to Google than most other actors, including your ISP, because they're the ones in a position to monetize it. Your ISP has more data than they know what to do with anyway. Should they try to monetize it despite the murky legal waters (they really wouldn't want to knowingly help copyright infringement, for example), the realistic option would be for them to sell it to someone very much like Google. It should not come as a surprise that the latter is happy to shortcut the process.

desdiv 8y ago
>Yes, Google is likely to have more cached data than you, but they can also be a half a country or two away. Google has edge nodes in pretty much every ISP of every country except China. A query to Google's public DNS never leaves your ISP, much less your country. >When clients send queries to Google Public DNS, they are routed to the nearest location advertising the anycast address used (8.8.8.8, 8.8.4.4, or one of the IPv6 addresses in 2001:4860:4860::). The specific locations advertising these anycast addresses change due to network conditions and traffic load, and include nearly all of the Core data centers and Edge Points of Presence (PoPs) in the Google Edge Network. https://developers.google.com/speed/public-dns/faq https://developers.google.com/speed/public-dns/faq

vbezhenar 8y ago

Is there resolver with Tor proxy? I love being able to use onion websites from browser.

mobilemidget 8y ago

Google cache only exists at local nodes in the cluster behind 8.8.8.8; chances you hitting the same node behind 8.8.8.8 for non fb like domains are slim.

kingo55 8y ago

> Privacy? With DNS over TLS/DNS over HTTPS, your ISPs can't see what you are doing. If you run DNS locally, they can. Use DNS Crypt Proxy. It acts as a DNS forwarder, cache and ad blocker for your entire network whilst also encrypting your lookups to 8.8.8.8 and 1.1 (Fun fact: did you know 1.1 is short for CloudFlare 1.0.0.1 BTW?)

psyclobe 8y ago

I run my own dns server with a forward rule to a local cloudflared dns proxy. ISP's can't see my queries.