8 ms·

All-in-One DNS block list

183 points by foray1010 6y ago

op03 6y ago
Are there any Firefox, Ubuntu ppl around? Can you guys bake this stuff (host blacklists) into browser, os autoupdates?
- Havoc 6y ago
  If you're familiar with *nix stuff I'd suggest just run a pi-hole.
  - chrisan 6y ago
    if youre not familiar I would suggest it too :) I've gotten a few friends started on pi hole and ended up with minecraft servers, free nas, kodi, and retropi
- wubbert 6y ago
  You can use hblock auto-update host lists on Ubuntu and other Linux distributions. https://github.com/hectorm/hblock https://github.com/hectorm/hblock
- jedisct1 6y ago
  Use DNSCrypt-proxy.
bertman 6y ago
I really appreciate projects like this because I'm sure keeping these lists up-to-date is not an easy task, and many people benefit from the efforts. That said: maybe it's just me, but I find their website[1] a bit...strange? It looks like one of those SAAS startup landing pages, you can pick your "pack of block list" ranging from "Tru lite" to "XTreme" etc... Or maybe it's supposed to be ironic and I just don't get it :) [1] https://energized.pro/ https://energized.pro/
- r1ch 6y ago
  Ironically their website doesn't work if you block 3rd party JS due to Cloudflare.
  - cptskippy 6y ago
    I abandoned NoScript because I felt like I spent more time whitelisting JS than browsing the web and other people just couldn't borrow my browser. NoScript really needs the ability to whitelist a TLD for providers like cloudflare.
    - aroch 6y ago
      They do? You can globally whitelist urls
      - Mathnerd314 6y ago
        The issue for Cloudflare is whitelisting the DDoS protection script included on each page. It's under a lot of URLs so it would have to be content-based.
- jlgaddis 6y ago
  > Let's make an annoyance free better open internet, altogether! As a part-time grammar Nazi, there are several things here that annoy the hell outta me! After reading that, I had to quickly abort and close the tab.
praveenweb 6y ago
I have been using NextDNS with a few block lists configured at the router level and device level. The internet experience has improved a lot since ads and trackers are blocked system wide. A few block lists that I would recommend: 1. Steven Hosts - https://github.com/StevenBlack/hosts https://github.com/StevenBlack/hosts 2. Adguard DNS - https://github.com/AdguardTeam/AdguardSDNSFilter https://github.com/AdguardTeam/AdguardSDNSFilter 3. disconnect.me The amount of DNS requests made silently in the background is astonishing across all devices.
- wjnc 6y ago
  Me too! It's blocking a whopping 25%-30% of all requests without a single negative change in my browsing comfort.
  - t212 6y ago
    I really want to use it but it blocks Spotify playlist links when I try to open them from Reddit.
    - wjnc 6y ago
      If your really want to use it you should try diving into what blocklist is causing the block OR whitelist the URL.
    - mbdesign 6y ago
      Check your Nextdns.io logs and see what gets blocked. Then add the blocked domain (or the whole root domain) to the Whitelist. The whitelist always overrides any blocklist entry.
    - MattieTK 6y ago
      They added a new setting recently to allow affiliate links like those used for tracking which might include this, try it out.
    - garren 6y ago
      On Apple devices it’s as easy as toggling a switch to temporarily disable it. Not perfect but easy enough. Still trying to figure out a similar solution on Linux...
jedisct1 6y ago
Some other blacklists here: https://github.com/dnscrypt/dnscrypt-proxy/wiki/Public-blacklists https://github.com/dnscrypt/dnscrypt-proxy/wiki/Public-black... Along with a script that aggregates data from multiple lists, removes duplicates/overlaps, whitelists, etc.
Wronnay 6y ago
I am pretty happy with https://www.reddit.com/r/oisd_blocklist/ https://www.reddit.com/r/oisd_blocklist/ as a All-in-One Solution ...
- foray1010 6y ago
  Not bad too! But I really prefer opensource solution more, at least we know how it builds. Because in theory the list can be used for DNS hijacking
- chance_state 6y ago
  I use this with my Pi-Hole. Works very well. Along with a few other lists the Pi-Hole blocks about 30% of requests with almost no changes on the user end.
stblack 6y ago
This repo has a 3-week track record, by one contributor. Disclosure: Some of us have been actively curating such amalgamated lists for a long time. https://github.com/StevenBlack/hosts https://github.com/StevenBlack/hosts
- ciarannolan 6y ago
  What are the dangers of using a sketchy blocklist?
  - willis936 6y ago
    I'm no expert, but the most obvious answer is "you MITM yourself". If there is a userbase for a list, they have to trust the list to not filter out domains that shouldn't be filtered. I have a hard time thinking how this could lead to hidden repercussions, other than some security flaw that is only exploitable when some subset of requests go through.
    - ciarannolan 6y ago
      Perhaps they can point a particular host to a malicious IP rather than "0.0.0.0". In a list of several hundred thousand domains, you wouldn't be able to notice this manually. ex., make Bank of America resolve to a phishing site rather than the real BoA IP. Pi-Hole and others might check for this though, I don't know.
      - willis936 6y ago
        I am under the impression that the blocklist programs (such as ublock origin or pi-hole) do not have an option to redirect to anything other than the void. I can only see downsides to allowing this.
        contravariant 6y ago
        Actually ublock-origin has some options to replace Javascripts with custom (presumably less intrusive) scripts. Although I don't think 3rd party lists can do this. Anyway the repository in this post also provides host files, which most definitely can redirect you to malicious IPs. Edit: Turns out 3rd party block lists can use the redirect feature but only to Ublock Origin managed resources: https://github.com/gorhill/uBlock/wiki/Resources-Library https://github.com/gorhill/uBlock/wiki/Resources-Library
3ace 6y ago
right now I actively using https://doh.tiar.app https://doh.tiar.app maintained by my friend https://twitter.com/pengelana/ https://twitter.com/pengelana/
bluedino 6y ago
I've been using blockslists from a couple github repositories for a long time, heck probably since they were found on regular web pages. They work pretty good, but can be a little cumbersome to turn off or to enable certain domains from time to time (such as when a site has so many ads it breaks the site). But the increases safety and speed while surfing is well worth it.
surround 6y ago
How to make a block list: 1) Find a bunch of high-quality block lists on the internet which have been painstakingly curated my their maintainers for many years 2) Combine it all into one big list. Tell everyone that you will quickly whitelist any domains if they are causing breakage. 3) Once enough people start using your list, get an advertiser to pay you to silently remove their domains. If anyone notices, just say it was to fix breakage on some obscure site. I’m not saying that Energized or StevenBlack are doing step 3, but please realize that there are issues with using lists like these. Even if they aren’t getting paid, they might still have some undesirable whitelisted domains. They also deprive the original block list maintainers of views (meaning they might be less inclined to continue maintaining them). You also won’t receive updates from the lists as quickly because of the middle man. If you are using Pi—hole, OPNsense, or any other tool which can run multiple block lists simultaneously, I recommend taking a look at https://firebog.net https://firebog.net for a list of original-source block lists.
- stblack 6y ago
  I can see how it might seem that way — I'm Steven Black. I can point to thousands of combined... * issues https://github.com/StevenBlack/hosts/issues?q=is%3Aissue+is%3Aclosed https://github.com/StevenBlack/hosts/issues?q=is%3Aissue+is%... * ...pull requests... https://github.com/StevenBlack/hosts/pulls?q=is%3Apr+is%3Aclosed https://github.com/StevenBlack/hosts/pulls?q=is%3Apr+is%3Acl... * ... and commits https://github.com/StevenBlack/hosts/commits/master https://github.com/StevenBlack/hosts/commits/master ...that indicate, it's not so easy. The sources we use are all vetted. Some sources are remarkable in terms of activity, and responsiveness to problems as they occur. Overall I think this area is far more dynamic than many realize. Some good people curate the lists we carry.
  - surround 6y ago
    I don’t mean to belittle your project - Consolidated block lists like yours are still important for people who are using the traditional etc/hosts file. I believe the Pihole project would like to focus more on the software and less on the block lists - so they include your list, which has a good track record of vetting sources and responding to issues. I also appreciate that your project has produced its own original block lists (which happen to be included on the Energized list and firebog.net). I just wish more people would use the original source when possible. I dislike this Energized list, partly because I had a bad experience using one of their non-primary lists which wasn’t well maintained, and partly because their website (https://energized.pro/ https://energized.pro/) makes it sound like a commercial product.
rndomsrmn 6y ago
Also a ref to: https://github.com/notracking/hosts-blocklists https://github.com/notracking/hosts-blocklists They have a public whitelist and updates are pushed on a daily basis.
balboah 6y ago
This list can be used on Android or iOS by downloading the Blokada app
depressedCorgi 6y ago
Very cool, I’m gonna try this out with PiHole.
lightswitch05 6y ago
Looks like my lists are intended to be included, but it was linking to the raw Github source instead of the hosted Github pages version. I went through a major refactor 21 days ago that moved my sources lists around a bit - but preserved the links that are supplied all over the README and the Github hosted pages. So, not only is the project linking to the wrong place, but my list has been broken in it for 21 days now without notice. Its fine that people love creating these massive all-in-one lists. But I recommend just using the sources directly. That way, if a list gives you trouble, you know who to open a ticket with, or just disable that specific list if its too aggressive for your tastes. My lists: https://www.github.developerdan.com/hosts/ https://www.github.developerdan.com/hosts/
foobar_ 6y ago
Fundamentally D.N.S is a naming system but each site has a separate naming system via user names. Something like this should also be applicable for social networks as well. I found this for twitter - https://blocktogether.org/ https://blocktogether.org/ not sure if it is possible for others like facebook.
goldfix 6y ago
Alternative client to update your hosts file: https://github.com/goldfix/pigHosts https://github.com/goldfix/pigHosts
dastx 6y ago
Is there any tools out there that I can use to generate my own aggregated lists from a set of other blocklists? Ideally it leverages things like GitHub Actions (or another CI tool) + GH Pages/GH releases/Netlify to relief the burden of having to host it myself. The reason for this is so that I can use NetGuard, which allows for only 1 blocklist. Currently I'm flipping between Blokada and DNS66 because they allow for multiple lists.

183 points by foray1010 6y ago

op03 6y ago

Are there any Firefox, Ubuntu ppl around? Can you guys bake this stuff (host blacklists) into browser, os autoupdates?

Havoc 6y ago
If you're familiar with *nix stuff I'd suggest just run a pi-hole.
- chrisan 6y ago
  if youre not familiar I would suggest it too :) I've gotten a few friends started on pi hole and ended up with minecraft servers, free nas, kodi, and retropi
wubbert 6y ago
You can use hblock auto-update host lists on Ubuntu and other Linux distributions. https://github.com/hectorm/hblock https://github.com/hectorm/hblock
jedisct1 6y ago
Use DNSCrypt-proxy.

bertman 6y ago

I really appreciate projects like this because I'm sure keeping these lists up-to-date is not an easy task, and many people benefit from the efforts. That said: maybe it's just me, but I find their website[1] a bit...strange? It looks like one of those SAAS startup landing pages, you can pick your "pack of block list" ranging from "Tru lite" to "XTreme" etc... Or maybe it's supposed to be ironic and I just don't get it :) [1] https://energized.pro/ https://energized.pro/

r1ch 6y ago
Ironically their website doesn't work if you block 3rd party JS due to Cloudflare.
- cptskippy 6y ago
  I abandoned NoScript because I felt like I spent more time whitelisting JS than browsing the web and other people just couldn't borrow my browser. NoScript really needs the ability to whitelist a TLD for providers like cloudflare.
  - aroch 6y ago
    They do? You can globally whitelist urls
    - Mathnerd314 6y ago
      The issue for Cloudflare is whitelisting the DDoS protection script included on each page. It's under a lot of URLs so it would have to be content-based.
jlgaddis 6y ago
> Let's make an annoyance free better open internet, altogether! As a part-time grammar Nazi, there are several things here that annoy the hell outta me! After reading that, I had to quickly abort and close the tab.

praveenweb 6y ago

I have been using NextDNS with a few block lists configured at the router level and device level. The internet experience has improved a lot since ads and trackers are blocked system wide. A few block lists that I would recommend: 1. Steven Hosts - https://github.com/StevenBlack/hosts https://github.com/StevenBlack/hosts 2. Adguard DNS - https://github.com/AdguardTeam/AdguardSDNSFilter https://github.com/AdguardTeam/AdguardSDNSFilter 3. disconnect.me The amount of DNS requests made silently in the background is astonishing across all devices.

wjnc 6y ago
Me too! It's blocking a whopping 25%-30% of all requests without a single negative change in my browsing comfort.
- t212 6y ago
  I really want to use it but it blocks Spotify playlist links when I try to open them from Reddit.
  - wjnc 6y ago
    If your really want to use it you should try diving into what blocklist is causing the block OR whitelist the URL.
  - mbdesign 6y ago
    Check your Nextdns.io logs and see what gets blocked. Then add the blocked domain (or the whole root domain) to the Whitelist. The whitelist always overrides any blocklist entry.
  - MattieTK 6y ago
    They added a new setting recently to allow affiliate links like those used for tracking which might include this, try it out.
  - garren 6y ago
    On Apple devices it’s as easy as toggling a switch to temporarily disable it. Not perfect but easy enough. Still trying to figure out a similar solution on Linux...

jedisct1 6y ago

Some other blacklists here: https://github.com/dnscrypt/dnscrypt-proxy/wiki/Public-blacklists https://github.com/dnscrypt/dnscrypt-proxy/wiki/Public-black... Along with a script that aggregates data from multiple lists, removes duplicates/overlaps, whitelists, etc.

Wronnay 6y ago

I am pretty happy with https://www.reddit.com/r/oisd_blocklist/ https://www.reddit.com/r/oisd_blocklist/ as a All-in-One Solution ...

foray1010 6y ago
Not bad too! But I really prefer opensource solution more, at least we know how it builds. Because in theory the list can be used for DNS hijacking
chance_state 6y ago
I use this with my Pi-Hole. Works very well. Along with a few other lists the Pi-Hole blocks about 30% of requests with almost no changes on the user end.

stblack 6y ago

This repo has a 3-week track record, by one contributor. Disclosure: Some of us have been actively curating such amalgamated lists for a long time. https://github.com/StevenBlack/hosts https://github.com/StevenBlack/hosts

ciarannolan 6y ago
What are the dangers of using a sketchy blocklist?
- willis936 6y ago
  I'm no expert, but the most obvious answer is "you MITM yourself". If there is a userbase for a list, they have to trust the list to not filter out domains that shouldn't be filtered. I have a hard time thinking how this could lead to hidden repercussions, other than some security flaw that is only exploitable when some subset of requests go through.
  - ciarannolan 6y ago
    Perhaps they can point a particular host to a malicious IP rather than "0.0.0.0". In a list of several hundred thousand domains, you wouldn't be able to notice this manually. ex., make Bank of America resolve to a phishing site rather than the real BoA IP. Pi-Hole and others might check for this though, I don't know.
    - willis936 6y ago
      I am under the impression that the blocklist programs (such as ublock origin or pi-hole) do not have an option to redirect to anything other than the void. I can only see downsides to allowing this.
      - contravariant 6y ago
        Actually ublock-origin has some options to replace Javascripts with custom (presumably less intrusive) scripts. Although I don't think 3rd party lists can do this. Anyway the repository in this post also provides host files, which most definitely can redirect you to malicious IPs. Edit: Turns out 3rd party block lists can use the redirect feature but only to Ublock Origin managed resources: https://github.com/gorhill/uBlock/wiki/Resources-Library https://github.com/gorhill/uBlock/wiki/Resources-Library

3ace 6y ago

right now I actively using https://doh.tiar.app https://doh.tiar.app maintained by my friend https://twitter.com/pengelana/ https://twitter.com/pengelana/

bluedino 6y ago

I've been using blockslists from a couple github repositories for a long time, heck probably since they were found on regular web pages. They work pretty good, but can be a little cumbersome to turn off or to enable certain domains from time to time (such as when a site has so many ads it breaks the site). But the increases safety and speed while surfing is well worth it.

surround 6y ago

How to make a block list: 1) Find a bunch of high-quality block lists on the internet which have been painstakingly curated my their maintainers for many years 2) Combine it all into one big list. Tell everyone that you will quickly whitelist any domains if they are causing breakage. 3) Once enough people start using your list, get an advertiser to pay you to silently remove their domains. If anyone notices, just say it was to fix breakage on some obscure site. I’m not saying that Energized or StevenBlack are doing step 3, but please realize that there are issues with using lists like these. Even if they aren’t getting paid, they might still have some undesirable whitelisted domains. They also deprive the original block list maintainers of views (meaning they might be less inclined to continue maintaining them). You also won’t receive updates from the lists as quickly because of the middle man. If you are using Pi—hole, OPNsense, or any other tool which can run multiple block lists simultaneously, I recommend taking a look at https://firebog.net https://firebog.net for a list of original-source block lists.

stblack 6y ago
I can see how it might seem that way — I'm Steven Black. I can point to thousands of combined... * issues https://github.com/StevenBlack/hosts/issues?q=is%3Aissue+is%3Aclosed https://github.com/StevenBlack/hosts/issues?q=is%3Aissue+is%... * ...pull requests... https://github.com/StevenBlack/hosts/pulls?q=is%3Apr+is%3Aclosed https://github.com/StevenBlack/hosts/pulls?q=is%3Apr+is%3Acl... * ... and commits https://github.com/StevenBlack/hosts/commits/master https://github.com/StevenBlack/hosts/commits/master ...that indicate, it's not so easy. The sources we use are all vetted. Some sources are remarkable in terms of activity, and responsiveness to problems as they occur. Overall I think this area is far more dynamic than many realize. Some good people curate the lists we carry.
- surround 6y ago
  I don’t mean to belittle your project - Consolidated block lists like yours are still important for people who are using the traditional etc/hosts file. I believe the Pihole project would like to focus more on the software and less on the block lists - so they include your list, which has a good track record of vetting sources and responding to issues. I also appreciate that your project has produced its own original block lists (which happen to be included on the Energized list and firebog.net). I just wish more people would use the original source when possible. I dislike this Energized list, partly because I had a bad experience using one of their non-primary lists which wasn’t well maintained, and partly because their website (https://energized.pro/ https://energized.pro/) makes it sound like a commercial product.

rndomsrmn 6y ago

Also a ref to: https://github.com/notracking/hosts-blocklists https://github.com/notracking/hosts-blocklists They have a public whitelist and updates are pushed on a daily basis.

balboah 6y ago

This list can be used on Android or iOS by downloading the Blokada app

depressedCorgi 6y ago

Very cool, I’m gonna try this out with PiHole.

lightswitch05 6y ago

Looks like my lists are intended to be included, but it was linking to the raw Github source instead of the hosted Github pages version. I went through a major refactor 21 days ago that moved my sources lists around a bit - but preserved the links that are supplied all over the README and the Github hosted pages. So, not only is the project linking to the wrong place, but my list has been broken in it for 21 days now without notice. Its fine that people love creating these massive all-in-one lists. But I recommend just using the sources directly. That way, if a list gives you trouble, you know who to open a ticket with, or just disable that specific list if its too aggressive for your tastes. My lists: https://www.github.developerdan.com/hosts/ https://www.github.developerdan.com/hosts/

foobar_ 6y ago

Fundamentally D.N.S is a naming system but each site has a separate naming system via user names. Something like this should also be applicable for social networks as well. I found this for twitter - https://blocktogether.org/ https://blocktogether.org/ not sure if it is possible for others like facebook.

goldfix 6y ago

Alternative client to update your hosts file: https://github.com/goldfix/pigHosts https://github.com/goldfix/pigHosts

dastx 6y ago

Is there any tools out there that I can use to generate my own aggregated lists from a set of other blocklists? Ideally it leverages things like GitHub Actions (or another CI tool) + GH Pages/GH releases/Netlify to relief the burden of having to host it myself. The reason for this is so that I can use NetGuard, which allows for only 1 blocklist. Currently I'm flipping between Blokada and DNS66 because they allow for multiple lists.