3 ms·

I dumped the list of blocked domains to this pastebin, since it wasn't documented anywhere. https://pastebin.com/VHuHTeqp https://pastebin.com/VHuHTeqp Intere

I dumped the list of blocked domains to this pastebin, since it wasn't documented anywhere. https://pastebin.com/VHuHTeqp https://pastebin.com/VHuHTeqp Interesting that ycombinator.com is on the list, but not reddit And why would you block Soundcloud? Lots of starving musicians are scraping by on there, and many of them are doing great work.

dawnerd 2y ago
There’s at least one dot gov on there too. What a weird list
- Dalewyn 2y ago
  I saw at least two from a quick skim: bls.gov and cdc.gov. Assuming the dump is accurate, this is at best an impractical and at worst a misguided blacklist.
- duskwuff 2y ago
  There's a whole bunch of US government sites on the list (notable inclusions: fda.gov, nasa.gov, whitehouse.gov) for some reason, as well as quite a few non-corporate web properties: * Stack Exchange, but only one of their sites: superuser.com * Wikimedia: wikimedia.org, wikidata.org, and wiktionary.org (but not wikipedia.org) * Image hosting: imgur.com, giphy.com, tenor.com, deviantart.com, artstation.com * Audio and video hosting: soundcloud.com, vimeo.com, dailymotion.com * Blogging and web hosting: livejournal.com, squarespace.com, substack.com, typepad.com * Fan fiction: tvtropes.org, archiveofourown.org, fanfiction.net * Miscellaneous indie web properties: bogleheads.org, craigslist.org, ifixit.com, instructables.com, knowyourmeme.com, mozilla.org Overall this is a very strange list.
  - petertsfn 2y ago
    I hear you. I wrote my reasoning on a comment above, but the gist is: It's an afternoon project, I fucked up the list because I didn't think much about it, and this project is probably better off being crowdsourced. I think it's a proof of concept on how we can theoretically make searches better. Anyways, I'm pushing an update removing all the sites you mentioned. Do you have any other suggestions?
    - celdon25 2y ago
      That's fair tbh, I agree there is an unmet need for something like this, so no arguments there. It is only fair that the users should have at least an equal say in keeping companies off of their search results. The search engines are making the choices for you because you're the product, not the customer. The implementation details take time to get right. And getting this far in an afternoon as a non-developer is pretty respectable. If you want to have the most reach / impact with something like this though, it will take consistent, sustained effort over a long period of time, and it will be a thankless job, at least for awhile.
petertsfn 2y ago
You guys made a valid point. I got a list of 1000 highest traffic sites in the us in 2023, and I did a skim to remove sites that might be actually useful, but apparently I missed a lot. This was an afternoon project after I read the article about how 16 media companies own ~600 top media sites that's getting organic traffic. I submitted an update with a more cleaned list (no more .gov and a few other misfires). Truth is this project is probably better if it uses a crowdsource approach to the source list (like SponsorBlock). It's a bit out of scope for me since I'm not a developer. I just make things I think is fun. My hope with this thing is to raise more awareness to this problem of large brands publishing SEO garbage, and then maybe someone smarter will come along actually make searches usable again.
- ASalazarMX 2y ago
  If the default list is the weak point, why not start empty and let users hide as they go? You keep the value of the extension without the extra maintenance.
  - celdon25 2y ago
    He said he wasn't really a developer and this was more of a proof of concept. Having a configuration screen is a bigger task than having something hard coded

dawnerd 2y ago

There’s at least one dot gov on there too. What a weird list

Dalewyn 2y ago
I saw at least two from a quick skim: bls.gov and cdc.gov. Assuming the dump is accurate, this is at best an impractical and at worst a misguided blacklist.
duskwuff 2y ago
There's a whole bunch of US government sites on the list (notable inclusions: fda.gov, nasa.gov, whitehouse.gov) for some reason, as well as quite a few non-corporate web properties: * Stack Exchange, but only one of their sites: superuser.com * Wikimedia: wikimedia.org, wikidata.org, and wiktionary.org (but not wikipedia.org) * Image hosting: imgur.com, giphy.com, tenor.com, deviantart.com, artstation.com * Audio and video hosting: soundcloud.com, vimeo.com, dailymotion.com * Blogging and web hosting: livejournal.com, squarespace.com, substack.com, typepad.com * Fan fiction: tvtropes.org, archiveofourown.org, fanfiction.net * Miscellaneous indie web properties: bogleheads.org, craigslist.org, ifixit.com, instructables.com, knowyourmeme.com, mozilla.org Overall this is a very strange list.
- petertsfn 2y ago
  I hear you. I wrote my reasoning on a comment above, but the gist is: It's an afternoon project, I fucked up the list because I didn't think much about it, and this project is probably better off being crowdsourced. I think it's a proof of concept on how we can theoretically make searches better. Anyways, I'm pushing an update removing all the sites you mentioned. Do you have any other suggestions?
  - celdon25 2y ago
    That's fair tbh, I agree there is an unmet need for something like this, so no arguments there. It is only fair that the users should have at least an equal say in keeping companies off of their search results. The search engines are making the choices for you because you're the product, not the customer. The implementation details take time to get right. And getting this far in an afternoon as a non-developer is pretty respectable. If you want to have the most reach / impact with something like this though, it will take consistent, sustained effort over a long period of time, and it will be a thankless job, at least for awhile.

petertsfn 2y ago

You guys made a valid point. I got a list of 1000 highest traffic sites in the us in 2023, and I did a skim to remove sites that might be actually useful, but apparently I missed a lot. This was an afternoon project after I read the article about how 16 media companies own ~600 top media sites that's getting organic traffic. I submitted an update with a more cleaned list (no more .gov and a few other misfires). Truth is this project is probably better if it uses a crowdsource approach to the source list (like SponsorBlock). It's a bit out of scope for me since I'm not a developer. I just make things I think is fun. My hope with this thing is to raise more awareness to this problem of large brands publishing SEO garbage, and then maybe someone smarter will come along actually make searches usable again.

ASalazarMX 2y ago
If the default list is the weak point, why not start empty and let users hide as they go? You keep the value of the extension without the extra maintenance.
- celdon25 2y ago
  He said he wasn't really a developer and this was more of a proof of concept. Having a configuration screen is a bigger task than having something hard coded