15 ms·
Show HN: Extension-blocking domains removed by threat from other blacklists
- josteink 9y agoFirefox also supports webextensions. Please consider publishing a version at addons.mozilla.org too. The tool web-ext makes this almost effortless. https://github.com/mozilla/web-ext https://github.com/mozilla/web-ext
- paulgb 9y agoThanks, I plan to, the main reason I targeted Chrome at first is that I've heard the review period is in months for Firefox.
- josteink 9y agoThat's FUD. It's automated and takes minutes at tops. Edit: in my experience.
- OzzyB 9y agoNo, it takes months, stop FUDing the FUD.
- josteink 9y agoLast time I checked publishing and signing was even built into the tooling itself, making it much more seamless and CI-friendly than the Chrome counter-part. And the amo team is super responsive on email. What am I missing?
- detaro 9y agonot sure about details, but AFAIK there's two steps: an automated review when you sign an extension. You then can distribute that extension yourself. But if you want it to be hosted on Mozilla Addons, it needs human review, which takes more time.
- orclev 9y agoReading their process and guidelines I'm guessing that human review is highly variable as well, mostly depending on the size and complexity of the addon. Something massive that's structured poorly is going to take a long time to review since the poor person doing the review needs to actually be able to understand the code. A small addon or one that's structured very cleanly on the other hand should be a fairly fast review.
- anon1385 9y agoAs far as I can see it takes over 5 days at least 50% of the time: https://discourse.mozilla.org/t/queue-weekly-status-2017-08-11/17992 https://discourse.mozilla.org/t/queue-weekly-status-2017-08-... (E: and that's currently; go back a few months and it's nearly 90% taking over 10 days. Possibly well over 10 days. https://discourse.mozilla.org/t/queue-weekly-status-2017-05-12/15564 https://discourse.mozilla.org/t/queue-weekly-status-2017-05-...)
- josteink 9y agoI'm curious. Does this statistic mix old XUL based add-ons with the new web extension standard addins? As far as I can see the new web extensions based approached has much better tooling with automatic analysis and reviewing.
- DiThi 9y agoIt doesn't show which one are web extensions. Even then, users can access the unreviewed versions of addons, IIRC.
- Akkuma 9y agoThis isn't true if you use uglify at all. I literally had to go into IRC and try and figure out why a hotfix for our extension was taking so long to get through the process (we had been sitting in it for at least 3 days if not a week IIRC). The response I got in IRC was that I would have to literally come and ask for someone's assistance to bump up the review on my company's extension each time we hotfixed, as we got placed in a special queue that required an admin and not just any volunteer in part due to our use of uglify causing the code to be obfuscated. The worse part is that uglify is required for us to even get the extension signed, due to Firefox's arbitrary individual file size limit. Prior to Mozilla getting their act together it literally was over a month of waiting for them to review our extension, so we just bailed altogether as their entire process is terrible for pretty much no real gain. I then tried again once the queue had dropped from ~400 to ~150 and the wait time finally became reasonable. I mean the process was/is so bad, they literally had an issue on github to just automatically approve extensions based on some criteria due to the huge backlog. I cannot get an unlisted extension released with the same version as a listed extension, so we completely bailed on using Mozilla's hosting at all, since all it does is cause a liability until we can push out fixes the same day without manual intervention. To top this off, every single time we make a new release, we'd have to explain the extension, how to build it, etc. and provide sources to the code. Our extension is also a regular web extension.
- orclev 9y agoI'm curious to know what extension you're creating that's so massive it needs a minifier/obfuscator to be run on it so that it's small enough to fit under the file size limits. Also just to clarify, what is the file size you're being requested to be under? Are we talking kilobytes, or megabytes here, because a megabyte of JS is an awful lot of code in a single file. Taking a look at mozilla's publishing guidelines they're quite clear that you can publish an addon in obfuscated or minified form, but that you need to provide them an unobfuscated/unminified copy of the of the source code to review as well as instructions on how you performed the obfuscation/minification (presumably to run it themselves and compare the output). All of that seems fairly reasonable since you don't want people distributing malware on AMO. One thing I don't see mentioned anywhere in there is a size limit on files though, so I'm very interested to hear what this apparently undocumented requirement is.
- kbrosnan 9y agoYou can self host a signed extension in minutes. Just upload it and then download the signed extension from the developer portal and add it to the github releases for the project. https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Publishing_your_WebExtension https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Pu...
- s_kilk 9y agoNah, I've had extensions reviewed in a few hours
- paulgb 9y agoUpdate: I've submitted it for review by Mozilla.
- paulgb 9y agoApproved! https://addons.mozilla.org/en-US/firefox/addon/barbblock/ https://addons.mozilla.org/en-US/firefox/addon/barbblock/
- dvfjsdhgfv 9y agoKudos to the author for this. I wonder what the reaction of Admiral will be.
- marksomnian 9y ago"Well, st."
- jwilk 9y agoI think your comment fell victim to an asterisk eating monster.
- marksomnian 9y agoWell, st.
- troisx 9y agoConsidering that the CEO of Admiral considers himself a "beloved VC" (literally his words), I'm guessing he'll keep stepping up his agressive tactics. Typical sociopathic behavior from the type that believes that money trumps everything and that everyone worships him.
- cyphar 9y agoAs the author notes, the Streisand effect at work. But more importantly, I am quite happy that someone actually decided to stand their ground and call the bluff of future malware distributors (sorry, advertising companies). I've seen the chilling effects the DMCA has had on reasonable discourse in the YouTube community, but it extending it to what people can block in their browsers is absolute insanity.
- reificator 9y agoYoutube's setup, where anyone can claim any video, and the first line of defense creators have is to plead innocent to the same party making the claim, is the most user hostile (creator hostile?) thing I've ever seen.
- spaceribs 9y agoI've always found it funny that piracy still totally exists on youtube, but the pirates just add obfuscation techniques to prevent immediate takedowns. Then you watch the "Important Videos" playlist (https://www.youtube.com/playlist?list=PL7XlqX4npddfrdpMCxBnNZXg2GFll7t5y https://www.youtube.com/playlist?list=PL7XlqX4npddfrdpMCxBnN...) and half the videos are taken down because the account had too many strikes. It's really sad.
- _jal 9y agoFunny in a terrible sort of way. The legacy copyright industry follows the same logic as various drug war participants - winning means a loss of leverage to ask for more power.
- ethbro 9y agoSame discrepancy between reality and rhetoric. Stated goal: eliminate all of a thing for moral reasons. Actual goal: eliminate the easiest majority of a thing for economic reasons.
- SoreGums 9y ago
- AdmiralAsshat 9y agoI'm a fan of "Code as protest", but it seems like the more practical solution would be to simply have a separately maintained domain list that could be easily integrated into the adblockers that already used EasyList.
- deleted 9y ago[deleted]
- sergers 9y agoBelieve e that's how it was originally maintained, off GitHub, was just a http link to a text file that gets imported(still the same but off GitHub now). Just make the Ad blocking extension retrieve an extra file outside of github automatically(default on, but leave it configurable like the other filter lists in options) or I am sure there are custom lists based on easylist or other lists available already on the net that include/can include But props to the creator of this, great ingenuity.
- paulgb 9y agoI'm not sure how that would be done but I'm open to pull requests!
- maxerickson 9y agoYou just publish a set of rules like https://easylist-downloads.adblockplus.org/easylist.txt https://easylist-downloads.adblockplus.org/easylist.txt
- nileshtrivedi 9y agoAnd the file could be published on IPFS so hosting it is not relevant. Hash would ensure that you're getting the right file - from any IPFS node. There is even an HTTP bridge for IPFS content.
- StavrosK 9y agoThere are many HTTP bridges, I run one: https://www.eternum.io/ https://www.eternum.io/ Latest easylist: https://www.eternum.io/ipfs/QmdDtvs1QaQQG6NMpc2cpncA4zVZqgUKUs3LQdTtLditx7 https://www.eternum.io/ipfs/QmdDtvs1QaQQG6NMpc2cpncA4zVZqgUK... If you decide to use IPFS for distribution of your list, I'd be happy to donate some Eternum credits so you can pin it.
- suresh70 9y agoThis is one of the many situations where website owners,content creators and individuals being intimidated by using DMCA take down notice. Sometimes there are even fake notices as in here(https://www.eff.org/deeplinks/2016/10/samsung-sets-its-reputation-fire-bogus-dmca-takedown-notices?page=174 https://www.eff.org/deeplinks/2016/10/samsung-sets-its-reput...).There should be proper checks to avoid misuse of DMCA take down notice.
- jwilk 9y ago> This is not my first DMCA-takedown rodeo What's the story?
- paulgb 9y agotl;dr I found a major ticketing vendor was vulnerable to data interception because of a faulty HTTPS setup. They issued a DMCA takedown of a private video I had sent them. I fought it and they threatened to sue but they backed down after the EFF sent a response on my behalf.
- ishitatsuyuki 9y agoDMCA is targeted at service provider. Unless the code is self hosted, there's a risk that GitHub can take it down to avoid lawsuits.
- paulgb 9y agoYep, but I can file a counter-takedown. I know the process from past experience.
- chii 9y agoawesome - it would be good to publicize this process, and also to post about those experiences. Far too many are intimidated into compliance without knowing their rights.
- notyourday 9y agohttps://news.ycombinator.com/item?id=14992858 https://news.ycombinator.com/item?id=14992858 I'm rather shocked HN is not well versed in this.
- littlecranky67 9y agoThats why it would make more sense to create a list (not a custom extension), and host that list outside of github on a non-US server, where DMCA takedown notices can just be safely ignored.
- chii 9y agoi think that has the bad side effect of not disputing invalid DMCA takedowns. Instead, we ought to be fighting these invalid DMCA takedowns, and increase the cost of making a false claim. In aggregate, it will make DMCA less abusable, but will require a tough fight. Making bad press for companies that make false claims won't hurt either. Moving information to outside the US juristiction will only allow abusers to continue to harass those who can't move out.
- rocky1138 9y agoA much easier solution is to just add those domains to your hosts file. Then it'll work across any program or browser you use.
- deleted 9y ago[deleted]
- visarga 9y agoWhat if Samsung issues another bogus DMCA? Would you dare blacklist Samsung? If the inconvenience level is high enough, almost nobody would use the blacklist. This only works for small players.
- michaelmrose 9y agoIts not the businesses primary website that would normally be the target of blacklists. Its domains serving undesirable content like ads. For samsung.com to be added to the list first an ad-blocker would have to list samsung.com THEN Samsung would have to use legal shenanigans to get samsung.com removed from blacklists THEN it would get added. It seems pretty unlikely. Its likely in fact that if samsung had content worthy of blocking it would be served in a way that would be easy to block without blocking samsung.com. Example nonsense.samsung.com or samsung.com/whatever.
- dvl 9y agoNot first time, it's stupid how GitHub accepts anything as DMCA request.
- paulgb 9y agoDon't blame GitHub, blame the DMCA!
- hungerstrike 9y agoI can still blame GitHub for not fighting against the DMCA with us.
- dvl 9y agoI blame both. Even without any copyright violation GitHub takes the DMCA request.
- WillPostForFood 9y agoYou have to take the DMCA request or else you lose safe harbor.
- chii 9y agothe problem is when the safe-harbour entity doesn't actually try to protect their content creators, but blindly accepts any takedown. Youtube is notoriously obedient - to the point where it can threaten content creator's livelihood (see https://www.youtube.com/watch?v=QfgoDDh4kE0 https://www.youtube.com/watch?v=QfgoDDh4kE0 for a famous one).
- WillPostForFood 9y agoDoesn't matter - they have to accept the requests and sort it out later to retain safe harbor. That's just the way the law was set up. The whole point is that they claim neutrality - they are the carrier of the information, not screeners or judges of it. The legal back and forth is between the the entity posting the content and the entity submitting the DMCA takedown. After the fact, YouTube to Github can and should and sometimes do help fight back against abuse.
- est 9y ago> used DMCA takedowns to force removal from other blacklists Could a hashed tld blacklist help? Each person downloads a unique hashed tld blacklist. Browser would calculate tld against list of hashes (or bloomfilters for what's worth) > https://github.com/easylist/easylist/commit/a4d380ad1a3b33a0fab679a1a8c5a791321622b3 https://github.com/easylist/easylist/commit/a4d380ad1a3b33a0... In this case, what if a rule says - domain starts with "functio" - domain ends with "onalclam.com" - domain is no longer than 18 bytes. Instead of cleartext?
- AndyMcConachie 9y agoYou have an extra "o" in your string :)
- jstanley 9y agoI don't think so... "functionalclam.com" both begins with "functio" and ends with "onalclam.com"
- stordoff 9y agoAs the claim is regarding circumventing access control, I doubt the specific technical means by which it is blocked is relevant - Admiral's interpretation of the DMCA considers the outcome (the domain is blocked), not that the list includes a domain which they own.
- est 9y agoHow about split the blacklist into multiple entities (like Shamir's Secret Sharing) with absolute no affiliation with each other. Each one downloaded can not block anything But if a user combines some of the data, certain website gets blocked.
- zb3 9y agoThis time I really don't understand why you've been downvoted. Now it feels like some powerful users know something I don't, but they won't tell
- acdjuiamadfn 9y agoCan we go one step further and block websites which serve contents from these domains? This would be good first step towards eliminating toxic advertisements. We'll give up a bit first but may win eventually.
- bitshiffed 9y agoI've put together a list of sites using Admiral's services (the domains this extension blocks) here: https://gist.github.com/daumiller/114989e6967eb0d4c54b9ab9ffe63bb5 https://gist.github.com/daumiller/114989e6967eb0d4c54b9ab9ff... . I do not know how complete it may be. It was interesting putting the list together to see how most (but not all) of the users appear just as sleezy as Admiral itself. (Blocking these sites automatically should be done based on requests pointing to the original list, rather than this derived one, but it's here for reference.)
- just_cool 9y agoThat's a great technical achievement! How did you find all the sites containing Adimiral's library without crawling the whole web?
- 2ion 9y agoHosting such a project on Github, a US-based company which responds to DMCA requests, is perhaps not the most sensible choice.
- shmageggy 9y agoI think it's intentional, because the whole idea is that these kinds of requests are invalid and an abuse of the DMCA, so if someone were to issue a request the author would fight it to prove its invalidity.
- paulgb 9y agoExactly :)
- aaron_m04 9y agoAre there any good non-US Git hosting companies?
- gruez 9y agoSelf hosted gitlab?
- gsich 9y agogitea
- pishpash 9y agoGit doesn't need hosting... It's distributed for a reason.
- ue_ 9y agoI assume the reason why GP is asking is beacuse they want to be able to distribute a repository, i.e have people know where it is. Without a git host then you need to run a http/ssh server from your computer, which isn't an option for many people. And then you'd still need a website to list a link to it, if you want people to find it at all. And that website is just as liable to receive a DMCA for listing the allegedly infringing content, just like torrent sites are.
- loeg 9y agoWhy not just distribute a list, like easylist, that can be added to existing extensions like uBlock Origin?
- discreditable 9y agoThis seems to be a list of other admiral-owned domains: https://pgl.yoyo.org/adservers/admiral-domains.txt https://pgl.yoyo.org/adservers/admiral-domains.txt
- tofof 9y agoAs I note in the Github for this project, that list is woefully incomplete. See my analysis at the Anti-Adblock Killer repo for more detail - https://github.com/reek/anti-adblock-killer/pull/2502#issuecomment-277863986 https://github.com/reek/anti-adblock-killer/pull/2502#issuec... Admiral has thousands of domains across Google Cloud and AWS hosts.
- ksrm 9y agoPresumably they don't all point to unique IP addresses, so would blocking their IPs be more effective?
- oelmekki 9y agoWouldn't put those domains in a blockchain be the proper answer? It could not be "took down", then (if it's a solid one, like btc or eth).
- michaelmrose 9y agoPlease go and read the actual article.
- oelmekki 9y agoWhat article? The readme of the of the github repos? I read it. If you have a point to make, please make it.
- drraid0 9y agoThat's a stupid idea. What if you ever wanted to start a revocation list, when a domain is added genuinely by mistake? You'd have to use the same priv key for signing those as well, at which point you might as well have a single text file with provenance (aka git)
- oelmekki 9y agoWell, maybe that if it sounds stupid to you, you didn't think about it enough (that's usually the case). Make a address which creates transactions containing the domains blocked. To find the list, check the transactions from this address. If an error is made, then you can just use an other address to build the list again without the domains you want to remove, and make adblocks look for transactions from this address. The adblockers only reference an address so they have no illegal content. Domains can't be removed from transactions. Adblockers can use an other address as basis if they agree too.
- drraid0 9y agoThen go ahead and build it. Maybe it'll get adopted like name coin by dozens of people. Actually, name coin was actually a decent idea because dns needs to be decentralized. Text files like this do not, at least no more than git already is.
- JackC 9y agoAs a legal fine point: the list I want isn't just "sites which have used DMCA takedowns to force removal from other blacklists," but more like "sites which allege that they are legally required to be loaded if embedded in other sites." This would include all Admiral-owned domains (including those that haven't been included in DMCA takedowns yet), and all domains owned by any other companies that believe there is some legal obligation to load their trackers. It's an important list to have. Echoing other comments, this list should be in a standard .txt form so it can be included by other extensions, so I can pick an extension that does what I want when it encounters such a site (e.g., decline to visit the page that embeds the site).
- Retr0spectrum 9y ago"sites which allege that they are legally required to be loaded if embedded in other sites." In that case, maybe the best technical and legal solution is to block any sites that contain their domains completely. I.e. boycott anyone who does business with these people, without "violating" someone's interpretation of DMCA.
- deleted 9y ago[deleted]
- dweekly 9y agoFYI there appear to be a much longer set of Domains owned by this company, all of the same format of nonsensical word pairings. https://www.google.com/amp/s/amp.reddit.com/r/uBlockOrigin/comments/6swfvi/ad_domain_taken_down_from_easylist_due_to_dmca/ https://www.google.com/amp/s/amp.reddit.com/r/uBlockOrigin/c...
- jwilk 9y agoDirect link: https://www.reddit.com/r/uBlockOrigin/comments/6swfvi/ad_domain_taken_down_from_easylist_due_to_dmca/ https://www.reddit.com/r/uBlockOrigin/comments/6swfvi/ad_dom...
- hyperdunc 9y agoIt seems to me that domain names jewelcheese.com and futuristicfairies.com could be put to much better use than they are by this company.
- k__ 9y agodoesn't stuff like privacy badger generate these lists on the fly locally?
- preinheimer 9y agoI'm a privacy badger user and I just checked my list, I don't see this domain listed. Privacy badger will only block a domain if it appears that it's tracking you. Sites linking to this JS, that don't try to send tracking info will not be blocked.
- CodeWriter23 9y agoI take it as a good sign a player in the online ad industry is starting to squirm.
- snakeanus 9y agoI wonder, why is it an extension for a specific browser instead of a blacklist that can be used in any browser that has a blocker?
- jrwr 9y agoWhy not just switch to something like md5 for matching some domains that do this
- Grollicus 9y agoThe argument is not "they are spreading our trademark" but "they are circumventing our copy protection" and that would happen regardless of the way it is technologically done. This is a huge attack on the way we all consume the web, because in the end it leads to a society where we are forced to consume ads to participate in the society.
- appleflaxen 9y agothe original DCMA take-down was against a uBlock Origin list. why make an entirely new plugin when you can simply make a new list? As cool as your idea is, I don't need two browser extensions to manage when the first one will happily incorporate your list.
- paulgb 9y agoTo be honest? Because I was on a plane without internet, and I didn't know how to create a blacklist but I had another extension I'd created to use as reference :) I'm not opposed to doing both, once I have an hour to sit down and figure out what is actually involved. There's an issue to track it here: https://github.com/paulgb/BarbBlock/issues/5 https://github.com/paulgb/BarbBlock/issues/5
- appleflaxen 9y agocongratulations on using your flight so productively :) Thanks for filing the issue! I considered it, but didn't want to be pushy.
- paulgb 9y agoSomeone has now contributed a list: https://raw.githubusercontent.com/paulgb/BarbBlock/master/BarbBlock.txt https://raw.githubusercontent.com/paulgb/BarbBlock/master/Ba...
- smegel 9y agoThank you good Sir.
- 0x10101 9y agoNext step is to just build a script that parses the DMCA takedown notices here[1] and automatically builds a block list out of those domains. https://github.com/github/dmca https://github.com/github/dmca
- chii 9y agoyou'd have to differentiation between a legitimate DMCA takedown vs a false claim...may be some NLP on the work being taken down to see if it falls under fair use?
- kerkeslager 9y agoI don't think this needs to be that complicated. If I'm understanding correctly, the DMCA takedowns in this case are against pages blocked by EasyList. You would just have to correlate DMCA takedowns with historical EasyList filters.
- atomical 9y agoWhat is functionalclam?
- lost-theory 9y agoIt looks like an anti-adblock measure: https://github.com/reek/anti-adblock-killer/pull/2502 https://github.com/reek/anti-adblock-killer/pull/2502 > after the page finished loading. It removed the real page content and replaced it with a box asking me to whitelist the site. The reason why they're using so many domains is to circumvent blacklisting, and then the DMCA takedowns are another layer of circumvention on top of that.
- ajarmst 9y agoThis seems to beg an important question: are people who use legal means to remedy their inclusion on a blocklist necessarily doing so for nefarious reasons? Not everyone who's used a cease-and-desist or the DMCA process is a bad actor. This extension isn't necessarily bad---if its purpose is simply to ensure that DMCA takedowns and cease-and-desist orders are properly supported and enforced only with good cause, then that seems valuable. If it ends up as a tool that starts making people who are legitimately trying to protect their livelihood or interests give up by making their legal remedies unenforceable or too onerous to undertake, then maybe we need something a little less cavalier.
- mholt 9y agoYou would only use a DMCA takedown on a blacklist if you want to force your content to be viewed by others. That is bad acting. DMCA is to protect the rights of content owners from their content being spread maliciously, not for it to be blocked from viewing in an individual's private home or device.
- ajarmst 9y agoFair enough. But I was just accepting the OP's conflation of DMCA takedowns and cease-and-desist orders. The concern remains valid in the latter case (I'm not sure the DMCA could even legitimately be used in this rather odd, inverted way, and agree that it's hard to imagine a valid use). My point is that nothing I do to a blacklist could "force" anyone to view my content---but I might have a legitimate interest in correcting a blacklist's mischaracterization or miscategorization of my content.
- tonybaker 9y agoGET RESULTS IN ONE HOUR TIME FOR ALL KIND OF EMAIL HACK.Change School Grades? Hack Banks? Erase Criminal Records? Hack Websites? Hack Database? Hack Drivers license? Hack Call Log? Hack Visichat and Flashchat Rooms? Hack FTP User and Pass? Hack Facebook,Whatssap,Twitter,Instagram,Webcam etc.Hack VB Forum? Hack Wordpress Blog? Hack CC any Country? Hack Money Booker Account? Hack Liberty Reverse Account? Hack Paypal Account? Root Server? By Pass Google Phone Verification? Install Red on Linux Server? Hash Crack? DDOS Server? Retrieval of Lost Files and Documents CONTACT: empiricalhackers4187@gmail.com
- tonybaker 9y agoGET RESULTS IN ONE HOUR TIME FOR ALL KIND OF EMAIL HACK.Change School Grades? Hack Banks? Erase Criminal Records? Hack Websites? Hack Database? Hack Drivers license? Hack Call Log? Hack Visichat and Flashchat Rooms? Hack FTP User and Pass? Hack Facebook,Whatssap,Twitter,Instagram,Webcam etc.Hack VB Forum? Hack Wordpress Blog? Hack CC any Country? Hack Money Booker Account? Hack Liberty Reverse Account? Hack Paypal Account? Root Server? By Pass Google Phone Verification? Install Red on Linux Server? Hash Crack? DDOS Server? Retrieval of Lost Files and Documents CONTACT: empiricalhackers4187@gmail.com
- tlavoie 9y agoBesides web browser extensions, this sort of thing works nicely in firewalls that support DNSBL (DNS block lists). I've got pfSense running pfBlockerNG, and it whacks both incoming (known malicious) and outbound requests. As a result, anyone on my home network gets the same ad-blocking and other protection, including guests on the Wi-Fi. Nobody has to do anything special, it Just Works.
- ezoe 9y agoThey tried to use DMCA to censor non-copyright-able information such as this? Ridiculous.