27 ms·

EasyList is in trouble and so are many ad blockers

• randoglando 4y ago
  Would DNS blocking be affected by this (I presume not since the lists are hardcoded)?
• disadvantage 4y ago
  > There’s an open source Android browser (now seemingly abandoned) that implements ad-blocking functionality > The problem is that this browser has a very serious flaw. It tries to download filters updates on every startup, and on Android it may happen lots of times per day. It can even happen when the browser is running in the background EasyList should be offered as a version-controlled copy you grab once, that then gets bundled with an app, rather than offering a download to be called from an app: https://easylist.to/easylist/easylist.txt https://easylist.to/easylist/easylist.txt (Currently down as of writing). The only caveat is such a list needs to be updated, so then a version system should be implemented for EasyList and you periodically bundle the new version via app updates. It would save a lot of bandwidth doing this.
  • stillbourne 4y ago
    You can use firefox on android and install ad block on it as a plugin
    • ElectricalUnion 4y ago
      How does that help when you can't download easylist anymore since it's under DDoS?
  • muststopmyths 4y ago
    Call me lazy but I’d just support an If-modified-since header in such a simple case and call it good
    • comboy 4y ago
      Bad app will just brutally fetch it every time with not even a cache on its side. As a quick fix, there are many options for limiting per IP per timespan, e.g. fail2ban, you could configure it to punish bad apps without crippling functionality for others. Well, maybe crippling a little bit in some very special use cases, still better than it simply not working.
    • kevincox 4y ago
      The list does support an ETag and the 304 even appears to not be rate-limited by Cloudflare.
  • Maxburn 4y ago
    PiHole and PFBlockerng are two big ones that use these resources too and setting those up it struck me as it did you that simply polling these resources on a set schedule was a waste. Podcasting 2.0 has been talking about podping as a solution because podcasting basically has the same problem with periodic polling of the RSS feed. Basically you subscribe and then receive notice there's been an update, THEN you go get it. https://www.podcasthelpdesk.com/podping-and-other-stuff-with-dave-jones-from-podcast-index-and-podcasting-2-0-phd165/ https://www.podcasthelpdesk.com/podping-and-other-stuff-with...
    • RobotToaster 4y ago
      The difference is pihole only updates weekly.
      • Maxburn 4y ago
        Didn't know that, in fact clicking around in the UI I don't see a way to change that so good on them for being friendly in this area. PFBlocker seems to default to once a day.
        • tssva 4y ago
          To change when pihole updates you edit the cron entry at /etc/cron.d/pihole.
          • Maxburn 4y ago
            That's what I meant, it's buried in a config file. Not one of those things that should be surfaced in the UI that opens it up to abuse.
  • pwinnski 4y ago
    They're offering a text file. Presumably with an If-modified-since header, although it's hard to check now. There is no approach you can describe that doesn't run afoul of the described badly-behaved browser app which willfully retrieves the entire file afresh at every init. If it can be downloaded, it will be downloaded directly by the badly-behaved mobile apps.
• tejtm 4y ago
  hack-n-patch worm? I recall a white hat doing this for some IoT annoyance.
• bombcar 4y ago
  If I recall correctly there was some image on wikipedia that was getting billions of downloads a day or something, all from India, because some smart phone had made it a default "hello" image and hot linked it. Unfortunately, I can't find a reference to it anymore.
  • alphabet9000 4y ago
    https://www.vice.com/en/article/qjpmyx/why-is-this-flower-on-wikipedia-suddenly-getting-90-million-hits-per-day https://www.vice.com/en/article/qjpmyx/why-is-this-flower-on...
  • robin_reala 4y ago
    Not that you’d do it, but the temptation there is always to repoint your real application to a different URL and change the original image to something subtly NSFW.
    • o_m 4y ago
      Or something less malicious, like "Donate to Wikipedia", or some other organization.
      • Someone 4y ago
        > Or something less malicious, like "Donate to Wikipedia", or some other organization. https://en.wikipedia.org/wiki/Censorship_of_Wikipedia https://en.wikipedia.org/wiki/Censorship_of_Wikipedia: “Wikipedia has been blocked in China since 23 April 2019” ⇒ putting ads for Wikipedia on sites likely isn’t safe everywhere. I think it will be very hard to find “some other organization” that is universally ‘approved’ everywhere.
    • Raed667 4y ago
      I was debugging a similar issue where a small marketplace run by a friend was being scrapped and the listings were being used to make a competing marketplace look more active than it actually was. The thing is, they didn't host the scrapped images themselves, they just hot-linked everything. So through a little nginx config, we turned their entire homepage to an ad for my friend's platform :)
• wnevets 4y ago
  A lazy/bad developer ruining something so many people depend on is incredibly annoying.
  • shadowgovt 4y ago
    On the internet, popularity is sometimes indistinguishable from being targeted by a low-orbit ion cannon.
• Eisenstein 4y ago
  I am using a pi-hole which uses easylist, but it grabs it from https://v.firebog.net/hosts/Easylist.txt https://v.firebog.net/hosts/Easylist.txt which seems to be working fine.
  • celsoazevedo 4y ago
    That's a different list. It only contains the hosts/domains on easylist, because that's all pi-hole (and all host based blockers) can block. It's also hosted by someone else (and they too use Cloudflare, see firebog.net). The normal easylist is way bigger and has lots of rules for ad blockers like uBlock Origin.
• Tepix 4y ago
  This sucks. I hope they can reach the app authors and get it fixed the source. Meanwhile perhaps some other CDN provider wants to create some goodwill if Cloudflare isn't willing?
  • avani 4y ago
    I am not speaking on behalf of the company, but if someone involved with EasyList can contact me (avani@cloudflare.com), I'll see if there is a way to help out.
    • ameshkov 4y ago
      Thank you! I've passed that to EasyList maintainers.
• gst 4y ago
  > EasyList is hosted on Github and proxied with CloudFlare. What is the reason for proxying through Cloudflare? Are there any bandwidth limits or performance issues when directly serving those files from GitHub?
  • rapind 4y ago
    From https://docs.github.com/en/pages/getting-started-with-github-pages/about-github-pages#usage-limits https://docs.github.com/en/pages/getting-started-with-github... --- > GitHub Pages sites have a soft bandwidth limit of 100 GB per month. > In order to provide consistent quality of service for all GitHub Pages sites, rate limits may apply. These rate limits are not intended to interfere with legitimate uses of GitHub Pages. If your request triggers rate limiting, you will receive an appropriate response with an HTTP status code of 429, along with an informative HTML body. > If your site exceeds these usage quotas, we may not be able to serve your site, or you may receive a polite email from GitHub Support suggesting strategies for reducing your site's impact on our servers, including putting a third-party content distribution network (CDN) in front of your site, making use of other GitHub features such as releases, or moving to a different hosting service that might better fit your needs. ---
    • Macha 4y ago
      I wonder how e.g. NixOS or Homebrew deal with this
      • YoshiRulz 4y ago
        NixOS wouldn't be affected by that quota because it doesn't use GitHub Pages as far as I can tell.
  • tomudding 4y ago
    GitHub has a soft limit of like 100 GB/month on transfers for Pages. According to the Adguard blog post traffic was already several TBs a day before the issue arose.
• metadat 4y ago
  100TB of "Access Denied" replies per month, how many requests served is this? This is hilarious in an unbelievably terrible and tragic way. The scale is mind boggling. I wonder which browser it is.
  • ameshkov 4y ago
    Author here. Actually, it's getting better, I've just looked up the stats and for the last 30 days we only served 70TB of access denied pages, this is about 33-34B requests.
    • matthewaveryusa 4y ago
      That's 2 trillion requests a month, or 780,000 requests/second. That would be just a little shy of 1% of all of akamai's rps traffic. mind-boggling
      • ameshkov 4y ago
        Oh no, that were numbers for a month, not a day
  • deleted 4y ago
    [deleted]
• MuffinFlavored 4y ago
  > Even so, we continue to serve about 100TB of “Access Denied” pages monthly! What's the carbon footprint of this?
  • ignoramous 4y ago
    The thing with Access Denied is that these deprived clients retry with some vengeance. So, you're instead draining more resources than you'd like. I run a content-blocking DoH resolver, and this happened to us when we blocked IPs in a particular range and the result was... well... a lot of bandwidth for nothing.
    • ameshkov 4y ago
      In this particular case this is not the case and they don't retry. They just really want to download updates REALLY often.
    • sneak 4y ago
      Why serve any HTTP replies to those at all? If you are doing it at the IP level, why not just drop all inbound packets from the L3 address?
      • ignoramous 4y ago
        We were on Netifly way back then. So, no L3 blocks. Now on pages.dev and workers.dev, but haven't needed to enforce any rules yet.
      • DigitallyFidget 4y ago
        This is what I was wondering. I'm taking a wild guess that maybe they don't have that level of firewall access and it was being done through filtering by the webserver to provide an access denied.
    • DigitallyFidget 4y ago
      But why bother with deny? Just send a blank text file (or one with as minimal data as needed to satisfy the rogue adblock) to the "blocked IPs" to mitigate the traffic for now. If firewall access exists, just drop the offending incoming traffic entirely.
• tomschwiha 4y ago
  I'm confused about the ToS comment by Cloudflare. The txt is on a website so it is a web content? So robots.txt is not supported by Cloudflare to cache/proxy it? That would be a weird regulation. And I bet everyone violates the Cloudflare ToS then.
  • andiareso 4y ago
    Yeah... That just doesn't seem right. All web content is text...
    • naikrovek 4y ago
      text/html is not text/plain but that doesn't matter: it's not a technical limitation that caused cloudflare to draw this line. it's cloudflare deciding to protect "web content" and not videos or .iso images or other things that normally are not commonly served while you browse a contemporary website and read HTML.
      • thaumasiotes 4y ago
        > it's cloudflare deciding to protect "web content" and not videos or .iso images or other things that normally are not commonly served while you browse a contemporary website and read HTML. That's false in two ways: first, text is normally served while you browse a contemporary website; second, so are images, which are explicitly called out as potentially violating this clause. Text is the only data that isn't covered by this clause.
        • naikrovek 4y ago
          • deleted 4y ago
            [deleted]
    • jonny_eh 4y ago
• hnaccy 4y ago
  can they not block Indian IPs from cloudflare dashboard
  • fdfaz 4y ago
    I think the free plan allows this. Seems an easy solution.
    • bluehatbrit 4y ago
      I'll have a huge impact on anything making legitimate use of the list. Adblockers on sensible browsers will stop working etc. It may be easy, and it may even be the only option, but it's a bad one that will need some thought from the maintainers I expect.
      • deleted 4y ago
        [deleted]
      • zedadex 4y ago
        The only thought it would require would be thought they should have been putting into their fetching strategy in the first place.
  • Raed667 4y ago
    That would prevent 1.4 billion people from ad-blocking. Not sure if we want to use these kind of blanket measures as a first response.
    • celsoazevedo 4y ago
      The logic here is that it's better to block 1 country and keep it working for everyone else than to leave it as it is and break it for everyone. It's not ideal, but until the problem is fixed/better solutions are found, I think it's a good "first response".
    • sneak 4y ago
      No, it would block 1.4 billion people from that one specific URL.
• rc_mob 4y ago
  Phew. Is just a bandwidth issue. This goofy title made me think advertisers found a way around ad blockers.
  • ignoramous 4y ago
    It is a bandwidth issue for a volunteer-run project.
  • laundermaf 4y ago
    This issue caused CF to irreversibly ban them though, so it's not "just a bandwidth issue" anymore. > Based on the URL that are being requested at Cloudflare, it violates our ToS as well. All the requests are txt file extension which isn't a web content > you cannot use Cloudflare to cache or proxy the request to these text files
    • deleted 4y ago
      [deleted]
    • nicce 4y ago
      Well, CF is just one service provider. There are bigger issues if they have already such a monopoly that their decisions kill projects worlwide.
    • jsty 4y ago
      > This issue caused CF to irreversibly ban them though Do you have a source for that? The article only mentions them being throttled + the screenshot with the support engineer saying they seem to be breaking the ToS and asking them politely to move back into compliance.
    • ziml77 4y ago
      Where did you get that they were irreversibly banned? Or banned at all for that matter?
      • codalan 4y ago
        They didn't get banned. They got an email from CF support saying that they cannot cache TXT files and that they'd need to disable the proxy. This does not mean banned.
• nnopepe 4y ago
  serve a modified version to rate limited IP's that only contains popular indian sites and I'm sure it'll be resolved in a day or two
  • bionade24 4y ago
    Limit this to the specific headers of these Webbrowsers though, please.
    • stereo 4y ago
      They already have that part figured out. From the article: > When we encountered a similar problem last year, we found a simple solution: block the undesired traffic from these apps. Even so, we continue to serve about 100TB of “Access Denied” pages monthly!
      • bombcar 4y ago
        100TB of Access Denied is only 38 MB/s, so not even a minor DDOS these days.
      • d12bb 4y ago
        The difference is that serving Access Denied Leads to the users of these malicious browsers just getting more ads over time, as the filter lists can’t be updated anymore. Serving a special list containing popular sites would result in the users almost instantly not being able anymore to access these popular sites, resulting in requests to the developers to fix their shitty browser or switching altogether.
        • deleted 4y ago
          [deleted]
      • georgebarnett 4y ago
        Blocking the browsers isn’t a solution because they likely fall back to being open, so the user doesn’t notice. Instead, you need to break the user experience so they complain to the developer of the app, thus impacting reputation. It’s unfortunate that the browsers developers are unresponsive and this circumstance limits the available options to easy list.
• Reallyneed 4y ago
  Open and public utility data could be served with p2p technologies.
• tredre3 4y ago
  I'm sympathetic to their trouble but we're talking about serving a 330KB text file (150KB compressed), surely this isn't an insurmountable technical hurdle to overcome? A 1000mbps dedicated server could serve it 70 MILLIONS times per day. Considering that most wouldn't be served (E-tags and whatnot), it can probably sustain a billion requests a day. What am I missing?
  • ameshkov 4y ago
    You're missing three orders of magnitude.
  • ajsfoux234 4y ago
    > The overall traffic quickly snowballed from a couple of terabytes per day to 10-20 times that amount. A 1000Mbps server could only serve 10.8TB, and that's not even accounting for overhead/daily usage patterns/etc
  • edf13 4y ago
    It’s not really the serving that’s the issue - it’s the amount of bandwidth used… in the case of serving simple content (like txt) bandwidth is always going to be the expensive element
  • celsoazevedo 4y ago
    The last copy of the .txt file saved by the WayBackMachine is 1.4MB: https://web.archive.org/web/20220901000327if_/https://easylist.to/easylist/easylist.txt https://web.archive.org/web/20220901000327if_/https://easyli... If you're getting a 330KB file, maybe the server issues are causing the download to fail?
  • deleted 4y ago
    [deleted]
• CityCobra 4y ago
• JoyfulPanda 4y ago
  I'm not sure, but is IPFS capable of solving the issue?
  • fabianhjr 4y ago
    The gateways would similarly not be thrilled about it. I would add a redirect to the makers of the browsers in question (so that the leechers got to deal with the traffic themselves; https://en.wikipedia.org/wiki/Inline_linking https://en.wikipedia.org/wiki/Inline_linking)
  • sneak 4y ago
    No, the issue is with a misbehaving client program accessing a specific HTTP URL.
  • Reallyneed 4y ago
    Yes. But client must be native IPFS, not relying on gateway...
• deleted 4y ago
  [deleted]
• drexlspivey 4y ago
  just add the easylist.to domain to easylist!
• noitpmeder 4y ago
  It seems they need some form of UUID in order to rate-limit individual clients. I wonder what percentage of their traffic would drop if they started requiring some form of authentication to download this list?
• Traubenfuchs 4y ago
  I would just put it on a public git repo and let it sort itself out.
• pseudosavant 4y ago
  It would seem like you could prevent hotlinking by adding 1-5 minutes of latency to every request to a list. Almost no dev would hotlink an asset that took that much longer to display, at least in critical/common paths. It would force consumers (devs/businesses) of the lists to provide a caching/mirroring solution of some kind for their users. But on the bankend, the request would be designed just for updating the list cache. Handling 1-5 extra minutes per request, on a request that runs less than a few dozen times a day, to update the mirror/cache is trivial.
  • cogman10 4y ago
    The issue with this approach is it's too late. It might work if you designed it from the start, but adding it now would only destroy your poor balancer with all the connections they have to maintain (waiting for the 5 minutes to expire). It was mentioned in this article that they are now serving up accessed denied, but the problem is one of just too many requests. At this point, it's likely easier to just kill the domain all together and get a new one.
    • pseudosavant 4y ago
      This is certainly not a cure to the problem Easylist has right now. This is prevention. About how to design publicly consumable resources to naturally discourage hotlinking, before it is a problem.
      • xeromal 4y ago
        That's what the person you're replying to said.
• fweimer 4y ago
  pool.ntp.org hands out specific subdomains for large-scale pool users. This way, it is possible to retire service for a subset of users that use devices that aren't updated anymore and are misbehaving. The traffic issue is not just punted to DNS service. It's possible to return a cachable 127.0.0.1 response, and it's somewhat rare for DNS caches to be constantly powered up and down and reach out directly to authoritative DNS servers.
• Raed667 4y ago
  I worked on an ad-blocker a few months ago. I made the decision to have the filter-list files hosted on our own domain and CDN (similar to what Adguard does with their filters.adtidy.org). This was done for 2 reasons: 1- Avoid scenarios like this where you ship code (extension in this case) that is hard to update. Then make that code depend on external resources outside of your control. 2- Leak our users' IP addresses to each random hosting provider. So the solution was simple: Run a CRON once a day then host the files ourselves. Pretty happy with that decision now.
  • chrismeller 4y ago
    Except neither of those would help in this case. They’re already using their own domain name, and it’s unclear how they would even build their own CDN since they’re using that scale of bandwidth - AdGuard said they’re still pushing 100tb of access denied pages a month for their similar case. That is a LOT of bandwidth just for access denied messages.
    • lolinder 4y ago
      Their point isn't that EasyList could have done anything differently, their point is that they are glad that they didn't decide to rely on others' infrastructure for their own ad blocker, because that makes them resilient against the fallout from this and similar.
      • chrismeller 4y ago
        Except it wouldn’t make them resilient since, as I pointed out, neither of the things they did would be of any help at all to Easylist in this situation. It’s great that they’re happy with their choices, but the choices would, in this same situation, likely saddle them with a crippled infrastructure and/or some insane bandwidth bills for suddenly pushing 100 extra TB/m.
        • girvo 4y ago
          It was never suggested that what the original commenter was doing would help EasyList.
• deleted 4y ago
  [deleted]
• eis 4y ago
  Regarding the 100TB of Access Denied pages: just drop the connection instead. To make the system more scalable: instead of directly serving the file, serve a bunch of URLs to mirrors plus a checksum. The client must pick one of them. You can randomize the URLs and maybe add some geo logic to it. Let people provide mirrors. An additional indiraction step like this can prove incredibly powerful for systems that need to scale massively.
  • deleted 4y ago
    [deleted]
  • pebcakID10T 4y ago
    My first thought as well. Why even provide a response?
• JimWestergren 4y ago
  A great opportunity right now for CloudFlare to win some goodwill and PR by helping out EasyList for free right now. But what about simply enable a firewall and show captcha or similar if the origin IP is from India and requesting that URL until the situation is under control? I did that with the free plan recently in CloudFlare in a similar situation and it worked perfectly (of course on a much smaller scale).
  • metalliqaz 4y ago
    that would break everyone in India not using one of those broken browsers
    • iforgotpassword 4y ago
      They are already serving access denied replies, so I assume they can identify the browsers via user agent or similar? If so, returning a bogus file that blocks everything and adding a comment in that list asking the developers to use caching or mirroring the file should be fine. I wonder if those browsers honor the list when fetching the update though. Would be awesome if you could just add easylist and lock out further requests right on the device.
      • democra 4y ago
        Browser developers can choose to fake user-agents. Brave uses a generic chrome user agent so it cannot be differentiated from regular Chrome.
    • acdha 4y ago
      Everyone in the world is impacted if the site goes down under load. Changing that to everyone in a particular country (perhaps with a given user agent if the free plan allows expressions) would still be an improvement even if other work is needed.
  • bluehatbrit 4y ago
    Most requests will be in the background or in Cron jobs. Captcha wouldn't be possible in those situations as it would never be seen by anyone.
• Joel_Mckay 4y ago
  Rate-limit the GeoIP list for the affected areas to drop if more than 20% of active traffic. i.e. the service outages get co-located only with the problem users areas. Also, when doing auto-updates: always add a chaotic delay offset 1 to 180 minutes to distribute the traffic loads. Even in an office with 16 hosts or more this is recommended practice to prevent cheap routers hitting limits. Another interesting trend, is magnet/torrent being used for cryptographic-signed commercial package file distribution. Free API keys are sometimes a necessary evil... as sometimes service abuse is not accidental.
  • codalan 4y ago
    That would only work if they had an API; AFAICT, they're just hosting a file. At this point, they might be better off coordinating with the other major adblocker providers and just outright move the file elsewhere. Breaking other people's garbage code is better than breaking yourself trying to fix it. Especially on a budget of $0.00. If the defective code for the browsers are in public repos, it might also be more effective for someone to just fork the code, fix the issue (i.e. only download this file once a month, instead of every startup), and at least give the maintainers a chance to merge the fix back in.
    • Joel_Mckay 4y ago
      It is very common to see API keys in urls for access to what are essentially flat files. Thus, fairly trivial to change from: https://127.0.0.1/file.csv https://127.0.0.1/file.csv to https://127.0.0.1/file.csv?apikey=abc123 https://127.0.0.1/file.csv?apikey=abc123 This could allow client specific quotas, and easy adoption with maintained projects in minutes. Thus, defective and out-of-maintenance projects would need manually updated or get a 404. =)
      • codalan 4y ago
        Ahhh, good point!
      • Programmatic 4y ago
        API keys are most successful when they're issued for server-side use; when used client-side the usual pattern that I see is for individual clients to request their own API key? In this case, it would need to be distributed to myriad users who legitimately need to ask for the lists and then could be scraped by the "attacker", but at least then they'd have to be knowingly malicious vs. accidentally malicious.
• bluehatbrit 4y ago
  I wonder what would happen if they renamed the file to robots.txt and then did a redirect at the cloudflare level for the current URL to robots.txt. I imagine some (many?) clients would handle it poorly, but it would then be cachable at least. It's not exactly easy to test though without a level of unknown damage to legitimate users.
• bugmen0t 4y ago
  Can’t they run an open collective just to pay the bandwidth bill?
  • codalan 4y ago
    They shouldn't have to foot the bill for this. This is some bad developers releasing a bad fork of a bad browser. At a minimum, they should have either reduced the requests to something like a monthly download (not great, but far better than requesting a file every startup), or ideally, hosted and updated the file themselves, on their infrastructure. At least that would force them to look at their own hosting bills, instead of crippling a prominent and important contributor to ad-blocking software.
• kccqzy 4y ago
  I can't imagine slowing down could be a good idea. At their scale, the sheer number of connection count probably matters more and may contribute to a higher proportion of cost. Bandwidth is expensive yes, but keeping a connection alive means consuming extra memory for the socket in the kernel, in the app, and in many other places.
• jijji 4y ago
  Why not just limit one request per week per IP by those suspect netblocks
• mmastrac 4y ago
  Scaling serving of a static text file would make a fantastic job interview question, especially as you explore what happens as the text file gets bigger and the number of downloads gets higher. The "correct answer" isn't necessarily obvious in any of these cases.
• PaywallBuster 4y ago
  Cloudflare claims that R2 have free egress/bandwidth You could try that instead of the "CDN" service -- Alternatively, try the "cheap" CDN services like Bunny or Beluga, which have packages for high volume like 0.005c/gb Cloudflare is not really selling a CDN, but all the "smart" services on top of it. That's why you don't have as much control (like blocking IP/Geos without Enterprise), or run into issues for breaking their ToS.
  • Jabbles 4y ago
    > like 0.005c/gb You're off by a factor of 100. https://www.belugacdn.com/cdn-pricing/ https://www.belugacdn.com/cdn-pricing/ $5000/PB = $5/TB = 500c/TB = 0.5c/GB
    • PaywallBuster 4y ago
      beluga > You pay 1¢ (or less!) for every Gigabyte of data accelerated over our cloud network. doesn't seem clear cut as they bundle it in a subscription packages, but at this volume you'd surely need their enterprise package (10Tb+), which you'd expect to get 0.01 or less bunny > First 500TB $0.005 /GB > From 1PB-2PB $0.002 /GB
      • mafuy 4y ago
        > Even so, we continue to serve about 100TB of “Access Denied” pages monthly! That is several thousand $ a year.
      • Jabbles 4y ago
        You wrote 0.005c not $0.005
        • PaywallBuster 4y ago
          my bad, not native speaker
• nfriedly 4y ago
  I have my pi-hole configured to use the Domains list from https://oisd.nl/ https://oisd.nl/ which incorporates a bunch of other lists (including easylist), de-duplicates, and removes a few false positives. They also have an Adblock Plus Filter List that can be used with uBlock Origin.
• publicarray 4y ago
  Maybe try the open source programs at Fastly.com or Bunny.net https://www.fastly.com/open-source https://www.fastly.com/open-source https://bunny.net/contact/ https://bunny.net/contact/
• neilv 4y ago
  Assuming it's not a kind of DoS attack, and since it sounds like they can detect the abusing clients (maybe by User-Agent)... some very desperate technical options involve serving an alternate small blocklist that does one of: 1. Try having it block subsequent requests for EasyList itself, just in case the frequent update requests are made with the prior blocklist in effect. (I accidentally did this before, in one of my own experimental blocklists, atop uBlock Origin.) Then the device vendor can fix their end. 2. If the blocklist language and client support it (I suspect they don't), you might safely replace or alter some Web pages, to add a message saying to disable EasyList in the client, or pressure the vendor, or similar. If this affects a lot of users, the meaning will also be spread in other languages to other users, even if not all of them understand any of the languages in the message. But be careful. 3. If you can't get a better message to the user, another option might be to block all requests, to prompt users to disable EasyList or vendor to fix the problem. But before doing this, you'll need to have verified that a combination of shoddy client/device software won't prevent users from using important functions of their devices for significant time. (Imagine this might be their only means of being connected online, and some shoddy client software pretty much prevents it from working, and the user is unable to access critical services.) But before doing any of these desperate technical measures... First, I'd really try to reach people in the country who'll know what's going on, and who can reach and possibly pressure the vendor who's causing the problem. If tech industry people aren't able to help quick enough, reaching out to that government, directly or through your own country's diplomats/officials, might work. Communicating the risks of the desperate technical measures that you're trying to avoid (e.g., possibly breaking critical communications) could help people understand the urgency and importance of the situation.
  • zedadex 4y ago
    > Try having it block subsequent requests for EasyList itself Now you're thinking with portals
• extantproject 4y ago
  https://archive.ph/PKyUT https://archive.ph/PKyUT
• tux 4y ago
  GitHub @ https://github.com/easylist/easylist https://github.com/easylist/easylist Alternative Lists @ https://filterlists.com/ https://filterlists.com/
• deleted 4y ago
  [deleted]
• russellbeattie 4y ago
  I used to host my blog on my own server years ago and some popular site "hotlinked" to an image I had posted. My traffic - which back then I paid for by the Gb - spiked like crazy. I decided to put in Apache referrer checks for images and started serving some porn when it didn't match my domain. That solved the problem pretty quickly. Easylist should do something similar - requests from India should include a list with all the popular sites like Google, YouTube, aajtak.in, etc. When their browsers suddenly stop working the problem will be solved.
• secondcoming 4y ago
  How much of that 100TB is for the stupid HTTP 'Date' header?
• therealmarv 4y ago
  Bittorrent, switch in long-term to that. Not saying every end-user should be a seeder but there is big bittorrent community out there and everyone could help a little bit. Other options: - A kind of mirror network (it only needs to keep sure that integrity can be checked, maybe with a public key) - And while doing that why not also support compression (why not? only devs need to read it and they can run easily a decompression command), every bit saved would help.
  • pmlnr 4y ago
    Oh, but yes every user should be a seeder. Why not?
    • swinglock 4y ago
      Wouldn't need all users to, and many are mobile or on metered connections.
    • matheusmoreira 4y ago
      Mobile connections are usually metered. I'd still seed it 24/7 on my home connection though.
  • ignoramous 4y ago
    > Bittorrent, switch in long-term to that. S3 buckets in IAD with <5GB blobs can double-up as bit-torrent seeders. I'd imagine, some tech IPFS/Filecoin/Sia might come in handy, too, but unsure of how healthy most of these web3 projects are right now. There's also fosstorrents.com that help seed projects.
    • srvmshr 4y ago
      Is IPFS strictly Web3? Sure it gained traction around Blockchain, crypto DeFI - but the storage technology is ELI5 a massive distributed storage. I could hazard a guess that in terms of philosophy its closer to BitTorrent than Blockchain
      • 4y ago
• louison11 4y ago
  1. Restricting access until developers fix it. 2. Consider encouraging the use of webtorrent within the extension? = each user hosts and serves the list.
  • the8472 4y ago
    Webtorrent isn't distributed, so you'd just shift the problem to the tracker/signalling server. And in this particular case even BitTorrent proper may not have helped because steady-state BT is distributed but if a client doesn't persist its state after a bootstrap - and lack of persistence is the issue here - it'd hit the bootstrap server every time. Granted, it'd only be about one UDP packet per client, much less traffic than what is easylist is seeing, but foolish code deployed at scale can still overload services provided on a budget.
• runlevel1 4y ago
  1. Add a ToS to the EasyList website that prohibits this sort of abuse. (I don't see any currently.) 2. Send a cease and desist letter to the app creator. 3. If they don't respond, also send a C&D to Google demanding they cease distribution of the malware responsible for the DDoS. Anyone can send a cease and desist -- it's just cautionary letter. You aren't obligated to follow through with the threatened legal action. It doesn't have the force of law behind it, but it'll at least get their attention. (IANAL)
  • Phelinofist 4y ago
    Would someone in India care about a C&D letter sent from someone, say, in Europe or the US? I don't think so
    • sinuhe69 4y ago
      It’s not just somebody. If it’s from the creator of the things your livelihood depends on, you better pay attention.
  • account42 4y ago
    A C&D not coming from either a law firm or at least a big company is likely to go straight to the trash.
• mensetmanusman 4y ago
  Perfect application for a functioning system like torrent.
  • arglebargle123 4y ago
    You'd still need the coordinator node (tracker) for every single request, so you'd still be serving an absolutely insane amount of traffic off one central box. If you baked something like IPFS or DHT torrent capability into the application and requested your blocklists that way you'd solve the single point of failure problem, but that's asking a whole lot from a shittily maintained and poorly configured browser fork.
• mensetmanusman 4y ago
  I’m surprised ad companies have not tried this as an ‘attack’ vector already.
• slt2021 4y ago
  just ban all Indian IPs from the website on firewall. proper solution would be to use DNS to forward all Indian traffic to one of the local VPCs. The idea is to rate-limit users - let them pull blacklist.txt only once a day, so you serve the file and add requestor's IP to denylist on the firewall. So that any subsequent requests are blocked by firewall +Cloudflare has Geofencing feature
• noasaservice 4y ago
  Really? "text files" arent web content? What the fuck does cloudflare think CSS and HTML files are?
  • cr3ative 4y ago
    To be fair to them, this is configuration data, not a piece of a website you would read in a browser. I don’t agree with the policy but it is reasonably clearly worded.
• sershe 4y ago
  Since they added "Access denied" for misbehaving browsers, can they instead serve them some sort of bad response that will "surface" issue to the users? Depending on what would work better and cost less... (1) a small list that would block major legitimate sites. Whoops, the browser is unusable, now users complain to the developer to fix the issue, or abandon it. (2) "hang" the request if the browser loads the list synchronously; blocking UI thread is a hallmark of a bad developer, so they might (3) stream /dev/zero. Might be expensive; maybe serve a compressed zip-bomb if HTTP spec allows and/or browsers will process it?
  • porbelm 4y ago
    Too much work. Just blackhole all requests originating from India in the firewall, as a start.
    • d3nj4l 4y ago
      Right, block roughly 10% of the world. Great idea!
• ff7c11 4y ago
  Easylist should serve the Indian browser (based on user-agent) with a giant file (expensive), a corrupt file, or some response which causes the app the crash. If the browser crashes on every startup due to a malicious response from the Easylist server, users will likely delete it.
  • paxys 4y ago
    Serving a giant file is going to affect their servers more than the end device. If they could identify the user agent it would be a lot easier to just block it entirely.
    • zamalek 4y ago
      Could be done with a gzip bomb.
      • jefftk 4y ago
        Or a brotli bomb, for far greater expansion!
        • wink 4y ago
          And a terrible idea because the users are not at fault, but the developers of the app. And in this case it might crash so they'd only report "app crashes on startup", if at all.
  • angch 4y ago
    Just a separate rules file that blocks .in and other popular domains for India will probably work just as well.
  • aembleton 4y ago
    Then the browser could just pretend it is Chrome (if it isn't already) and that would easily work around your solution. I think something like this would be best served by moving to IPFS or Bittorrent. A magnet link could be provided and then browsers and plugins could use that to download the file. That way, you can distribute the load.
• yellowapple 4y ago
  Cloudflare's rationale for a ToS violation is absolutely bonkers. Who cares what the file extension is? By virtue of being accessible on the web via a URL it is by definition web content.
  • r1ch 4y ago
    Cloudflare's TOS only permit caching for HTML content and related assets. It's probably a useful catch-all they can pull out any time someone uses too much bandwidth.
    • yellowapple 4y ago
      > It's probably a useful catch-all they can pull out any time someone uses too much bandwidth. That's exactly what it is, and Cloudflare should be honest about that instead of coming up with an excuse as pathetic as ".txt files ain't web content".
    • celestialcheese 4y ago
      Ok, so then why not make a structured HTML page that's served and parsed as the list.. that would be perfectly fine? Cloudflare's stance on this makes no sense
      • kenmacd 4y ago
        That would be fine because then CF could replace that HTML with their "Checking your browser before accessing" content. What's the app going to think of that though?
        • yellowapple 4y ago
          > That would be fine because then CF could replace that HTML with their "Checking your browser before accessing" content. They could do that anyway by putting up the "Checking your browser before accessing" page and redirecting to whatever file was being accessed. There's precisely zero need to outright modify the HTML files being served to inject such checks.
• Havoc 4y ago
  Just block all of India for a week. That’ll sharpen minds a bit to track down the culprit And given that this seems like a near existential threat drastic action seems warranted.
  • quintushoratius 4y ago
    Better yet, serve an empty file to the offending user agents
    • pwdisswordfish9 4y ago
      Even better, serve a file that blocks literally everything.
• User23 4y ago
  Yet another example of how centralization sucks. Some kind of gossip protocol seems perfect for distributing lists like this.
• wernerb 4y ago
  This would be a great use case for IPFS
• bitbang 4y ago
  Serve up a text file to connections from India that just has a wildcard entry that causes everything to be blocked. Those crappy browsers will very quickly loose their popularity.
• ck2 4y ago
  Will the downloading browsers obey a 304 response? Can that be edge cached with a future GMT expire date? Alternately a 302 response to trusted local mirrors?
• password4321 4y ago
  Why would anyone host files at a provider that charges for egress? EasyList would need more than the 100mbs Scaleway offers for €2/month but Scaleway has more capable instances.
• minerva23 4y ago
  Is Google’s AMP-cache a candidate for distributing the file?
• bombcar 4y ago
  Just serve an easy list to these browsers that blocks every image and all css. Problem solved.
  • aembleton 4y ago
    How do you determine that it is these browsers requesting it and not uBO? It would be easy for the browsers to set their useragent to something like Chrome.
    • bombcar 4y ago
      The browsers don’t update - they’re already returning 100b access denied or something.
• KingOfCoders 4y ago
  Here we go, text/plain is no web content. Cloudflare defines the internet.
• wraptile 4y ago
  > Even so, we continue to serve about 100TB of “Access Denied” pages monthly! I might be ignorant of the scale of things but surely a 404 page can't weigh more than half a kilobyte right? I think EasyList maintainers have every right to break stuff in this case. Also could easily turn the list into a HTML file as it's just as parsable as txt.
• cimnine 4y ago
  Several opensource projects, especially Linux distributions, have solved this problem long ago by setting up a web of volunteer mirrors. They docs and scripts to quickly set up an additional mirror. The central HTTP server of easylist could then hand out 302s to fetch the actual file from one of the mirrors. Alternatively to 302s, a modern scriptable DNS server, that uses the mirror list, responds with different IPs, round-robin (or even better if it's geo-aware). DNS TXT records could maybe be used to serve a digest of the file, do that mirrors can't modify it without that being detected.
• vasco 4y ago
  From the perspective of someone who got their start from building small web tools and slapping ads on them and putting them online, which paid for most of my expenses to get myself through university, it's kind of funny that the people that want to use the internet "for free" at the expense of builders of websites, now also want to be able to block ads to continue to extract value out of everyone else also for free by asking Cloudflare for a free plan. Pay for your own darn servers, everyone else has to and can't even use ads to support the costs due to you.
  • MrDresden 4y ago
    There is a big difference between showing someone ads (think radio and tv ads) vs spying in on their browsing habbits (almost all internet ads). I am fine with one but not the other. Thus I block them.
• deleted 4y ago
  [deleted]
• topranks 4y ago
  Just register a new domain and serve it off that instead. If this browser is not maintained then it won’t update what it requests, but everyone else can. Obviously this becomes very whack-a-mole but in this instance it might be an easy win.
• sinuhe69 4y ago
  Download a blocking list on each start >:( That shows the discrepancy between linear program execution model (on the classic desktop) and the quasi permanently running model of modern apps. Maybe the site should identify these rowdy browsers and give them a kick.
• rgrieselhuber 4y ago
  Can’t this just be mirrored?
• Spare_account 4y ago
  This is the first Cloudflare related thread that I haven't seen eastdakota or jgrahamc get involved
  • jgrahamc 4y ago
    https://news.ycombinator.com/item?id=33272560 https://news.ycombinator.com/item?id=33272560
• self_awareness 4y ago
  Stupid question, but can't they rename the txt to .html, put <html> on the front, </html> on the back, and use cloudflare to cache/proxy the content?
  • kevincox 4y ago
    I suspect that the real problem isn't the type of content but the fact that it is being requested by robots at a high rate. If you had a text file that humans were actually browsing and reading I doubt Cloudflare would care, but humans read files at a much lower rate than is occurring for this text file.
  • kenmacd 4y ago
    Say you do that and I DDoS the easylist.html. Cloudflare will start applying their DDoS mitigation. Now everyone's app receives the Browser Integrity Check instead of the list.
• samson8989 4y ago
  Just report these bad apps to google playstore, and get them taken down for D.O.S.ing your website. Google playstore will remove them. Trust in the process.
  • stop50 4y ago
    But this won't uninstall the app from the phones
• black_puppydog 4y ago
  One lesson I guess many people (I for one) learned over the years: never ever serve anything directly on your domain, always use a subdomain. You never know which part you'll want to move off of your server...
• porbelm 4y ago
  Just blackhole every request originating from India. After a while the developers will ask why their users are mad that they get ads all the time. You then give them the finger, tell them to get their lists elsewhere, or pay for access due to their incompetency. They will just have to do the needful.
  • black_puppydog 4y ago
    Pretty simple to put 1B people into the same drawer, for a few bad apples. Think about what that would mean if the world applied that to (say) the US...
  • howinteresting 4y ago
    Haha, yeah. Just proposing to blanket punish an entire country for a few developers screwing up, and throwing a little racist joke in there at the end. Good one.
• chmod775 4y ago
  It seems their total traffic is 1000-2000TB per month and they're mostly just serving a single text file. You can serve that easily from a dozen dedicated servers for a low four figures amount. Or just get one server with a 10gbit/s connection if you don't care about being geographically close to everyone. These numbers aren't large, but many CDN companies (and AWS) would like you to believe they are and you should be paying them insane amounts of money to serve that kind of traffic.
• deleted 4y ago
  [deleted]
• squarefoot 4y ago
  This seems the perfect use case for letting a secure BitTorrent tracker share the lists, then either implementing the client in the browser, or having it as a system service that syncs the necessary files.
  • thiht 4y ago
    Torrent was also my first thought. Is there any reason why this is not a good solution?
    • kevincox 4y ago
      One downside is that Torrent doesn't really share blocks between torrents. So updating the torrent would result in a full download. That being said the data is small so it probably isn't a big deal. A better solution may be something like IPFS where with a rolling-checksum chunking algorithm you only need to download the changed parts of the list. You could even use IPNS to distribute updates for a fully decentralized setup.
  • RandomWorker 4y ago
    Yeah! Just share through BitTorrent, this is the perfect example where it can be useful. To be honest I’m not sure why this isn’t built in to browsers yet Oracle browser has one.
• dhdgrygev 4y ago
  The solution to this problem is to use a public R2 bucket with caching enabled. Block all requests to non-existent files with firewall. Now you don't even have to block those browsers from India!
• greyhair 4y ago
  So I understand some of the comments streaming down on CloudFlare, but I would like to get to another point entirely. So people writing apps using crappy coding are cratering Easylist basically through unintentional DDoS. It is the apps that suck. Full Stop. I have noted over the last five to ten years the general retrograde usability of phone apps and full desktop apps in general. Bad UIs, inconsistent behavior, performance issues on stuff that, at least on the surface, appears to be trivial. I don't understand what is causing the general slide in quality, but it is clearly visible, and it seems to me, untenable over the near term. Is it the tools? Is it the app churn pressure? I really do not know, because I work in a very different part of the industry. We have our own issues there, but that has more to do with technology churn (particularly in wireless standards) than in the tools and platforms. So what is up with the app world, in general, and the web app world in particular? I am all ears, because as I said, I don't work in that space.
• diebeforei485 4y ago
  I think GeoIP blocking is a good strategy here. Wikipedia has used this strategy and it has been reasonably successful. People can still log in to GitHub and download the list regardless of where they are located.
• JeffAllen 4y ago
  Why not just serve a special list to all android users from India that contains the top 100 most popular Indian websites (https://www.similarweb.com/top-websites/india/ https://www.similarweb.com/top-websites/india/)? Surely users will stop using that browser when they can't visit those websites anymore because those are now blocked instead of the ads.
• KETpXDDzR 4y ago
  What about some form of P2P for this?