4 ms·
Server load goes really high on that day, and if you get more popular, you'll need more servers and hence more money.
by rmc 15y ago
Server load goes really high on that day, and if you get more popular, you'll need more servers and hence more money.
- rb2k_ 15y agoIsn't that something that nginx/varnish should easily be able to handle? It is just a static file download after all...
- onli 15y agoEven serving a static file is a burden on the server when you have millions of requests.
- rb2k_ 15y agoWhat are the exact numbers? A quote from: http://wiki.nginx.org/Main http://wiki.nginx.org/Main : > I currently have Nginx doing reverse proxy of over tens of millions of HTTP requests per day (thats a few hundred per second) on a single server. At peak load it uses about 15MB RAM and 10% CPU on my particular configuration (FreeBSD 6).
- deleted 15y ago[deleted]
- onli 15y agohttps://easylist.adblockplus.org/blog/2011/09/01/easylist-statistics:-august-2011 https://easylist.adblockplus.org/blog/2011/09/01/easylist-st... is the first thing i found, talking about 11.5 million total users and 80% of them using easylist, 9.2 million. According to the blogpost, the still existing (i just noticed that this was after the update-behaviour change) monday-peak was 118.5% of the expected (week total / 7) - 73 million download in august, so (73 / 4 / 7) * 1.185 = 3 million for that specific list and 3.75 for all of them. Hope i didn't miscalculate ^^ Add to that the growth till than. The easylist.txt seems to have a size of 528kb.
- j_s 15y ago+ SSL
- ComputerGuru 15y agoCPU and bandwidth are entirely different issues. Sure, nginx can handle the processing. But do you have the piping to match? A run of the mill dedicated server has a 100mbit uplink. Do the math. (Hint: it's easy to saturate in no time).
- prostoalex 15y agoHas anybody tried https://developers.google.com/speed/pagespeed/service https://developers.google.com/speed/pagespeed/service for this?
- oconnor0 15y agoThis is just downloading a single static text file so there's nothing to optimize.
- kijin 15y agoUse a CDN? It's not like the list is tailored to anybody.
- tantalor 15y agoMaybe it wasn't clear, but in this case the load is still distributed evenly across all days. I always download on Monday, you always download on Tuesday. The author suggests that this is not desirable. FTA, The “Total” row looks very nice, it’s exactly what we would like to have. However, our download interval matches the number of days now, with the effect that people always download on the same weekday (and those currently downloading on a Monday will continue downloading on a Monday).