I agree. I used to have a website serving some code and some tarballs of my software. I used to be able to handle the traffic (including from ALL Linux distribu

I agree. I used to have a website serving some code and some tarballs of my software. I used to be able to handle the traffic (including from ALL Linux distributions, who are packaging this software) from a home server and home connection, over for the 30+ years I've been serving it. In the last few months, there's so much crawler traffic (specially going over all the source files over and over), ignoring crawl-delay and the entirety of robots.txt , that they have brought the server down more than once.