3 ms·
There's a reddit thread from 9 years ago with people complaining the site blocking crawlers and thus not being indexed by search engines. So I'm not sure what t
by pmdr 2mo ago
There's a reddit thread from 9 years ago with people complaining the site blocking crawlers and thus not being indexed by search engines. So I'm not sure what they consider "overload." At this point they should dump the db on thepiratebay.
- progval 2mo agoDumping the database won't stop the scrappers because humans running the scrappers are not even aware Gentoo exists. They're just scrapping every website their bots find.