For anyone who wonders, the summary of the mistake: created a web crawler without any stop condition and without adding any checks for not visiting the same pag

For anyone who wonders, the summary of the mistake: created a web crawler without any stop condition and without adding any checks for not visiting the same page twice. The crawler just kept running for a day, making around 9 million requests per minute.