3 ms·
FWIW, bingbot basically DDOS'ed our site a few months ago by crawling links that were labeled nofollow/noindex in the hrefs. Adding a rule to robots.txt fixed
by kgo 15y ago
FWIW, bingbot basically DDOS'ed our site a few months ago by crawling links that were labeled nofollow/noindex in the hrefs. Adding a rule to robots.txt fixed the problem.