3 ms·

I know about that because i use robots.txt on my personal website to exclude, but how do you automatically exclude links that were already archived?