3 ms·
shrug Things like /search?q= are in my robots.txt because crawling those pages is just a waste of everyone's time and resources.
by eli 16y ago
shrug
Things like /search?q= are in my robots.txt because crawling those pages is just a waste of everyone's time and resources.