4 ms·
Reddit now blocks all search engines other than Google following 'misuse'
- nness 2y agorobots.txt has always been a gentleman's agreement and for such a large site to abuse it in that way is ridiculous, and possibly, anti-competitive.
loading thread…