3 ms·
It's seems that Facebook uses robots.txt to block this pages https://www.facebook.com/robots.txt https://www.facebook.com/robots.txt But, depending of the amo
by nico-roddz 14y ago
It's seems that Facebook uses robots.txt to block this pages
https://www.facebook.com/robots.txt https://www.facebook.com/robots.txt
But, depending of the amount of inbound links, Google will index the urls anyway.
It's a common issue.
- cleverjake 14y agoGoogle ignores robots.txt if the number of inbound links is > N? Also - any speculation as to how so many sites were lining to peoples login pages?