3 ms·
Actually i believe its only from the "www" subdomain. Take a look at the robots without the "www" https://www.twitter.com/robots.txt https://www.twitter.com/ro
by Paulods 11y ago
Actually i believe its only from the "www" subdomain. Take a look at the robots without the "www"
https://www.twitter.com/robots.txt https://www.twitter.com/robots.txt
https://twitter.com/robots.txt https://twitter.com/robots.txt
This is likely just to prevent content duplication/nudge users to visit without the "www".
- thenextcorner 11y agoyep makes a huge difference... as explained here as well http://webmarketingschool.com/no-twitter-did-not-just-de-index/ http://webmarketingschool.com/no-twitter-did-not-just-de-ind...
- sp332 11y agoYou should never use Google's estimate as a real estimate, especially once it gets past 100. There are better ways to move content, like 301 redirects or rel=canonical.