3 ms·
IMO, that's exactly the reason. Before search engines could scrape the data and load the content for free. Now they'll need to reach firehose data agreements.
by guiambros 11y ago
IMO, that's exactly the reason. Before search engines could scrape the data and load the content for free. Now they'll need to reach firehose data agreements.
- Paulods 11y agoTake a look at the robots file without the "www" subdomain. Its likely to prevent content duplication/push for only the non "www" url to appear on search engines. https://twitter.com/robots.txt https://twitter.com/robots.txt
- guiambros 11y agoUh... Excellent point. Seems the entire speculation is/was unfounded.