4 ms·
A good example of why this won’t work that’s 10x more extreme and has legal consequences… in theory… Didn’t the FBI admit that it just bought U.S. citizen data
by s4mw1se 3y ago
A good example of why this won’t work that’s 10x more extreme and has legal consequences… in theory…
Didn’t the FBI admit that it just bought U.S. citizen data instead of spying on them? Almost two years in a row?
This has been happening non-stop for over 20 years without any repercussions.
I don’t think robots.txt will make a difference. If your site is internet facing, nothing will prevent it from being crawled and scraped.
If anything, i can almost guarantee AI crawling will be tired to SEO if it’s not already.