2 ms·
Not sure about legality, but one thing you could test would be if that site obeys robot tags. Add "noarchive,nosnippet" to a pages robot meta tag and see if it
by LinuxBender 6y ago
Not sure about legality, but one thing you could test would be if that site obeys robot tags. Add "noarchive,nosnippet" to a pages robot meta tag and see if it will archive it.