3 ms·
(and continuing to crawl a website that specifically forbids it through the standard robots.txt is a bit icky). I find the idea you, in control of a server, ca
by Causality1 5y ago
(and continuing to crawl a website that specifically forbids it through the standard robots.txt is a bit icky).
I find the idea you, in control of a server, can send someone data and then demand they erase it a bit ridiculous. If you want to control distribution then control it. Require a login to access any data you don't intend to be public.