6 ms·
if you do data mining in the EU you are legally required to respect robots.txt afaik
by FinnKuhn 3y ago
if you do data mining in the EU you are legally required to respect robots.txt afaik
- LawTalkingGuy 3y agoThis is how it is a failed system. It doesn't really do what most people who have one think it does and yet it gets rolled into law and everyone now has to deal with it constantly while still not fixing the original issue. It's like the EU doesn't understand that bad law has a negative value.