3 ms·

Are there legal obligations to adhere to robots.txt?