4 ms·
But it doesn't make it available with "no conditions at all", their robots.txt implies the condition that you aren't scraping it with a bot.
by verall 9y ago
But it doesn't make it available with "no conditions at all", their robots.txt implies the condition that you aren't scraping it with a bot.
- oh_sigh 9y agoSo there would be a material difference between me using the data when a bot scrapes it, versus a human doing it?