3 ms·

Kieran! Kieran (OP) is a friend of mine, highly competent, and I highly recommend his comprehensive legal guide to web scraping: https://mccarthygarberlaw.com/a

• divbzero 5y ago
  Thank you sharing the link. I find it amusingly appropriate that viewing this legal guide requires agreeing to Terms & Conditions, but appreciate that it’s one of the shortest Terms & Conditions I’ve ever seen easy to read and understand within seconds.
  • KieranMac 5y ago
    Haha! Given that the article is about scraping, I figured some prudence with respect to people copying and pasting my work somewhere else was warranted!
• KieranMac 5y ago
  Thanks, Tucker!
• ape4 5y ago
  Just for fun, here is the robots.txt from that site: # Default robots file version:2 User-agent: \* Disallow: /calendar/action\* Disallow: /events/action\* Allow: /*.css Allow: /*.js Disallow: /\*? Crawl-delay: 3 Sitemap: https://mccarthygarberlaw.com/sitemap_index.xml
• sneak 5y ago
  [flagged]