2 ms·

If you can identify the scraper you should have a valid legal case to recover damages.

• thisislife2 6mo ago
  Only if they had a robots.txt for their site.
  • razingeden 6mo ago
    I hadn’t even considered that. Don’t know why that comment is greyed out or downvoted. It’s a static site that hasn’t been updated since 2016—- so it’s .. since been moved to cloudflare r2 where it’s getting a $0.00 bill, and it now has a disallow / directive. I’m not sure if it’s being obeyed because the cf dash still says it’s getting 700-1300 hits a day even with all the anti bot, “cf managed robots” stuff for ai crawlers in there. The content is so dry and irrelevant I just can’t even fathom 1/100th of that being legitimate human interest but I thought these things just vacuumed up and stole everyone’s content instead of nailing their pages constantly?
  • gzread 6mo ago
    No, it's still illegal to DDoS sites that don't have robots.txt.
    • thisislife2 6mo ago
      You are right, I hadn't considered that aspect.