4 ms·
Publishers Demand Accountability from Common Crawl over Unauthorized Use
- sharemywin 6mo agomaybe robot.txt should be upgraded with license specifics. not for commercial use, etc. so I believe all content is covered under fair use which to me means common crawl has a right to scrape everything and it's the user of common crawl to sort out the details.
- robtherobber 6mo agoWhen publishers complain it's never to protect the authors or journalists, but a specific (extractive) business model. Not that I side with Common Crawl on this one.