"Our goal here is to be able to distinguish between the good type and bad type of scraping and give webmasters full transparency. Obviously this is a hard probl

"Our goal here is to be able to distinguish between the good type and bad type of scraping and give webmasters full transparency. Obviously this is a hard problem. If you have any feedback on any of this we'd love to hear it." Yes as said before plus: - Obey robots.txt to the full extend - Name your access, i.e. label your bot - Don't use shady tactics such as IP rotation - Provide web site owners the option to fully block access of your bots (yes, communicate your full IP ranges) Again - this is from a content owner who paid for his content.