It is an asocial state of mind. We have locks and security systems that prevent people from stealing. But if all people agreed to not steal, then we could save

It is an asocial state of mind. We have locks and security systems that prevent people from stealing. But if all people agreed to not steal, then we could save that efforts for something better. The ideal approach doesn't work with the stealing, and now it doesn't work with HTTP either. It just raises costs for a society with no lasting benefit for anyone: site owners just figure out ways to restrict access and no more scraping of pages that they do not want to be scraped. A healthy society relies on a cooperation between members. It relies on them accepting some rules that limits their behavior. Like we agreed not to kill others, and now I can go outside without weapons and anti-bullet defenses.