I knew there was a reason why I used client certificates and alternate ports. Why is it so difficult to just respect robots.txt? Maybe there's an idea for a br

I knew there was a reason why I used client certificates and alternate ports. Why is it so difficult to just respect robots.txt? Maybe there's an idea for a browser plugin that determines if you can easily scrape the data or not. If not, then the website is blocked and then traffic will drop. I know this is a naive idea...