3 ms·
It’s within anyone’s rights as website maintainers to block malicious IP addresses that scrape or otherwise within their discretion. Nobody is legally forcing
by throwwebmaster 5y ago
It’s within anyone’s rights as website maintainers to block malicious IP addresses that scrape or otherwise within their discretion.
Nobody is legally forcing websites to allow access to everyone, and accordingly, nobody is altering the law by blocking access to people (crawlers, hackers, spammers, malcontents, or anybody really) that they feel are not welcome. So exercising one’s existing rights isn’t an act of making or altering laws.
I suggest reading up on what robots.txt is to further understand this.
- ClumsyPilot 5y agoHacking and other malicious behavious are actually illegal. Either Crawling does not belong on that list, or google exects should be in jail. Given that crawling is not malicious, what we are discussing now is 'someone is crawling my website in a way I dont like' which is a different gripe. It mighthave some merit, but robots txt is not legally binding.
- throwwebmaster 5y agoAs I stated before, Illegal or not, it is within the website owners’ rights to restrict access.