Their IP range can be blocked. Google has known IP ranges and if the website admin wants can allow them based on that rather than robots.txt, and that would rea

Their IP range can be blocked. Google has known IP ranges and if the website admin wants can allow them based on that rather than robots.txt, and that would really mess up everyone else.