Legality isn't the question here. If you want to speak to the legality, anyone circumventing a robots txt that explicitly has your bot's user-agent and 'disallo

Legality isn't the question here. If you want to speak to the legality, anyone circumventing a robots txt that explicitly has your bot's user-agent and 'disallow: *' is unauthorized access (I imagine it's more nuanced for 'user-agent: *'). No website is required to allow anyone to visit and can discriminate against any client or software any way they want.