robots.txt is just a guideline between well-meaning actors for the majority of their traffic, like helping a bot not waste its time nor your bandwidth by crawli

robots.txt is just a guideline between well-meaning actors for the majority of their traffic, like helping a bot not waste its time nor your bandwidth by crawling dynamically-generated, endless-scrolling /calendar.php pages. Google does use it to that extent. It's not a firewall. Seems like you're describing cloaking (https://developers.google.com/search/docs/advanced/guidelines/cloaking https://developers.google.com/search/docs/advanced/guideline...), one of the oldest SEO tricks, and you can imagine that search engines started defeating it on Day 2 of crawling the web.