Years ago i added a url to not be crawled nor indexed to a web app I worked on to block bad crawlers, so I disallowed crawling through robots.txt and added a hi

Years ago i added a url to not be crawled nor indexed to a web app I worked on to block bad crawlers, so I disallowed crawling through robots.txt and added a hidden link in the footer with the "noindex" attribute. Google, Bing, Archive.org, and people normally browsing were just fine while bad crawlers got a blank page.