Today I learnt OogleBot is ignoring the robots.txt file on my private Gitea server: git[dot]habd.as/robots.txt A quick search for "after dark habdas" reveals

Today I learnt OogleBot is ignoring the robots.txt file on my private Gitea server: git[dot]habd.as/robots.txt A quick search for "after dark habdas" reveals the second of two rules in the robots file, which explicitly blocks Oogle at the site root, is clearly being ignored. Of course I created this rule before putting this server online and Andex is the only webmaster console I registered this site with. But that's besides the point. There's obviously a lack of respect for the robots file or a gross error in OogleBot as far as I can tell. So now you know.