That's okay. You and the three people on this planet who actually use a robots.txt because they don't want to be crawled are not the ones they care about. The 9

That's okay. You and the three people on this planet who actually use a robots.txt because they don't want to be crawled are not the ones they care about. The 99% that just blindly copied some tutorial which said something bright like "only allow Googles bot" without thinking about it are.