I don't have a robots.txt, so -for this url alone- I'm expecting a 404. What I've read in the google documents is that when a request to get the robots.txt retu

I don't have a robots.txt, so -for this url alone- I'm expecting a 404. What I've read in the google documents is that when a request to get the robots.txt returns a 404, it will consider it as OK to crawl.