4 ms·

can this been seen as a initiative to make google robots.txt parser the internet standard? every webmaster will want to be compliant with google corner cases...

• H8crilA 7y ago
  That's probably the hidden agenda.
  • djsavvy 7y ago
    It's not even hidden, Google explicitly says that in the blog post.