1) Presuming they're using a CMS perhaps it's an accident? 2) To clarify a bit, afaik, robots.txt is a request / suggestion, a nice to have if you will. It doe

1) Presuming they're using a CMS perhaps it's an accident? 2) To clarify a bit, afaik, robots.txt is a request / suggestion, a nice to have if you will. It doesn't prevent indexing.