Gold chains etc should be behind authentication. Robots txt is more like a warning sign that says the hedgemaze in the garden goes on forever, so probably stay

Gold chains etc should be behind authentication. Robots txt is more like a warning sign that says the hedgemaze in the garden goes on forever, so probably stay out of it.