your assumptions: - the "intention" of the robots.txt standard is as you state - the url is included in the information not allowed by that standard - if

your assumptions: - the "intention" of the robots.txt standard is as you state - the url is included in the information not allowed by that standard - if the url is not included it should be because of the "intention" - toolbars are subject to the same standards I'm not disagreeing with you as much as just pointing out that I don't think EVERYONE agrees on these standards.