4 ms·

I wouldn't be surprised to hear that they did put that url into their robots.txt because it is a search result page generated by a bot and that has been visited

• hurin 12y ago
  I'm not up to date on how google's current search indexing algorithm works. Supposing someone has a lot of bots that post links to `extra-small-teen-pony` or some other /sr query on nordstrom.com from other sites - would google index that?
  • cmdrfred 12y ago
    If they threw a custom 404 that reports 200. That's how I assumed those phone number lookup sites work.