10 ms·

Some parameter are useful, but they are not part of the standard. e.g. Crawl-Delay, prevent DDOS from YAHOO! Slurp

• soult 15y ago
  Is Yahoo! Slurp still active? I thought they turned it off when they switched to using Bing search results on Yahoo!
  • tszming 15y ago
    I don't think so. I still receive a lot of requests from Yahoo! Slurp / Yahoo! Slurp China this month.
  • yaix 15y ago
    I saw a few hits from Slurp with Y!/USA IP just 2 days ago in my logs.