When Yahoo! Pipes was still running (long time ago), their official position was: > Because Pipes is not a web crawler (the service only retrieves URLs when re

When Yahoo! Pipes was still running (long time ago), their official position was: > Because Pipes is not a web crawler (the service only retrieves URLs when requested to by a Pipe author or user) Pipes does not follow the robots exclusion protocol, and won't check your robots.txt file.