Here's a video that explains how and why we handle robots.txt that way: http://www.mattcutts.com/blog/robots-txt-remove-url/ http://www.mattcutts.com/blog/robot

Here's a video that explains how and why we handle robots.txt that way: http://www.mattcutts.com/blog/robots-txt-remove-url/ http://www.mattcutts.com/blog/robots-txt-remove-url/