If robots.txt allows Google and Bing but nobody else, it should be ignored. If it blocks everyone, then I agree. We need to make sure that the next Google has

If robots.txt allows Google and Bing but nobody else, it should be ignored. If it blocks everyone, then I agree. We need to make sure that the next Google has a chance to succeed.