Can't they do a robots.txt or some other way of stopping Google from crawling their site? Or, do they want to let it happen so they can whine about it? As oth

Can't they do a robots.txt or some other way of stopping Google from crawling their site? Or, do they want to let it happen so they can whine about it? As others have pointed out, there's multiple sources of news (especially those in other countries).