Search engines aren't a public service. The article half-way acknowledges the robots.txt solution, and quickly dismisses it as unworkable. It's clear, however,

Search engines aren't a public service. The article half-way acknowledges the robots.txt solution, and quickly dismisses it as unworkable. It's clear, however, that publishers have a recourse if they feel the search engines are doing them more harm than good. If they want finer granularity than is currently available, they need to talk to Google, not the government.