3 ms·
Any newspaper that has a paywall and doesn't want their articles on Google News, can trivially prevent that from happening with a well-known two-line robots.txt
by ploxiln 12y ago
Any newspaper that has a paywall and doesn't want their articles on Google News, can trivially prevent that from happening with a well-known two-line robots.txt.
Google News doesn't even republish full text. Just intro paragraph.
But newspaper sites seem to not be willing to pull their articles from Google News. They seem to be afraid that if they do, they'll lose all their traffic to the newspaper sites that are still in Google News. I don't know one way or the other., but I don't care that much.