3 ms·
Will this break Google Reader and other RSS readers from legitimately using the RSS feed here? After all, Google Reader uses a bot to read the feed and allows u
by petercooper 17y ago
Will this break Google Reader and other RSS readers from legitimately using the RSS feed here? After all, Google Reader uses a bot to read the feed and allows us to search it from within their app.. much like their search engine does with regular pages :-) (That is, "web robots" aren't just spiders.)
Also things like http://hacker-newspaper.gilesb.com/ http://hacker-newspaper.gilesb.com/ and http://hnsort.com/ http://hnsort.com/ become less legitimate due to this. If the reason is to reduce the SEO benefits of getting a link on HN, just "nofollow" everything instead..
(Update: Googling on this topic brought up a page of my own where a Google Reader engineer explained how Google Reader deals with robots.txt - http://www.petercooper.co.uk/google-reader-ignores-robottxt-rules-51.html http://www.petercooper.co.uk/google-reader-ignores-robottxt-... - though their definition of Web robot is far from universal)