3 ms·

Related: ROBOTS.TXT is a suicide note - https://news.ycombinator.com/item?id=13376870 https://news.ycombinator.com/item?id=13376870 - Jan 2017 (30 comments) R

by dang 1y ago

Related: ROBOTS.TXT is a suicide note - https://news.ycombinator.com/item?id=13376870 https://news.ycombinator.com/item?id=13376870 - Jan 2017 (30 comments) Robots.txt is a suicide note - https://news.ycombinator.com/item?id=2531219 https://news.ycombinator.com/item?id=2531219 - May 2011 (91 comments)