4 ms·
https://news.ycombinator.com/item?id=46681454 https://news.ycombinator.com/item?id=46681454 I think this is a recent change.
by mystraline 9mo ago
https://news.ycombinator.com/item?id=46681454 https://news.ycombinator.com/item?id=46681454
I think this is a recent change.
- daveoc64 9mo agoAll the comments there seem to suggest that there has been no change and that robots.txt isn't required.