4 ms·Yes, I feel like this person only read the title and not the text of the post and made an assumptionby circuit10 3y agoYes, I feel like this person only read the title and not the text of the post and made an assumptionbenatkin 3y agoNot an assumption. Just a response to the title of the article.circuit10 3y agoThey said "Using robots.txt as a model for anything doesn't work." but it does work for the case described in the textbenatkin 3y agoWhat works? This is an idea, not anything that's been throughly tried.