3 ms·
That doesn’t change anything regarding the actual point of the comment.
by celdon25 4y ago
That doesn’t change anything regarding the actual point of the comment.
- astrange 4y agoYour idea for a search competitor is to ignore robots.txt?
- VWWHFSfQ 4y agoor an advertising competitor that ignores DNT! oh wait
- berkle4455 4y agoIt's a just a text file.
- chihuahua 4y agoThis seems perfectly logical to some people: Google ignores robots.txt: "Google is evil! They're trespassing on my webserver!" Google follows robots.txt: "Google search results suck! They're not indexing GitHub!"
- celdon25 4y agoYes, clearly that's the best possible interpretation of what I said. /s