3 ms·
The suicide of the data listed in robots.txt? How? The whole point of the article is they ignore what you have written in your robots.txt, so they'll archive it
by QuercusMax 1y ago
The suicide of the data listed in robots.txt? How? The whole point of the article is they ignore what you have written in your robots.txt, so they'll archive it regardless of what you say.
- stickfigure 1y agoCorrect, they are challenging your written wish for data-suicide.